SEC rules require us to show performance inside a logged-in account.
Every strategy runs live. Inside, you see every holding, every trade, and every move to cash.
Before your AI agent trades real money, test its rules twice: once on past data, then live with virtual money. Here is how to do both, and the 3 traps that make tests look better than they are.

Before your AI agent trades real money, test its rules twice: once on past data, then live with virtual money. This guide explains both steps and the 3 traps that make tests look better than they are.
Testing tells you whether your rules do what you think before real money is at risk. An AI agent follows rules exactly. If the rules are flawed, the agent repeats the flaw on every run.
Testing cannot prove a rule will make money. It can catch rules that are broken, unclear, or built on luck.
A backtest runs your rules over past prices to see how they would have behaved. Many charting and strategy tools can run one without code.
1. Write your rules in plain words: what to buy, how much, and when to change.
2. Set aside the most recent part of your data. Do not look at it yet.
3. Run the rules on the older data.
4. Include trading costs and realistic order prices.
5. Look at how the rules behaved, including their worst stretches.
6. Run the rules once on the data you set aside.
That last step matters most. Data the rules never saw during building is a fairer test than data you tuned them on.
A live test, also called paper trading or forward testing, runs your rules on today's prices with virtual money. Nobody knows tomorrow's prices, so this test cannot be fooled by hindsight.
It also shows practical problems a backtest misses, such as orders that do not fill or data that arrives late. Run it long enough to see calm and rough markets.
One limit remains. A 2025 study by Sui and Wang found the same investors showed stronger biases with real money than in simulated accounts. So start small when you switch to real money.
Three traps make test results look better than real life. Each one is easy to fall into and hard to spot afterward.
Lookahead bias means your test used information that was not available at the time. For example, a rule might use a day's closing price to decide a trade made that morning.
Another form is survivorship bias. Testing only on companies that still exist today leaves out the ones that failed, which makes the past look kinder than it was.
Overfitting means tuning rules so closely to past data that they fail on new data. If you try enough versions, one will look great by chance.
It happens faster than most people expect. The American Mathematical Society shared research showing that with 5 years of data, as few as 45 versions can produce one that looks strong by luck.
A backtest that assumes free, instant trades at the printed price will look too good. Real trades pay fees, spreads, and slippage. Rules that trade often suffer most.
A believable result holds up after you remove every shortcut. Ask these questions about any test, yours or anyone else's.
Researchers apply the same caution to their own field. Harvey, Liu, and Zhu (2016) argue that most claimed findings in financial economics are likely false and need a higher bar of proof.
For more on why tested bots still fail, read do AI trading bots actually work?
1. Freeze the rules. Do not keep tweaking them after each week.
2. Give your agent the exact written rules.
3. Start with a small amount of real money.
4. Keep trade approvals on at first.
5. Compare live behavior to the test, and retire the rules if they break.
Rules that keep changing are a sign of overfitting. See why the best trading models refuse to learn and judge the behavior, not the returns.
Portfolio Lab is a strategy research and simulation tool from alphaAI Capital. It lets you build investment strategies and test them in live markets with virtual money. You can connect it to AI apps like Claude or ChatGPT, so your agent has a strategy's plan to follow.
Backtesting runs a set of trading rules over past prices to see how they would have behaved. It shows the past, not the future.
No. Follow it with a live test using virtual money, then start with a small real amount.
It is when a test uses information that was not available at the time of the trade, which makes results look better than they could be.
Overfitting is tuning rules so closely to past data that they fail on new data. Testing many versions makes it more likely.
It means testing rules on data you kept hidden while building them. It is a fairer check than the data you tuned on.
There is no set rule. Run it long enough to see both calm and rough markets.
AI apps can help write and run tests. You still need to check for lookahead bias, overfitting, and missing costs.
Build, test, and deploy your own strategies in Portfolio Lab. Free to start.
Educational & Research Disclosure: The content provided is for informational and educational purposes only and is not intended to constitute investment advice, a recommendation, solicitation, or offer to buy or sell any security. Any discussion of market trends, historical performance, academic research, models, examples, or illustrations is presented solely to explain general financial concepts and does not represent a prediction, guarantee, or assurance of future results. Past performance is not indicative of future results. All investing involves risk, including the possible loss of principal.