The Experiment
Everyone claims their AI trading bot beats the market. We decided to test it. We funded five accounts with $10,000 each and let the most popular AI trading platforms run for 90 days without intervention. Here's exactly what happened.
The Contenders
Bot A — Trade Ideas (Holly AI): The veteran. Holly scans the entire market daily and presents statistically-backed trade ideas. It's been around since 2003 and has a legitimate track record.
Bot B — Tickeron AI: Pattern recognition focused. Uses neural networks to identify chart patterns and predict breakouts. Heavy on technical analysis signals.
Bot C — Composer: No-code strategy builder. We used their top-performing community strategy — a momentum-based approach that rotates between sectors and bonds.
Bot D — Alpaca + Custom GPT Strategy: We built a custom strategy using Alpaca's API and GPT-4 for signal generation. The AI reads news, earnings, and technicals to make decisions.
Bot E — Kavout: Machine learning platform that assigns "K Scores" to stocks predicting 1-month forward returns. We bought the top 10 K Score stocks each month.
90-Day Results
| Bot | Starting | Ending | Return | Max Drawdown | Win Rate |
|---|---|---|---|---|---|
| Trade Ideas (Holly) | $10,000 | $11,340 | +13.4% | -6.2% | 62% |
| Tickeron | $10,000 | $10,780 | +7.8% | -8.1% | 55% |
| Composer | $10,000 | $10,920 | +9.2% | -4.3% | N/A (monthly) |
| Alpaca + GPT | $10,000 | $9,650 | -3.5% | -12.7% | 48% |
| Kavout | $10,000 | $10,450 | +4.5% | -7.8% | N/A (monthly) |
| S&P 500 | — | — | +5.1% | -5.5% | — |
Key Takeaways
1. The expensive, established platforms outperformed. Trade Ideas Holly and Composer — both with years of backtesting and refinement — delivered real alpha. The DIY GPT approach lost money.
2. AI doesn't eliminate drawdowns. Every bot had significant drawdowns. The Alpaca + GPT bot's 12.7% drawdown would have been psychologically devastating for most traders. Position sizing and risk management still matter more than signal quality.
3. The best AI trading tool is one that keeps you disciplined. The biggest advantage wasn't prediction accuracy — it was removing emotional decisions. No panic selling, no FOMO buying, no revenge trading after losses.
4. Garbage in, garbage out. Our custom GPT strategy underperformed because natural language models aren't optimized for quantitative trading. They're great at understanding context but terrible at predicting price movements. Purpose-built ML models (Holly, Kavout) significantly outperform general-purpose LLMs for trading.
Should You Use an AI Trading Bot?
If you're a discretionary trader who struggles with discipline, an AI assistant that generates signals and forces systematic execution can genuinely help. But don't expect a money printer. The best realistic outcome is slight edge + consistency + compounding over time.
If you're looking for "set it and forget it" passive income, you're better off with index funds. Every AI trading platform we tested required monitoring, parameter adjustments, and human judgment calls about when to override or pause the system.
