# Avoiding the Top 10 Mistakes in Automated Trading
We've made most of these mistakes. Some of them we've made twice. Here's the list so you can skip the expensive parts.
1. Overfitting to Historical Data
The mistake: You optimize the model on 5 years of data and get a perfect equity curve. Sharpe 3.0. Max drawdown 3%. Win rate 62%. You think you have a working system. You don't. You have a system that's perfectly tuned to the specific noise of those 5 years.
The reality: Every parameter you tweak creates an optimization surface. The more parameters, the more peaks. Your "optimal" parameter set is the highest peak in a 50-dimensional space of noise. It won't work on new data.
How to avoid it:
- Use walk-forward analysis. Always.
- Keep models simpler than you think you need. Fewer features, fewer parameters.
- Test on out-of-sample periods aggressively.
- If the model's performance degrades by more than 30% OOS, you're overfit.
- Never optimize more than 3-4 parameters at once.
2. Ignoring Transaction Costs
The mistake: Backtesting with $0.00 commission and perfect fills. Or using assumed costs that are lower than reality.
The reality: For ES:
- Commission: $2.50 per side per contract ($5 round trip)
- Slippage: 1 tick ($12.50) minimum on market orders, 2+ ticks in volatility
- Exchange fees: ~$0.10 per contract at the CME
- Platform fees: NinjaTrader charges for data and the platform itself
On a 2-contract ES trade with a 2-tick target:
- Gross profit: $50.00
- Commission: $10.00 (round trip × 2 contracts)
- Slippage: $25.00 (1 tick market entry × 2 contracts)
- Net profit: $15.00
Your target was $50. You keep $15. Transaction costs ate 70% of the trade.
How to avoid it:
- Model actual commission in the backtest
- Add minimum 0.5 tick slippage to every simulation fill
- Test on threshold edges — if the model only wins when you assume perfect fills, it's not a real edge
- Know your breakeven costs before you trade
3. Survivorship Bias
The mistake: Using current index constituents to backtest. You test on the S&P 500 as it exists today, not as it existed 5 years ago.
The reality: Companies get removed from the S&P 500. They go bankrupt, get delisted, or get acquired. A backtest on current constituents only includes the survivors. The losers are missing. Your backtest makes the market look less risky than it actually was.
For ES specifically: ES is an index, not individual stocks. Survivorship bias is less severe because ES tracks the index level, not individual components. But it's not zero — the index rebalancing process smooths out the worst failures.
How to avoid it:
- Use total-return index data (includes dividend adjustments)
- Be aware that ES index data includes the rebalancing mechanism, which hides the worst individual stock collapses
- The bigger risk for ES is regime bias: backtesting only on bull markets
4. Regime Bias in Backtesting
The mistake: Backtesting on 2 years of data that was all one market regime.
The reality: If you tested on 2020-2021 (COVID recovery, massive liquidity), your model learned to buy dips and hold. It will fail in 2022 (rising rates, no dip-buying). If you tested only on 2022 (bear market), your model is always short. It will fail in 2023.
How to avoid it:
- Backtest across at least 3-5 years covering different market conditions
- Look for: low volatility periods, high volatility periods, bull, bear, and range-bound regimes
- Test the model separately on each regime
- If the model only works in one regime, don't deploy it. Wait for the right conditions.
5. Trading Through the Model's Weaknesses
The mistake: The model has low conviction on a trade (45% probability), but you take it anyway because "the model sees something."
The reality: The model's confidence scores are calibrated. When it says 45%, it wins 45% of the time. Taking low-confidence trades turns a selective strategy into a noise-trader strategy.
How to avoid it:
- Set a minimum confidence threshold
- Backtest how the strategy performs at each confidence level
- Only trade the confidence bands where the model has a demonstrated edge
- If your model has 55% accuracy on all trades but 70% on the top 20% confidence trades, the top 20% is your actual strategy
6. Changing the Strategy During Drawdown
The mistake: The model hits a losing streak. You panic. You change the stop distance, the confidence threshold, the features, or the model itself. All while the drawdown is running.
The reality: Drawdowns are normal. A 50% win rate strategy with 2:1 R:R has a 3% chance of 5 consecutive losses. That happens once every 33 sequences of 5 trades (about once a month for a model that trades daily).
If you change the strategy during drawdown, you're making decisions based on recent results, not on the overall edge. You will:
- Turn a normal drawdown into a strategy change
- Introduce new errors
- Never give the original strategy time to recover
- Repeat this cycle indefinitely
How to avoid it:
- Pre-define your drawdown response: "After 10% drawdown, reduce size by 50% for 20 trades. Do not change the strategy."
- Only review strategy changes after X trades or Y days, not after Z losses
- Keep a journal of every change and why you made it
- If you can't follow this, you can't trade a system
7. Poor Data Quality
The mistake: Using free or low-quality market data for training and backtesting.
The reality:
- Free data has gaps, incorrect ticks, and missing volume
- Different brokers have different data quality (IBKR vs Rithmic vs CQG)
- NinjaTrader's historical data has known issues with certain time periods
- Continuously-adjusted contract data introduces artifacts at roll dates
Bad data = bad model. The model learns patterns in the errors, not in the market.
How to avoid it:
- Use reputable data sources (CME, IQFeed, Polygon, or broker-direct)
- Clean the data before training (remove zero-volume bars, outlier ticks)
- Validate data quality (check for: gaps, duplicate bars, volume anomalies)
- Test on multiple data sources to verify the patterns exist in all of them
8. Ignoring Order Book Dynamics
The mistake: Simulating fills that wouldn't happen in reality.
The reality: When your model signals a buy, it doesn't exist in a vacuum. Other traders are also buying. The order book absorbs your order. In illiquid conditions, your fill is worse than the simulation. In fast markets, your limit order sits unfilled while price moves.
How to avoid it:
- Model fills conservatively (assume 1 tick worse on market orders, 50% fill rate on limit orders)
- Test in market replay (historic tick data) before live
- Account for spread widening during the model's typical trading hours
- Don't assume you get filled at the exact signal price
9. Running Without a Watchdog
The mistake: The model trades autonomously with no monitoring. It's running on a VPS. You check it once a day.
The reality:
- The VPS can go down
- The data feed can disconnect
- NinjaTrader can crash
- The model can produce corrupt signals
- A single bad signal can cause significant losses if uncorrected
How to avoid it:
- Watchdog process monitors the model and platform (heartbeat every 5 seconds)
- Auto-restart on crash
- Flat-all-positions on extended absence of heartbeat
- Email/SMS alerts for connection loss or position mismatch
- Daily reconciliation: compare model's recorded positions with broker's actual positions
Our Watchdog setup: If the API doesn't respond for 10 seconds, flat all positions. If the VPS goes dark for 30 seconds, flat all positions and alert via Discord. If a position mismatch exceeds 1 contract, halt all trading.
10. Emotional Discretionary Overrides
The mistake: The model says no trade. You see a setup. You take it anyway because it "looks good."
The reality: This is the #1 cause of failure in systematic trading. The system was built to remove emotion. Every time you override it, you re-introduce emotion. You might win 2-3 discretionary trades in a row, which reinforces the behavior. Then you take a 10R loss that wipes out a week of systematic gains.
How to avoid it:
- No discretionary trades. Period.
- If you override the model three times in a month, shut down the system and reassess.
- The model is the trader. You are the operator. Operators don't make trading decisions.
- If you can't trust the model, you need a better model or more data. Not more discretion.
- Trade small enough that you can let the model run without intervention.
The Golden Rule
If you've made 5 of these 10 mistakes, you're in good company. We've made all of them. The key is recognizing them before they cost you money.
Build your system to protect against your own weaknesses. The market will take advantage of your mistakes if you let it. Don't let it.