Backtesting Pitfalls & Limitations
No backtest is perfect. This page explains common issues that affect all historical simulations and how our strategies address (or fail to address) each one. Read this before making any investment decisions based on backtest results.
Important Risk Disclosures
Backtests are not predictions.
Past performance does not guarantee future results. Historical simulations cannot account for future market conditions, regulatory changes, or black swan events.
This is not financial advice.
The strategies shown are for educational and research purposes only. Do not invest money you cannot afford to lose based on backtest results.
Data limitations exist.
Point-in-time fundamental data may contain errors, gaps, or survivorship bias. Small discrepancies can compound over 20-year simulations.
Execution differs from simulation.
Real trading involves slippage, market impact, liquidity constraints, and psychological factors not captured in backtests.
Strategy degradation is common.
Strategies that worked historically often lose effectiveness as they become crowded or market conditions change.
DO NOT blindly follow any strategy shown here.
Use these backtests as one input among many in your research. Consult a financial advisor before making investment decisions.
Why Understanding Pitfalls Matters
Backtests are hypothetical simulations, not real trading results. They show what would have happened if you had perfectly executed a strategy in the past with no emotional interference, perfect timing, and complete information access.
Real investing is messier. You may hesitate during crashes. You may not have access to the exact data at the exact time. Transaction costs may be higher. Companies may fail in ways not captured by the simulation.
The strategies displayed on this site - including the Dhando scores on the Superinvestor page - are validated using these backtests. If the backtests have flaws, the live scores may be overly optimistic.
Quick Reference: How Strategies Address Each Pitfall
| Pitfall | Pure Dhando | SI Clone | Dhando + SI | Simple Value |
|---|---|---|---|---|
| Survivorship Bias | Partial | Partial | Partial | Partial |
| Look-Ahead Bias | Addressed | Partial | Addressed | Addressed |
| Data Snooping | Partial | No | Partial | Addressed |
| Transaction Costs | Partial | Partial | Partial | Partial |
| Regime Change | No | No | No | No |
| Unrealistic Rebalancing | No | No | No | No |
Addressed = actively mitigated, Partial = partially addressed, No = not addressed (inherent limitation)
Backtesting Pitfalls Summary
Every backtest has limitations. Understanding these pitfalls helps you interpret results more accurately and avoid overconfidence in any strategy.
Survivorship Bias
High ImpactBacktests often only include companies that still exist today, excluding bankrupt or delisted companies. This makes historical returns look better than they actually were.
Look-Ahead Bias
High ImpactUsing information that wasn't available at the time of the simulated trade. Example: using Q4 earnings data on Jan 1 when they weren't reported until Feb 15.
Data Snooping / Overfitting
High ImpactTesting many strategies on the same historical data until finding one that "works." The more parameters tuned, the more likely the results are due to chance rather than genuine edge.
Transaction Costs & Slippage
Medium ImpactBacktests often underestimate the true cost of trading. Real trades have bid-ask spreads, market impact, and timing delays that erode returns.
Regime Change / Non-Stationarity
High ImpactMarket conditions change. What worked from 2005-2020 may not work from 2025-2040. Interest rates, technology, regulations, and market structure all evolve unpredictably.
Unrealistic Rebalancing
Medium ImpactBacktests assume you can rebalance exactly on schedule with perfect execution. Real investors face emotional biases, time constraints, and market closures.
Liquidity Constraints
Medium ImpactSmall-cap or thinly traded stocks may be difficult to buy/sell at backtest prices. Large orders can move the market.
Benchmark Selection Bias
Low ImpactComparing to an easy-to-beat benchmark can make performance look better. S&P 500 may not be the appropriate benchmark for small-cap value strategies.
The Bottom Line
No backtest is perfect. The strategies shown here have known limitations that may cause live performance to differ significantly from historical simulations. Use these results as one input among many in your investment research. Never invest based solely on backtest results.
What This Means for Live Scores
The stock scores displayed on the Superinvestor page and Stock Screener use the same scoring logic as the backtests. This means:
- Scores may be overly optimistic if survivorship bias inflated historical returns.
- High-scoring stocks today were identified by a formula that worked in the past - it may not work as well in the future.
- The scoring weights (40% downside, 30% upside, etc.) may have been influenced by historical data patterns that won't repeat.
- Small-cap stocks with high scores may be difficult to trade at the prices shown.
- Regime changes could make value investing (and these scores) underperform for extended periods.
Use these scores as one input among many. Do not invest solely based on Dhando scores or backtest results.