A profitable backtest is the beginning of a question, not the end of the research. The result says that a set of rules performed well on a specific historical sample under specific assumptions. It does not say that the strategy will continue to work, that it can be executed at the modeled prices, or that the drawdowns are acceptable for real capital.
The first problem is cost. Many simple strategies look strong before fees and weak after fees. A daily rebalanced strategy that trades heavily can lose most of its apparent edge to commissions, spreads, taxes, borrow costs, and market impact. Even in highly liquid markets, assuming perfect execution at the close or the next open can be optimistic.
The second problem is slippage. A backtest often uses one price point, but real orders face queues, spreads, gaps, partial fills, and latency. A strategy that depends on small price differences is especially sensitive to execution assumptions.
The third problem is data leakage. Leakage happens when information from the future enters the training or decision process. Examples include using revised macro data without point-in-time adjustment, ranking stocks with today's index constituents for a test that begins years ago, or normalizing features with statistics calculated from the entire data set.
The fourth problem is overfitting. If a researcher tests many parameters, indicators, filters, and date ranges, the best backtest may simply be the one that matched noise. A strategy can look scientific while quietly becoming a custom fit to history.
A stronger process uses out-of-sample tests, walk-forward validation, realistic costs, stress periods, and simple rules. The strategy should be evaluated not only by total return but also by maximum drawdown, volatility, turnover, exposure, capacity, and behavior during adverse regimes.
Backtests are valuable because they force ideas into measurable form. They become dangerous when they are marketed as proof. Treat them as research evidence, not as a promise.
Research Question
Why can a profitable historical simulation fail when exposed to live execution, costs, and changing regimes?
Why This Matters
Backtest charts are persuasive. They can also hide the assumptions that matter most. Readers need to understand why simulated profit is only evidence about one historical design, not proof of a live edge.
Practical Example
A strategy that earns small average gains per trade may look attractive before costs. If the spread widens, fills are delayed, or the signal is known only after the close, the same strategy can become negative. The result was not necessarily fraudulent; the original assumptions were incomplete.
Evidence Checklist
- Measure net performance after fees, spreads, and slippage.
- Check whether signals are known before the modeled execution price.
- Review out-of-sample performance and stress periods separately.
- Compare turnover and capacity with the size the strategy claims to support.
Known Limitations
- No cost model perfectly predicts live fills.
- Out-of-sample tests can still be overfit if reused too often.
- A backtest may omit operational failures that occur in production.
- Past crisis behavior may not cover the next crisis.
Reader Actions
- Rerun one result with higher costs.
- Write the execution time in the strategy rule.
- Separate gross return from net return.
- Look for performance concentration in one short period.