Quant Concepts · Vol 3 of 6
Walk-forward testing: judging a trading rule on data it never saw
The reality check: how testing rules on unseen data helps tell repeatable results from a lucky fit, and why one test is not proof.
The short answer
Walk-forward optimization, also called walk-forward testing or analysis, tunes a trading rule on one stretch of past data, judges it on the next stretch it never saw, then slides both stretches forward and repeats. The test pieces join into one longer record on unseen data. A pass makes a lucky fit less likely, but it is still one historical path.
Key ideas
- Build on one stretch, judge on the next, and keep a last stretch locked.
- Walk-forward catches most lucky fits, and it also fails real edges. In this article's simulation, a rule with a small real edge missed the common 50% "efficiency" rule of thumb (yearly test return divided by yearly build return) in 74.3% of simulated worlds, while at least one of nine rules with no edge passed it in 25.0%.
- Every look at test data uses some of it up. When 100 rules were compared on the same test stretch, the winner's score there averaged a Sharpe ratio of 1.13, and on the next five years it averaged 0.05.
Show the numbers
| Block | Textbook name | What happens there | Times looked at |
|---|---|---|---|
| Build | In-sample data | Ideas are tried and settings tuned | As often as needed |
| Test | Out-of-sample data | The finished rule is judged | Once per finished rule; each extra look turns it into build data |
| Locked | Final hold-out | Nothing until every choice is made | Once, at the end |
What is out-of-sample testing?
Tuning, or optimization, means trying many settings of a trading rule, such as the length of a moving average, and keeping the best scorer in a backtest on past data. That score flatters the pick: Vol 2 showed how the best of many tries carries a luck bonus. Out-of-sample testing splits the history in time order, so the rule never learns from the future:
- Build data (in-sample data) to design and tune the rule, as often as needed.
- Test data (out-of-sample data), the next stretch, where the finished rule is scored once.
- A locked final stretch (a hold-out) that stays unseen until every choice is made.
The score used throughout is the Sharpe ratio: yearly return above a risk-free rate, divided by yearly return swings. With the rate set to zero here, 0.5 means the average yearly gain is half a typical yearly swing.
Worked example: one split
The worked numbers come from a simulation on synthetic markets, where the answer is known in advance. Each simulated world has 20 years of daily returns and 10 rule families, each a rule with one setting that can take 20 values. One family has a small real edge: its true Sharpe ratio rises to 0.5 in the middle settings and fades toward zero at the ends. The other nine have none. There are no trading costs, and each experiment runs 1,000 worlds. Here the best setting of each family is picked on years 1-15 and tested once on years 16-20.
| Simulated, 1,000 worlds, build years 1-15, test years 16-20 | Build Sharpe of the pick (mean) | Test Sharpe of the pick (mean) | Share of picks above 0 on test |
|---|---|---|---|
| Real-edge family (true Sharpe up to 0.5) | 0.68 | 0.46 | 84% |
| No-edge families (9 per world, pooled) | 0.31 | 0.00 | 50% |
Illustrative, simulated; worked numbers.
Picking the best of 20 settings adds a luck bonus on the build years: the real-edge pick scores 0.68, above the family's highest true Sharpe of 0.5, and no-edge picks score 0.31. On the test years the bonus is gone. No-edge picks land above zero in 50% of cases, a coin flip; the real-edge pick in 84%. A single test separates the two on average, but any one test can mislead.
The reason is sample size, the topic of Vol 1. With no true edge and independent daily returns, luck alone moves a Sharpe ratio by about 1 divided by the square root of the years measured: about 1.0 over one year, 0.45 over five and 0.24 over 17. Those are typical spreads, not limits. So a five-year test's luck spread is almost as large as the real edge it is trying to find.
What is walk-forward optimization?
Walk-forward analysis repeats the out-of-sample test: tune on one window, test on the next, slide both forward.
Wikipedia's entry and several trading-platform guides credit the method to Robert E. Pardo's 1992 book Design, Testing and Optimization of Trading Systems (text not available to check). His 2008 book, The Evaluation and Optimization of Trading Strategies (Wiley, second edition), lists a chapter titled "Walk-Forward Analysis" in the publisher's contents. TradeStation's documentation describes two common versions:
- Rolling (unanchored) windows: "the in-sample window (e.g. 2 years) is shifted forward to encapsulate the most recent out-of-sample period". The build window keeps its length and slides.
- Anchored windows: "the beginning of the in-sample window is fixed at the initial/original starting point". The build window always starts on the first day and grows.
In the simulation, rolling windows use a 3-year build and a 1-year test, stepping a year at a time: 17 test years (years 4-20). The anchored version tests the same years.
Show the numbers
| Real-edge family, worked numbers | Rolling (3-year build) | Anchored (build from year 1) |
|---|---|---|
| This world: average build Sharpe of the picks | 1.00 | 0.86 |
| This world: Sharpe of the 17 joined test years | 0.34 | 0.56 |
| This world: efficiency (test return / build return) | 34% | 66% |
| This world: different settings picked in 17 steps | 6 | 2 |
| 1,000 worlds: joined test Sharpe, mean (10th to 90th percentile) | 0.38 (0.07 to 0.69) | 0.43 (0.11 to 0.74) |
| 1,000 worlds: efficiency, median (10th to 90th percentile) | 37% (8% to 61%) | 59% (18% to 95%) |
| 1,000 worlds: different settings picked, median | 8 | 3 |
What is walk-forward efficiency, and is 50% a pass mark?
TradeStation's walk-forward tool "calculates and compares annualized rates of return for the in-sample and out-of-sample results. This statistic is the Walk-Forward Efficiency". In plain terms: the yearly return on test data divided by the yearly return on build data, as a percentage. The same documentation gives the usual threshold: "As a rule of thumb, a Walk-Forward Efficiency of 50% or more is considered a measure of a successful walk-forward analysis."
A simulated family passes when its efficiency is at least 50% and its joined test record is profitable.
- The real edge often fails. With rolling windows its efficiency fell below 50% in 74.3% of worlds, because on a 3-year build window the picked setting's luck bonus inflates the build return, the bottom of the ratio. With anchored windows the share was 38.1%.
- No-edge rules sometimes pass, more often with anchored windows. Each no-edge family, on average, passed in 3.2% of rolling worlds and 16.7% of anchored ones; at least one of the nine passed in 25.0% and 79.5%. A long build window gives a no-edge rule a small build return, and half of a small number is a low bar.
Show the numbers
| Family | Build Sharpe | Test Sharpe | Efficiency | Passes 50% rule |
|---|---|---|---|---|
| Real edge | 1.00 | 0.34 | 34% | no |
| No edge 1 | 0.81 | 0.25 | 31% | no |
| No edge 2 | 0.82 | 0.13 | 16% | no |
| No edge 3 | 0.55 | −0.09 | −16% | no |
| No edge 4 | 0.74 | 0.11 | 15% | no |
| No edge 5 | 1.10 | 0.11 | 10% | no |
| No edge 6 | 0.73 | 0.01 | 2% | no |
| No edge 7 | 0.46 | −0.31 | −69% | no |
| No edge 8 | 0.63 | −0.12 | −19% | no |
| No edge 9 | 0.59 | −0.26 | −45% | no |
Across 1,000 simulated worlds:
| Worked numbers, 1,000 worlds | Rolling | Anchored |
|---|---|---|
| Real edge passes | 25.7% | 61.9% |
| Real edge efficiency below 50% | 74.3% | 38.1% |
| One no-edge family passes | 3.2% | 16.7% |
| At least one of nine no-edge families passes | 25.0% | 79.5% |
| At least one of nine passes the stricter four-part check | 24.8% | 65.9% |
| Real edge has the best joined test Sharpe of the 10 | 51.8% | 60.9% |
| No-edge build Sharpe, mean | 0.68 | 0.40 |
| No-edge joined test Sharpe, mean | 0.00 | 0.00 |
Stricter four-part check: the pass rule plus two of TradeStation's other default conditions, at least half of the 17 test years profitable and no single test year making half or more of the net profit. Drawdown is not checked.
What does walk-forward not fix?
Walk-forward works only while the test data stays unseen and the test design is fixed in advance.
Peeking turns test data into build data
Halbert White, in his 2000 paper "A Reality Check for Data Snooping", defines the problem: "Data snooping occurs when a given set of data is used more than once for purposes of inference or model selection." Each time a test result is used to pick or change a rule, the test data starts to become build data, and each walk-forward rerun is one more try in the count from Vol 2. Michael Harris (Price Action Lab blog) warns that "given enough permutations, one may find a spurious model that may pass cross-validation tests by chance alone." Walk-forward is one kind of cross-validation.
Bailey, Borwein, López de Prado and Zhu note that experienced researchers know how markets behaved over the test years, and "that information may well be used in the strategy design, consciously or not".
A second simulation mixes one rule with a small real edge into a pool of no-edge rules, up to 100 in all, and picks whichever scores best on the same 5-year test set.
Show the numbers
| Rules compared on one test set | Test-set Sharpe of the pick, mean (10th to 90th) | Next 5 years, never used, mean (10th to 90th) | Pick is the real rule | Same pool without the real rule: test-set Sharpe of the pick |
|---|---|---|---|---|
| 1 | 0.50 (−0.06 to 1.04) | 0.50 (−0.07 to 1.05) | every world | 0.00 |
| 2 | 0.58 (0.08 to 1.06) | 0.40 (−0.22 to 1.00) | 78.2% | 0.26 |
| 5 | 0.70 (0.29 to 1.12) | 0.26 (−0.37 to 0.89) | 54.1% | 0.52 |
| 10 | 0.80 (0.44 to 1.15) | 0.19 (−0.44 to 0.83) | 38.4% | 0.68 |
| 20 | 0.90 (0.62 to 1.22) | 0.13 (−0.47 to 0.75) | 25.2% | 0.84 |
| 50 | 1.04 (0.78 to 1.33) | 0.07 (−0.50 to 0.69) | 12.8% | 1.01 |
| 100 | 1.13 (0.90 to 1.38) | 0.05 (−0.49 to 0.67) | 8.5% | 1.12 |
With 10 rules, the pick scores 0.80 on the test set and 0.19 afterward, and it is the real rule in only 38.4% of worlds. Without the real rule in the pool, the test-set score climbs the same way: the luck curve from Vol 2, moved one step later.
One pass is not proof: Arnott, Harvey and Markowitz (2019) describe a strategy that sorts stocks by the letters of their ticker symbols, developed on data from 1963 to 1988 and "then validated out of sample with even stronger results over the years 1989 through 2015."
Window lengths can be overfitted too
Walk-forward has its own settings: how long the build window is, how long each test is, and how often the rule is re-tuned. Choosing them by their results is one more round of tuning. The simulation ran 18 window settings (build windows of 1 to 6 years, tests of 3, 6 or 12 months), judged them all on test years 7-15, then ran the best one unchanged on untouched years 16-20.
| Worked numbers | Real edge | No edge |
|---|---|---|
| Best of 18 window settings, years 7-15 (mean Sharpe) | 0.64 | 0.28 |
| Average of the 18 settings, years 7-15 | 0.38 | 0.00 |
| Best minus worst setting in the same world, median | 0.55 | 0.56 |
| Best setting run unchanged on untouched years 16-20 | 0.38 | 0.00 |
| Average of the 18 settings on years 16-20 | 0.39 | 0.00 |
Illustrative, simulated; 1,000 worlds, rolling walk-forward. The no-edge column pools the nine no-edge families.
On the untouched years, the no-edge rule's best window setting fell from 0.28 to 0.00, and the real edge's from 0.64 to 0.38, no better than the average setting. Best and worst settings in one world differed by a median of 0.55 for the real edge and 0.56 for no edge, so window choice alone moves a result by about as much as the real edge is worth.
A preprint by Mroziewicz and Ślepaczuk (2026), not yet peer reviewed, tested an exponential moving average rule across 81 window combinations and found its performance "is highly dependent on the chosen window size."
It is still one historical path
A walk-forward record is still a backtest on the one sequence of prices that happened. Daniel Palomar's lecture notes on backtesting (HKUST) state that in walk-forward "A single scenario is tested (the historical path), which can easily lead to overfitting". López de Prado's 2018 book Advances in Financial Machine Learning lists "Walk-forward (historical) backtesting" among ten common pitfalls. His remedy, combinatorial purged cross-validation, tests many build and test splits instead of one. It also drops build data whose outcomes overlap a test stretch (purging) and data just after it (an embargo), closing specific leak paths.
Look-ahead leaks survive walk-forward
Walk-forward does not catch look-ahead bias, where a test uses information that was not yet available. Valeriy Zakamulin (2018) re-examined a published moving-average timing study and showed that its "extraordinarily good" results came from "simulating trading with look-ahead bias." Once that was fixed, "the performance of the moving average strategy is indistinguishable from the performance of the buy-and-hold strategy."
What does the research say about the drop on new data?
- The best of thousands of rules did not repeat. Sullivan, Timmermann and White (1999) tested 7,846 technical trading rules on 100 years of the Dow Jones Industrial Average. The working-paper version reports that "the superior performance of the best technical trading rule is not repeated in the out-of-sample experiment covering the ten-year period 1987-1996".
- Published strategies lose part of their score after the paper's data ends. Quantpedia re-ran published trading strategies on the years after each source paper's sample. The median Sharpe ratio was 1.180 in-sample and 0.662 afterward, a drop of 43.90%. That later period mixes overfitting with edges that fade as markets change, so it is not a pure measure of either.
How do systematic researchers go further?
- Wait for data that does not exist yet. Bailey and co-authors cite Leinweber and Sisk, who proposed a strategy at a conference and announced they would publish its results six months later on data not yet observed, an approach they called "model sequestration". A live record kept after a rule is frozen works the same way.
- Count every try and test the winner against luck. The deflated Sharpe ratio from Vol 2 raises the bar with the number of rules and settings tried, including every walk-forward rerun. White's Reality Check does this formally, testing "the null hypothesis that the best model encountered in a specification search has no predictive superiority over a given benchmark model."
Common mistakes
- Treating 50% efficiency as a tested pass mark. It is a vendor's rule of thumb.
- Re-running the walk-forward after every tweak, or picking the best of many rules on one test set. Each is another try on the same data.
- Choosing window lengths by their results. The best of 18 did no better than average on untouched years.
- Calling walk-forward a forward test. A forward test, or paper trading, follows the rule live without real money, on prices that did not exist when it was built. Walk-forward uses only past data.
- Opening the locked stretch early. Once it has been looked at, no unseen data remains.
See it on a chart: the Strategy Passport
The Strategy Passport on the NovaLens homepage uses one build and test split with a dated boundary. Its period line reads: "Tuned on data to 2025-12-31; 2026 is unseen data."
The unseen stretch runs from 2026-01-01 to 2026-10-05, about nine months (0.76 years). Under the same no-edge assumptions, that gives a luck spread of about 1.15, a general figure rather than one fitted to this strategy, so those months are one short test. The Passport also shows 71 trades, labelled "Small sample". These are published backtest figures: past results, not a forecast. Whether a rule keeps working as markets change is a separate question, the subject of Vol 5.
Rule of thumb
A walk-forward pass counts only if the test data was used once and the window settings were fixed before the test. Every extra look is one more try, and the result is still one historical path.
Sources
All accessed 2026-10-12.
- Pardo, R. E. (1992). Design, Testing and Optimization of Trading Systems. Wiley, New York. Attribution as given in Wikipedia, "Walk forward optimization": https://en.wikipedia.org/wiki/Walk_forward_optimization (book text not read)
- Pardo, R. E. (2008). The Evaluation and Optimization of Trading Strategies, 2nd edition. Wiley. Publisher page with contents: https://www.wiley-vch.de/en/areas-interest/finance-economics-law/the-evaluation-and-optimization-of-trading-strategies-978-0-470-12801-5
- TradeStation Walk-Forward Optimizer help: "About the Walk-Forward Optimizer" https://help.tradestation.com/09_01/tswfo/topics/about_wfo.htm; "Walk-Forward Summary (Out-Of-Sample)" https://help.tradestation.com/09_01/tswfo/topics/walk-forward_summary_out-of-sample.htm; "Test Results - Criteria" https://help.tradestation.com/09_01/tswfo/topics/walkforward_test_results.htm; FAQ https://help.tradestation.com/09_01/tswfo/topics/frequently_asked_questions.htm
- White, H. (2000). "A Reality Check for Data Snooping." Econometrica 68(5), 1097-1126. https://doi.org/10.1111/1468-0262.00152
- Sullivan, R., Timmermann, A., and White, H. (1999). "Data-Snooping, Technical Trading Rule Performance, and the Bootstrap." Journal of Finance 54(5), 1647-1691. https://doi.org/10.1111/0022-1082.00163. Quotes from the working-paper version, LSE Financial Markets Group Discussion Paper 303 (1998).
- Bailey, D. H., Borwein, J. M., López de Prado, M., and Zhu, Q. J. (2017). "The Probability of Backtest Overfitting." Journal of Computational Finance 20(4). Open version: https://www.davidhbailey.com/dhbpapers/backtest-prob.pdf
- Bailey, D. H., Borwein, J. M., López de Prado, M., and Zhu, Q. J. (2014). "Pseudo-Mathematics and Financial Charlatanism." Notices of the American Mathematical Society 61(5). Open version: https://www.davidhbailey.com/dhbpapers/backtest-pseudo.pdf
- Arnott, R., Harvey, C. R., and Markowitz, H. (2019). "A Backtesting Protocol in the Era of Machine Learning." Journal of Financial Data Science 1(1), 64-74. Open version: https://people.duke.edu/~charvey/Research/Published_Papers/P138_A_backtesting_protocol.pdf
- López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley. Table 1.2 in the author's excerpt: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3104847
- Palomar, D. P. "Backtesting Portfolios." Lecture slides, MAFS5310, HKUST. https://palomar.home.ece.ust.hk/MAFS5310_lectures/slides_backtesting.pdf
- Zakamulin, V. (2018). "Revisiting the Profitability of Market Timing with Moving Averages." International Review of Finance 18(2), 317-327. https://doi.org/10.1111/irfi.12132
- Mroziewicz, Ślepaczuk (2026). arXiv preprint 2602.10785, not peer reviewed. https://arxiv.org/abs/2602.10785
- Harris, M. (2013). "Fooled By Out of Sample Testing." Price Action Lab blog. https://www.priceactionlab.com/Blog/2013/05/fooled-by-out-of-sample-testing/
- Dujava, C. "In-Sample vs. Out-Of-Sample Analysis of Trading Strategies." Quantpedia. https://quantpedia.com/in-sample-vs-out-of-sample-analysis-of-trading-strategies/
- NovaLens Strategy Passport (Kalman Velocity Strategy, BTC/USD daily). https://novalens.io/#approach
Simulations: computed for this article from synthetic data (no market data). Code, seeds and outputs are kept with the article's research pack.