Abstract
A directional trade rule needs a reference point: is today's model output high or low relative to normal? Our specialist shortlist computed that reference as the median prediction across the entire walk-forward window and self-flagged the resulting look-ahead as "small — expected magnitude ≤ 0.3 Sharpe." A causal re-run using a trailing rolling median (two-week lookback, two-week warm-up) revealed the true magnitude to be −1.5 to −2 Sharpe: six of the thirty promoted specialists flip to negative, and twenty-four of thirty change rank. We adopted honest rolling-median centring as the production default and re-promoted on it. The episode is recorded because the failure was internal estimate versus reality, and the only thing that caught it was running the honest computation rather than trusting the comment.
1 · The trade rule, and the offending line
For each (model, coin) specialist the entry logic is, at every bar:
pred_med = median over 5 seeds of the model prediction at t
centering = median(pred_med over the WHOLE window) # <-- look-ahead
direction = ic_sign * sign(pred_med[t] - centering)
open at t, hold 24 bars, close; net = direction * log(P[t+24]/P[t]) - 2*4bps
The centering term uses the median of the prediction
series computed across the full window — including bars in the future relative to
t. A live system standing at bar t
cannot know the median of predictions it has not yet made. The reference point is
therefore contaminated by future information, and because the sign of
(pred_med[t] − centering) sets trade direction, the
contamination flows straight into the direction decision.
2 · Why "≤ 0.3 Sharpe" felt right and was wrong
The intuition behind the small-magnitude estimate was that a median is a robust statistic — a few future bars shouldn't move it much, so the centring constant should be nearly the same whether computed in-sample or causally. That intuition ignores the mechanism: the damage is not in the value of the constant, it is in the sign decisions near the constant. Predictions that hover close to the centring level flip direction depending on whether the reference is nudged up or down by a handful of future bars, and those marginal trades are exactly the ones a full-window median gets "right" for free. The leak concentrates in the borderline trades, where it is most valuable and least defensible.
3 · The size of it
Re-pricing each top specialist with a strictly trailing rolling-median reference (336-hour lookback, 168-hour warm-up) against the inflated full-window version:
Every bar drops; six of the thirty cross zero into negative territory. The look-ahead was worth +1.5 to +2 Sharpe on average and far more on the most affected cells — an order of magnitude larger than the "≤ 0.3" estimate.
| Cell / coin | Full SR | Honest SR | Δ |
|---|---|---|---|
| nlinear_revin / PENDLE | +4.79 | +0.09 | −4.71 |
| lgbm / SEI | +4.07 | −1.45 | −5.52 |
| gru / NEAR | +3.90 | −1.66 | −5.56 |
| tcn / ARB | +3.20 | −2.58 | −5.78 |
| patchtst / WIF | +3.00 | −0.66 | −3.66 |
| vanilla_transformer / NEAR | +2.86 | −2.35 | −5.21 |
When all qualifying specialists are re-ranked by honest Sharpe, twenty-four of the original thirty change. The honest top-30 is genuinely strong — every cell clears +2.84 honest Sharpe — but it is a different thirty.
4 · The fix, and what is deployed
Centring switched to a causal trailing median by default, the promotion script
gained explicit --centring and
--sr-field flags so the choice is recorded rather than
implicit, and the per-bar backfill that drives the live dashboard now replays each
strategy under rolling centring — so the equity curves shown publicly are what the
strategy would actually have done, not the inflated version. The thirty
specialists now in paper validation were re-selected and re-priced on honest
centring throughout.
The bug was not the look-ahead — every researcher knows full-window statistics leak. The bug was trusting a code comment that estimated the magnitude instead of measuring it. The estimate was off by roughly an order of magnitude, and the only thing that surfaced it was building the causal version and diffing. We now require that any centring, normalisation or scaling choice that touches a trade decision be re-run causally before a Sharpe is published — no exceptions for "this one's obviously small."
5 · Connection to the broader discipline
This sits alongside two earlier corrections recorded on this site — a retracted +7.376 headline that was really −3.53 (Paper № 05), and a stop-overlay that read past PnL through a horizon overlap. The common thread is that an internally-consistent number can still be the wrong number, and the defence is to run the obvious honest check before publishing, not after a reader pushes back. The plausibility contract and the rolling-centring requirement are both responses to that pattern.
Sources & references
- Bailey, D. H., Borwein, J., López de Prado, M., & Zhu, Q. J. (2014). Pseudo-Mathematics and Financial Charlatanism. Notices of the AMS. (backtest overfitting)
- López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley. (leakage, purging, embargo)
- Axon Ridge internal — `research/experiments/results/Phase1_7_sharpe_audit_2026-05-28.md`
- Axon Ridge internal — `scripts/promote_specialists_to_paper.py` (`--centring rolling_336_168`)