/backtesting

name: backtesting
description: Use when running, interpreting, or designing backtests on Superior Trade — anything about backtest windows, trade-count thresholds, exit-reason mix, parameter sweeps, walk-forward validation, zero-trade diagnosis, compute-cost estimation, or "is this backtest result trustworthy?". Pair with the relevant strategy skill, such as mean-reversion or breakout.
metadata:
  version: 0.1.0
  updated: 2026-05-07

Backtesting Best Practices

Read [`../../references/unified-runtime.md`](../../references/unified-runtime.md) for the shared API lifecycle. Submit every run with `POST /runtime/backtests`; creation queues it, so there is no separate start request. Use the framework and venue fields defined by `GET /openapi.json` and the relevant venue skill. This page is about the **judgment calls** — picking a window that means something, telling signal from noise in the result, and knowing when to give up vs. iterate.

The trade-count bar (sample size first)

Trade count is the single most important number on a result page. Look at it before PnL, before Sharpe, before win rate.

| Trade count | Verdict | | ----------- | ---------------------------------------------------------------------------------------- | | < 30 | **Coincidence, not a strategy.** Don't promise anything; widen entries or extend window. | | 30-50 | Marginal. Sharpe is noisy. Treat results as directional, not numeric. | | 50-200 | **Useful.** Sharpe / profit factor start to mean something. | | 200+ | Statistical confidence. Now you can compare variants on micro-differences. |

**Watch for the trap**: backtests with 5-10 trades and a 100% win rate. They look like world-beaters and almost always disintegrate live. The strategy is too selective — every signal is a coin flip you've cherry-picked, not a repeatable edge. Widen the entry threshold, lengthen the window, or accept that there's no statistical signal here.

Pick a backtest window that means something

> A great backtest over the wrong window is a great fiction.

The window should answer: _"if I had deployed this strategy on day one of this window, what would have happened?"_ — **not** _"what's the prettiest curve I can fit?"_

Cover at least one regime change

Pure bull, pure bear, sideways chop — your window should include **at least two of the three**. A 90-day backtest in a one-direction market is a 90-day cherry-pick. A momentum strategy that prints +50% over a +60% trending window has told you nothing about itself; it's just measured beta.

Useful default windows

• **6 months of 1h data**, or
• **18 months of 1d data**

Less than that and you're really looking at noise. More than that and Hyperliquid's history may not cover the pair (HL was launched in 2023; many alts have < 12mo of data).

When you can't get a long window

Some HIP3 / new-listing pairs only have a few weeks of data. Two pragmatic responses:

1. **Walk-forward to a similar pair.** Pick a sibling perp with longer history (e.g. test the strategy on `BTC` if `BTC-AAPL` is too new), then assume the result transfers ±20%. 2. **Defer the deployment recommendation.** A 14-day backtest is too thin to recommend live capital, period. Tell the user that.

Don't optimize against the test set (the cardinal sin)

If you tune parameters and validate on the same window, your backtest is a souvenir, not a forecast. This is the single most common reason strategies that look brilliant on paper die in the first week of live trading.

The honest workflow:

1. Pick parameters from theory or a single observation, NOT from a sweep. 2. Run the backtest. 3. If results are bad, change the **idea**, not the **parameters**.

If you do sweep, **walk forward to a different period or pair** before promoting the winner. A parameter that wins on Q1 BTC and also wins on Q1 ETH (out-of-sample for the second pair) is real. A parameter that wins on Q1 BTC and gets re-validated on Q1 BTC is theatre.

Red flag

If the user's "strategy" started as a 5-variant sweep and now they want to deploy the winner — the result on the test window is overstated. Tell them to either run an out-of-sample check or accept they're deploying on optimistic numbers.

The 3-variant parameter sweep

When the user wants to find the right parameter (e.g. RSI threshold, ATR multiplier, lookback length), run **3 variants in one batch**, not iterative single backtests. The shape of the 3-result table tells you what to do:

| Variant | Config | PnL% | Trades | Sharpe | Max DD | | ------- | -------- | ---- | ------ | ------ | ------ | | A | RSI < 25 | … | … | … | … | | B | RSI < 30 | … | … | … | … | | C | RSI < 35 | … | … | … | … |

Read the shape:

• **All 3 profitable** → pick the **best Sharpe** (not best PnL — PnL rewards luck on small samples). Parameter is robust; proceed to walk-forward / deployment with confidence.
• **1-2 profitable** → pick the winner, but flag that the param is sensitive. Either (a) walk-forward to a 2nd pair, or (b) one tighter sweep around the winner.
• **All 3 unprofitable / < 10 trades each** → the **idea** doesn't work on this pair. Don't sweep again. Switch pair or strategy.
• **Monotonic edge** (PnL strictly improves A → B → C) → the best variant is at the edge of the grid. Run **one more** variant past it (e.g. RSI < 40). Don't run another full 3-grid; just extend by one.

Read the exit-reason mix

After the backtest completes, look at the breakdown of how trades ended. A healthy strategy ends through a mix of:

• Take-profit / `minimal_roi`
• Signal-based exit (`populate_exit_trend`)
• Trailing stop
• Time-based timeout (`custom_exit`)

Each one tells a different story. Distortions diagnose specific bugs:

| Distortion