Self-improving trading models: a closed loop that only keeps measured gains
How Sentio turns a failed replay into a repair loop — a multi-agent debate diagnoses from per-trigger evidence, edits the spec, re-validates on the same history, and keeps the change only if the numbers improved. Method, a worked case, and the limits.
Abstract
Traders describe setups in language; detectors run on parameters. The gap between the two is where most retail "strategies" die: a rule that reads well fires on noise, stops out inside normal range, or never fills. We describe a closed loop in which a compiled model that fails replay validation is handed back to the same agents that compiled it, together with the outcome of every trigger it produced. The agents diagnose from that evidence, change the fewest parameters that address it, and the edited model is re-validated on the same history. A revision is kept only if it measurably improves the validation result; otherwise it is reverted and the loop stops. Every round is written to a changelog with the plain-English reading of the model regenerated alongside the spec, so the trader can follow and undo each change. On a real draft the loop correctly refused two revisions that made the model worse. We report the method, an honest case, and the limits of validating on the same window that drives the edit.
1Introduction
Sentio's model builder compiles a trader's description into a spec of primitives: sweeps, gaps, displacement, structure, session windows, entries, stops and targets. Before a spec can go live it is replayed against history and must pass a gate: enough triggers for the timeframe, and at least a 40% win rate on decided trades1. Most first drafts fail that gate. The interesting question is what happens next.
The conventional answer is "the trader tweaks a number and tries again". That works when the trader can see why it failed, and in practice they cannot: a replay report says "1 win, 6 losses", which is a verdict, not a diagnosis. This note describes what we built instead: a loop that reads the losses individually, proposes a repair, tests it, and keeps a record that a human can audit.
2Method
2.1 Compile and debate
Every compile is a multi-agent debate2. A compile step writes the spec from the description. A critique step reads it and files concerns with a severity and a suggested fix. A revision step weighs each concern and produces a revised spec that keeps the intent. Revisions that fail the schema are discarded. This is the same machinery the repair loop reuses; nothing new is invented for it.
2.2 Validation
A validation run replays the spec over the trader's chosen lookback (30, 60 or 90 days; up to two years for hourly models). At every bar the evaluator is asked whether the rule fires. A fire counts as a trigger only when every step passed and price traded into the entry zone. Each trigger is scored against the bars that follow: target one before stop is a win, stop first is a loss, neither within the horizon is undecided. Whether target two was reached before price returned to entry is recorded separately, for the payout split.
2.3 Evidence, not totals
The repair round does not receive "1 win, 6 losses". It receives the trigger table: for each fire, the setup bar, the entry bar, entry, stop, targets, the outcome, and how many bars it took to resolve, plus a measured summary such as the median bars to stop and how many losses were stopped within three bars. It also receives the meaning of every parameter in the spec3, so a stop problem is repaired at the stop and not by changing an unrelated setting.
2.4 Acceptance
The edited spec is replayed over the same window with the same rules and its result is compared to the previous one. The revision is kept only if the validation result strictly improves: a model that now passes the gate, or a better win rate on decided trades. Otherwise the spec is restored and the loop stops. At most three rounds run per validation. The loop also runs on demand, and never on a model that already passes; a passing model gets suggestions, not edits.
2.5 Provenance
Each round is appended to the model's improvement log: what changed, as a line-by-line diff of parameters; why, as each critique concern paired with the revision's verdict; the validation numbers before and after; and whether it was kept. When a round is kept, the model's plain-English reading is regenerated from the new spec in the same write, so the description the trader sees never lags the spec. Any kept round can be restored, which is itself logged. Manual parameter edits enter the same log, so the trader's hand and the loop's are one history.
3A worked case
The draft below was compiled from: "On 5m NQ, long when price sweeps the prior day low and closes back above it during the NY AM killzone, entering at the sweep candle midpoint." Sixty days of 5m history, 30 September window shown for the loop.
| Stage | Triggers | W / L | Win rate | Decision |
|---|---|---|---|---|
| Validation, as compiled | 7 | 1 / 6 | 14% | fails gate |
| Round 1: stop buffer 2 → 8 ticks, swing strength 100 → 288 | 2 | 0 / 2 | 0% | reverted |
| Round 2: stop buffer 2 → 8 ticks, swing strength 100 → 200 | 2 | 0 / 2 | 0% | reverted |
The critique's diagnosis was correct and specific: six of six losses were stopped on the very next bar, so the stop sat inside 5-minute noise, and one trigger carried a 1.25-point stop, a micro-pivot. The revision's repair, widening the stop and demanding a stronger swing, was reasonable and still made the model worse: it cut triggers from seven to two with no wins. The loop kept the original and stopped. That is the behaviour we wanted. A loop that "improves" by changing something is worse than no loop.
Three of the critique's findings were flagged as needing the trader: whether "prior day low" means the session's absolute low, whether the killzone should widen, whether to add a minimum-risk filter. The loop does not guess at intent. Those questions go back to the person who wrote the sentence.
4Limitations
- In-sample acceptance. The revision is judged on the same window that produced the diagnosis. That is deliberate, because the alternative for a 60-day intraday history is no test at all, but it means a kept round is evidence of fit, not of edge. We mitigate with the strict-improvement key, a three-round cap, and by surfacing the sample size; the P&L card refuses to read a profit factor off fewer than 30 decided trades.
- Data depth. The provider serves about 60 days of 5-minute bars. Hourly models can validate over two years; intraday ones cannot, and the interface says so instead of extrapolating silently.
- Fill and excursion model. Fills are bar-range touches of the entry zone; intra-bar order is unknown, so a bar touching both stop and target is scored as the stop.
- Agents are not oracles. A critique can be wrong. The acceptance test is what protects the model from a wrong critique; the log is what protects the trader from a confident one.
5Reproducibility
Everything above is deterministic given the history: the same bars and the same spec produce the same triggers and the same gate. The agents are the stochastic part, which is why their output is never applied without the deterministic test. The improvement log on any draft is the full record; nothing is summarised away.
Sentio (2026). Self-improving trading models: a closed loop that only keeps measured gains. Sentio Research Note 2026-03. - Trigger floor by execution timeframe: three on 1–5m, two on 15m, one on 1h and above. Undecided trades do not enter the win-rate denominator.
- See the model builder page for the debate as it runs on every compile.
- The primitive glossary is the same document the builder shows the trader; the agents and the human read one source of truth.