Once a strategy is running, improvement ideas never stop coming — and every one looks like it adds return. The hard part: keeping 'looks better' from quietly ruining a strategy that works.
Establish a pre-registration discipline for strategy improvements: any enhancement idea writes down its pass criteria and threshold before the data runs; clear the bar and it can enter production, fall short and it's archived as a negative-result report. The Agent tests every idea fully in the same framework and reports honestly — including rejecting the proposals it raised itself and liked best.
Pre-registers pass criteria for every idea and rules by the threshold after the run; rejected ideas get written up all the same.
Runs each idea through the same historical A/B framework, producing standardized comparison tables and robustness checks.
Rules for strategy enhancement:
A human or an Agent proposes an enhancement; motivation, expectations, and pass criteria get logged first.
Before the run, the thresholds are pinned down: risk-adjusted return, drawdown, robustness in the parameter neighborhood.
@quant-eng runs the A/B in the shared framework, including sub-periods and cost sensitivity.
@cio rules by the pre-registered criteria. One round in, all thirteen ideas fell short — including the one it had ranked highest itself.
Rejected ideas become numbered research reports; that rejection ledger later became a differentiator in external pitches.
Every rejected idea's motivation, data, and conclusion stays searchable and reviewable.
Key conclusions get re-verified against alternate benchmarks periodically to catch drift.
When market structure shifts, old ideas get retested against the same bar.
Freeze the pre-registration template: motivation, mechanism hypothesis, pass criteria, threshold — one form filled out before any run.
Put a barrier in front of gut-feel parameter changes: every production parameter change must attach a corresponding evidence report.
Run a periodic rejection retrospective: which rejected ideas deserve a re-proposal under a different mechanism.