ADR 0009: Performance budget
Context
Analysis must stay fast enough for local use and CI. A budget is only meaningful with a fixed workload, a measurement method and a recorded baseline.
Decision
- Workload: every fixture listed in
benchmarks/run.mjs, includingfixtures/small-app(10 exported functions, 30 tests), run with--offline --include-proposedand no LLM. - Measurement: wall-clock time of one
veritas runprocess per fixture and profile, on the CI runners (ubuntu-latest, macos-latest, windows-latest). - Budget per run (
benchmarks/budget.json):quick60 s,standard120 s,deep300 s. A run must also end with a verdict exit code (0, 1 or 4). - Baseline (2026-11-19, macOS arm64, Node 26):
small-appquick 1.2 s, standard 2.9 s, deep 4.7 s; every other fixture under 2.5 s in every profile. Results are inbenchmarks/results/. - CI runs
node benchmarks/run.mjs --checkon every OS and fails when a run exceeds the budget. - Usage counters (LLM tokens, solver milliseconds, mutants, test runs) are recorded per stage in
run.jsonand in reports; their budget is ADR 0010.
Consequences
- The budget leaves a wide margin over the baseline, so it catches regressions such as hangs or an order-of-magnitude slowdown, not small drifts.
- Larger projects are expected to use
--sinceoranalysis.includeto limit scope; stages that stop early reportpartial. - Changing the workload or the budget requires a new ADR.