← CLI reference

rad-experiment-compute-delta(1) General Commands Manual rad-experiment-compute-delta(1)

rad-experiment-compute-delta - Compute direction-aware deltas from benchmark JSON files

rad-experiment compute-delta <--baseline> [-r|--repo] <--candidate> [-q|--quiet] [--pretty] <--primary-metric> <--criteria> <--bench-cmd> [--build-cmd] <--base-commit> <--head-commit> <--description> [--pending] [-h|--help]

Reads two JSON files produced by rad-experiment-benchmark(1) — a baseline and a candidate — computes direction-aware deltas, and emits a single JSON object plus an optional pending file for the auto-publish hook.

The primary metric and its criteria are passed explicitly (the benchmark files also carry criteria per metric, which is used for secondaries). bench_cmd and build_cmd passed here flow through into the output JSON so `publish --from-json` can lift them onto the COB.

The output JSON carries the canonical metric shape on both the top level (for the primary metric) and on every entry in the `metrics` map: • `delta_pct_x100` — raw, signed (candidate − baseline). • `improvement_delta_pct_x100` — direction-normalized. • `criteria` — `higher_is_better` | `lower_is_better`. • `verdict` — `improved` | `regressed` | `neutral`.

With --pending=true (the default), the same JSON is also written to /tmp/cc-experiment-pending/{head_sha}.json for the auto-publish hook.

Baseline benchmark JSON file
Repository path or Radicle RID (defaults to current directory)
Candidate benchmark JSON file
Suppress non-error output to stderr
Pretty-print JSON output (for human inspection; agents should omit)
Primary metric name
Criteria for the primary metric: lower_is_better or higher_is_better
Benchmark command (propagated into the output JSON so publish can lift it onto the COB)
Optional build command (propagated like bench_cmd)
Base commit ref (resolved to full SHA)
Candidate commit ref (resolved to full SHA)
Experiment description / hypothesis
Write a pending file to /tmp/cc-experiment-pending/ for auto-publish
Print help (see a summary with '-h')

EXAMPLE — end-to-end, no auto-publish hook:

rad-experiment benchmark --worktree /tmp/repo-base --bench-cmd 'bash ./bench/benchmark.sh' --metric 'duration_ms=ms:lower_is_better:duration\s*:\s*([0-9.]+)\s*ms' --runs 5 --label baseline > /tmp/baseline.json rad-experiment benchmark --worktree /tmp/repo-head --bench-cmd 'bash ./bench/benchmark.sh' --metric 'duration_ms=ms:lower_is_better:duration\s*:\s*([0-9.]+)\s*ms' --runs 5 --label candidate > /tmp/candidate.json

rad-experiment compute-delta --baseline /tmp/baseline.json --candidate /tmp/candidate.json --primary-metric duration_ms --criteria lower_is_better --bench-cmd 'bash ./bench/benchmark.sh' --base-commit 9b32764 --head-commit 5574144 --description "Hoist allocation" --pending=false > /tmp/delta.json

jq '.primary_metric, .delta_display, .secondary_flags' /tmp/delta.json

compute-delta