arcodangeandClaude Opus 5
a2cafc0d6b
The discipline (ADR-0003, the promote flow, the operating rules) was written down and still depended on whoever was driving choosing to follow it. On 2026-07-25 an agent session wrote five documents into the production ledger through direct API calls, bypassing the promote flow entirely — a correct result reached by a path nobody could audit. A rule an operator can skip is a recommendation. The five stages are now chained by artefacts on disk. Each refuses to run until the previous produced its file, and the file says what it needs to hear: rehearse (sandbox, host-guarded) -> judge --pre -> gate (human) -> apply (prod) -> judge --post. The gate binds to a manifest digest, so approving a change-set approves THAT change-set. An op is defined ONCE, as an API call, and replayed on the sandbox then on production — because the first design described each write twice (a sandbox script input and a prod API body) and the pre-gate judge immediately caught them diverging: the rehearsal was creating a EUR invoice with no due date while production would have received a USD one at 60 days. Two descriptions of the same write are two things that can disagree. Judges are context-free, cross-family per the PRD qa-strategy rule, and advisory: a BLOCK still lets the operator approve, and the override is recorded with their name. Blocking authority stays with the human gate and the host guards — an LLM verdict never silently starts or stops a production write. Verified end to end against the real 24/08 change-set (M3 deferred, USD 3,000): - pre-gate judge (Mistral) returned BLOCK twice, correctly — first on the sandbox/prod divergence, then on a duplicate left by a repeated rehearsal; - apply refuses after a rejected gate; - apply refuses without ARCO_PROD_CONFIRM; - editing an amount after approval invalidates the gate on digest mismatch. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
73 lines
3.8 KiB
Markdown
73 lines
3.8 KiB
Markdown
# fleet/harness/promote/ — the gated pipeline
|
|
|
|
Rehearse on the sandbox → an independent agent judges → **a human decides** →
|
|
production → a second agent verifies what actually landed.
|
|
|
|
## Why this exists as code
|
|
|
|
The discipline was already written down — [ADR-0003](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/ADR/0003-sandbox-state-lifecycle.md),
|
|
the operating rules in [`AGENTS.md`](../../../AGENTS.md), the promote flow — and it
|
|
still depended on whoever was driving choosing to follow it. On 2026-07-25 an
|
|
agent session wrote five documents into the production ledger through direct API
|
|
calls, bypassing the promote flow entirely. Nothing was wrong with the result;
|
|
everything was wrong with the path. A rule an operator can skip is a
|
|
recommendation.
|
|
|
|
So the stages here are **chained by artefacts**, not by good intentions. Each
|
|
stage refuses to run until the previous one has produced its file, and the file
|
|
has to say what the stage needs to hear:
|
|
|
|
| Stage | Produces | Refuses unless |
|
|
| --- | --- | --- |
|
|
| 1 `rehearse` | `01-rehearsal.json` | the target is the sandbox (host-guarded) |
|
|
| 2 `judge --pre` | `02-pre-verdict.json` | a rehearsal exists and its writes succeeded |
|
|
| 3 `gate` | `03-gate.json` | a pre-verdict exists; **a human types the decision** |
|
|
| 4 `apply` | `04-applied.json` | the gate says `approved`, by a named human, for *this* manifest |
|
|
| 5 `judge --post` | `05-post-verdict.json` | production was applied |
|
|
|
|
The gate binds to a **manifest digest**: approving a change-set approves *that*
|
|
change-set. Edit one amount afterwards and stage 4 refuses — the approval no
|
|
longer matches what is about to be written.
|
|
|
|
## The two judges
|
|
|
|
Both are context-free: they receive the manifest and the evidence, never the
|
|
conversation that produced them. Per the PRD
|
|
[cross-family rule](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/qa-strategy.md#independent-verification--no-self-grading),
|
|
a judge SHOULD be a different model family than whoever built the change-set —
|
|
the admitted runtimes are Mistral (`vibe -p`) and Ornith 35B (hermes MLX), both
|
|
proven at verdict parity in [erp#63](https://gitea.arcodange.lab/arcodange-org/erp/issues/63).
|
|
|
|
- **Pre-gate** — prompted to *refuse*: find why this change-set is not safe to
|
|
promote. It reads the rehearsal evidence, not a description of it. Its verdict
|
|
goes to the human as an opinion, not a veto: a `BLOCK` still lets the operator
|
|
approve, and the override is recorded in the gate file.
|
|
- **Post-gate** — prompted to *doubt the success*: compare what production now
|
|
holds against what the sandbox rehearsal predicted, and report drift. It runs
|
|
after the writes, so it cannot prevent them — it exists so a silent
|
|
discrepancy becomes a recorded finding instead of a surprise months later.
|
|
|
|
Judges are advisory by design. The blocking authority is the human gate and the
|
|
host guards; an LLM verdict never silently stops or starts a production write.
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
P=fleet/harness/promote/pipeline.py
|
|
python3 $P rehearse --manifest changeset.json --run-dir runs/2026-07-24-m3-deferred
|
|
python3 $P judge --run-dir runs/... --stage pre --runtime mistral
|
|
python3 $P gate --run-dir runs/... # interactive; records who and when
|
|
python3 $P apply --run-dir runs/... # needs ARCO_PROD_CONFIRM
|
|
python3 $P judge --run-dir runs/... --stage post --runtime ornith
|
|
```
|
|
|
|
Every stage appends to `journal.jsonl`. The run directory is the evidence pack:
|
|
it is what you keep, and what an auditor reads.
|
|
|
|
## What this does not do
|
|
|
|
It does not replace the host guards (`dol-write.sh` refusing non-sandbox hosts,
|
|
`guard.ts` requiring an explicit production opt-in, the chronology guard in
|
|
`invoice-create.sh`). Those are structural and stay underneath. This pipeline
|
|
adds sequence and evidence on top of them.
|