Merge pull request 'feat(fleet): multi-runtime harness + erp#63 parity/bench evidence' (#69) from arcodange/harness-portability into main

2026-07-18 19:59:01 +02:00
parent 9b23ee0d52 ceb4321224
commit 66b277abfa
21 changed files with 2638 additions and 1 deletions
@@ -38,6 +38,7 @@ The [AI back-office PRD](https://gitea.arcodange.lab/arcodange-org/factory/src/b
 - **Environment rules**: the [operating rules](#operating-rules-for-agents) above + [`.claude/skills/dolibarr-sandbox-write/SKILL.md`](.claude/skills/dolibarr-sandbox-write/SKILL.md) (the host-guarded sandbox write path and its promote gate).
 - **Autonomy ladder**: levels A0–A3 in the PRD [hub](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/README.md#the-autonomy-ladder); promotion/demotion per the PRD [qa-strategy gates](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/qa-strategy.md#autonomy-promotion-gates).
 - **Graduation**: an atom earns autonomy through its golden-set evals and unedited-approval streaks — the earned level + eval evidence live in its `atom.yaml` `autonomy` field, and a promotion is a PR changing that field with the evidence linked.
+- **Harness**: [`fleet/harness/`](fleet/harness/README.md) — the multi-runtime execution layer around the atoms: canonical verifier tests (locate-test, backlog audit), `run-verifier.sh` for any OpenAI-style local endpoint or `vibe -p` (Mistral), and `vibe-builder.sh` (the capped, worktree-guarded shell for scoped builders and recurring tasks). Runtimes are admitted per role by evidence ([erp#63](https://gitea.arcodange.lab/arcodange-org/erp/issues/63)); Claude is the escalation tier, not a prerequisite, per the PRD [harness portability](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/model-fleet.md#harness-portability).

 ## Before building anything

@@ -47,7 +47,8 @@ fleet/
 │       ├── prompt.md
 │       └── scripts/
 ├── golden/                    # per-atom golden sets — land with erp#39
-└── profile/                   # fiscal.yaml + calendar.yaml + ADC register — land with erp#54
+├── profile/                   # fiscal.yaml + calendar.yaml + ADC register — land with erp#54
+└── harness/                   # multi-runtime harness layer: verifier tests + builder bench (harness/README.md)
 ```

 ## `atom.yaml` — the contract, field by field
@@ -0,0 +1,72 @@
+# fleet/harness/ — the multi-runtime harness layer
+
+The **harness** is the orchestration layer around the atoms: builder sessions that
+execute backlog issues, cold verifiers that check them (locate-tests, backlog
+audits, refutation passes), and the evidence flow into Gitea. Per the PRD
+[model-fleet › harness portability](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/model-fleet.md#harness-portability)
+(operator direction 2026-07-15), this layer must not have Anthropic as a hard
+dependency: the same loop runs on **Mistral** (`vibe -p`, `mistral-medium-3.5`)
+or on **hermes-served local models** (Ornith / MLX, `127.0.0.1:18080`). Claude is
+an escalation tier, not a prerequisite. Admission of a runtime to a role is
+**evidence-gated** ([erp#63](https://gitea.arcodange.lab/arcodange-org/erp/issues/63)):
+verifier roles first, scoped builders benched second, and no acceptance gate is
+ever relaxed for a cheaper runtime.
+
+## Layout
+
+| Path | Role |
+| --- | --- |
+| `verifier/locate-test.md` | canonical locate-test: prompt, inputs, ground truth, pass rule |
+| `verifier/backlog-audit.md` | canonical cold-reader backlog audit: prompt, inputs, rubric |
+| `bin/run-verifier.sh` | run a verifier test against a runtime; emits a JSON transcript |
+| `bin/vibe-builder.sh` | run a scoped builder bench (`vibe -p`) inside a worktree, with caps + journal |
+| `runs/<date>/` | committed evidence transcripts, when they back an issue comment |
+
+## Runtimes
+
+| Runtime | How the harness reaches it | Typical role |
+| --- | --- | --- |
+| `claude` | a **context-free subagent** in a Claude Code session, given the exact assembled prompt (`run-verifier.sh <test> --print-prompt`) and nothing else | baseline verifier; multi-file builder (default per the PRD complexity ceiling) |
+| `ornith` | hermes MLX server, OpenAI-style `POST /v1/chat/completions` on `127.0.0.1:18080`, model `leonsarmiento/Ornith-1.0-35B-5bit-mlx` | verifier (candidate) |
+| `mlx --model <id>` | same endpoint, any model the server lists under `/v1/models` | verifier (candidate) |
+| `mistral` | `vibe -p` programmatic mode, tools disabled, model = the vibe `active_model` (today `mistral-medium-3.5`) | verifier (candidate); scoped builder via `vibe-builder.sh` |
+
+## Verifier protocol — no self-grading
+
+1. Assemble the prompt from the canonical test file + the pinned input documents
+   (`run-verifier.sh` embeds file contents verbatim and records their sha256).
+2. Run every candidate runtime on the **same assembled prompt**, temperature 0.
+3. **An independent, context-free judge** (never the session that built the thing,
+   per the PRD [qa-strategy](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/qa-strategy.md#independent-verification--no-self-grading))
+   scores each transcript against the test's ground truth and emits the parity
+   table. A runtime is **admitted to verifier duty** when it reaches verdict
+   parity with the Claude baseline on both tests.
+4. Once a non-Claude verifier is admitted, **prefer cross-family verification**:
+   the verifier SHOULD be a different model family than the builder — a foreign
+   family refuting the builder is stronger evidence than the builder's own family
+   agreeing with itself.
+
+## Builder bench protocol
+
+`vibe-builder.sh` runs one tightly-footered backlog issue end-to-end under a
+non-Claude runtime, against the **unchanged** Execution footer and acceptance
+gates. It measures completion, intervention count and wall-clock; a failed bench
+is a valid result — it sets the complexity ceiling honestly. Safety bounds:
+
+- refuses to run anywhere that is not a **linked worktree** (never the trunk —
+  same structural-guard pattern as `dol-write.sh`);
+- hard caps: `--max-turns` and `--max-price` are always set;
+- `--auto-approve` is acceptable only because the blast radius is bounded: a
+  disposable worktree, read-only API credentials, and the caps above;
+- the full `vibe` JSON journal is kept per run.
+
+## Recurring tasks on the Mistral tier
+
+A recurring task (T11 reminders, T13 drift checks, T14 backup freshness) is a
+**scoped builder with a standing prompt**: cron (hermes `cron` or the operator's
+scheduler) calls `vibe-builder.sh <worktree> <task-prompt.md>` and routes the
+journal into the digest. The task prompt lives with the atom
+(`fleet/atoms/<atom>/prompt.md` + its class skeleton); the harness only supplies
+the bounded execution shell. No recurring task writes outside its worktree, and
+anything ERP-write-shaped still goes through the sandbox + promote gate —
+runtime choice never changes the gates.
@@ -0,0 +1,127 @@
+#!/usr/bin/env bash
+# Run a canonical verifier test against a runtime; emit a JSON transcript.
+# See fleet/harness/README.md (runtimes, no-self-grading protocol).
+set -euo pipefail
+
+usage() {
+  cat >&2 <<'EOF'
+usage: run-verifier.sh <locate|backlog> <ornith|mlx|mistral> [options]
+       run-verifier.sh <locate|backlog> --print-prompt [options]
+
+options:
+  --agents-file PATH   AGENTS.md to pin (default: repo-root AGENTS.md)
+  --status-file PATH   STATUS.md to pin (required for the backlog test)
+  --model ID           model id (required for mlx; optional override for ornith)
+  --endpoint URL       OpenAI-style base (default: http://127.0.0.1:18080/v1)
+  --out DIR            transcript dir (default: $TMPDIR/harness-runs)
+  --print-prompt       print the assembled prompt to stdout and exit
+EOF
+  exit 2
+}
+
+TEST="${1:-}"; shift || usage
+case "$TEST" in locate|backlog) ;; *) usage ;; esac
+
+RUNTIME="" PRINT_ONLY=0 MODEL="" ENDPOINT="http://127.0.0.1:18080/v1"
+AGENTS_FILE="" STATUS_FILE="" OUT_DIR="${TMPDIR:-/tmp}/harness-runs"
+while [ $# -gt 0 ]; do
+  case "$1" in
+    ornith|mlx|mistral) RUNTIME="$1" ;;
+    --print-prompt) PRINT_ONLY=1 ;;
+    --agents-file) AGENTS_FILE="$2"; shift ;;
+    --status-file) STATUS_FILE="$2"; shift ;;
+    --model) MODEL="$2"; shift ;;
+    --endpoint) ENDPOINT="$2"; shift ;;
+    --out) OUT_DIR="$2"; shift ;;
+    *) usage ;;
+  esac
+  shift
+done
+[ "$PRINT_ONLY" = 1 ] || [ -n "$RUNTIME" ] || usage
+
+HERE="$(cd "$(dirname "$0")/.." && pwd)"
+REPO_ROOT="$(git -C "$HERE" rev-parse --show-toplevel)"
+[ -n "$AGENTS_FILE" ] || AGENTS_FILE="$REPO_ROOT/AGENTS.md"
+[ -f "$AGENTS_FILE" ] || { echo "missing AGENTS.md: $AGENTS_FILE" >&2; exit 1; }
+
+case "$TEST" in
+  locate)
+    PROMPT_HEAD="You are in the erp repo; using AGENTS.md alone, state where the atom registry, the class skeletons and the environment rules live."
+    ;;
+  backlog)
+    [ -f "${STATUS_FILE:-}" ] || { echo "backlog test requires --status-file" >&2; exit 1; }
+    PROMPT_HEAD="You are a cold reader auditing the Arcodange AI back-office backlog. Using ONLY the two documents below — no other knowledge, no tools — answer:
+1. What shipped most recently?
+2. What should be worked on next, and why that item?
+3. What would you verify before trusting these documents, and in what order?"
+    ;;
+esac
+
+PROMPT="$PROMPT_HEAD
+
+--- AGENTS.md ---
+$(cat "$AGENTS_FILE")"
+if [ "$TEST" = backlog ]; then
+  PROMPT="$PROMPT
+
+--- STATUS.md ---
+$(cat "$STATUS_FILE")"
+fi
+
+if [ "$PRINT_ONLY" = 1 ]; then
+  printf '%s\n' "$PROMPT"
+  exit 0
+fi
+
+mkdir -p "$OUT_DIR"
+TS="$(date +%Y%m%dT%H%M%S)"
+OUT_FILE="$OUT_DIR/${TEST}-${RUNTIME}${MODEL:+-$(basename "$MODEL")}-$TS.json"
+START="$(date +%s)"
+
+case "$RUNTIME" in
+  ornith|mlx)
+    if [ "$RUNTIME" = ornith ]; then MODEL="${MODEL:-leonsarmiento/Ornith-1.0-35B-5bit-mlx}"; fi
+    [ -n "$MODEL" ] || { echo "mlx runtime requires --model" >&2; exit 1; }
+    RESPONSE="$(PROMPT="$PROMPT" MODEL="$MODEL" python3 - "$ENDPOINT" <<'PY'
+import json, os, sys, urllib.request
+body = json.dumps({
+    "model": os.environ["MODEL"],
+    "messages": [{"role": "user", "content": os.environ["PROMPT"]}],
+    "temperature": 0,
+    "max_tokens": 3000,
+}).encode()
+req = urllib.request.Request(sys.argv[1].rstrip("/") + "/chat/completions",
+                             data=body, headers={"Content-Type": "application/json"})
+with urllib.request.urlopen(req, timeout=900) as r:
+    print(json.load(r)["choices"][0]["message"]["content"])
+PY
+)"
+    ;;
+  mistral)
+    # Plain -p, no tool filtering: --enabled-tools with a no-match pattern hangs
+    # vibe 2.21.0. --max-turns 1 makes tool use moot for a pure-answer test.
+    MODEL="vibe-active-model"
+    RESPONSE="$(vibe -p "$PROMPT" --max-turns 1 --output text)"
+    ;;
+esac
+
+LATENCY=$(( $(date +%s) - START ))
+RESPONSE="$RESPONSE" PROMPT="$PROMPT" TEST="$TEST" RUNTIME="$RUNTIME" MODEL="$MODEL" \
+ENDPOINT="$ENDPOINT" LATENCY="$LATENCY" TS="$TS" AGENTS_FILE="$AGENTS_FILE" STATUS_FILE="${STATUS_FILE:-}" \
+python3 - > "$OUT_FILE" <<'PY'
+import hashlib, json, os
+def sha(p):
+    return hashlib.sha256(open(p, "rb").read()).hexdigest() if p and os.path.exists(p) else None
+e = os.environ
+inputs = {"AGENTS.md": {"path": e["AGENTS_FILE"], "sha256": sha(e["AGENTS_FILE"])}}
+if e["STATUS_FILE"]:
+    inputs["STATUS.md"] = {"path": e["STATUS_FILE"], "sha256": sha(e["STATUS_FILE"])}
+print(json.dumps({
+    "test": e["TEST"], "runtime": e["RUNTIME"], "model": e["MODEL"],
+    "endpoint": e["ENDPOINT"] if e["RUNTIME"] != "mistral" else "vibe -p",
+    "timestamp": e["TS"], "latency_s": int(e["LATENCY"]),
+    "prompt_sha256": hashlib.sha256(e["PROMPT"].encode()).hexdigest(),
+    "inputs": inputs, "response": e["RESPONSE"],
+}, indent=2, ensure_ascii=False))
+PY
+echo "$OUT_FILE"
@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+# Scoped builder bench / recurring-task shell: run `vibe -p` inside a linked
+# worktree with hard caps and a JSON journal. Refuses to run in the trunk.
+# See fleet/harness/README.md (builder bench protocol, safety bounds).
+set -euo pipefail
+
+usage() {
+  cat >&2 <<'EOF'
+usage: vibe-builder.sh <worktree-dir> <prompt-file> [--max-turns N] [--max-price DOLLARS] [--out DIR]
+
+Runs: vibe -p "$(cat prompt-file)" --auto-approve --max-turns N --max-price D --output json
+inside <worktree-dir>, which MUST be a linked git worktree (never the trunk).
+Journal: <out>/builder-<worktree-name>-<ts>.json (+ .meta.json with wall-clock and exit code).
+Defaults: --max-turns 60, --max-price 3, --out $TMPDIR/harness-runs.
+EOF
+  exit 2
+}
+
+WORKTREE="${1:-}"; PROMPT_FILE="${2:-}"
+[ -d "$WORKTREE" ] && [ -f "${PROMPT_FILE:-}" ] || usage
+shift 2
+MAX_TURNS=60 MAX_PRICE=3 OUT_DIR="${TMPDIR:-/tmp}/harness-runs"
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --max-turns) MAX_TURNS="$2"; shift ;;
+    --max-price) MAX_PRICE="$2"; shift ;;
+    --out) OUT_DIR="$2"; shift ;;
+    *) usage ;;
+  esac
+  shift
+done
+
+# Structural guard: a linked worktree has .git as a FILE (gitdir pointer);
+# the trunk has .git as a directory. Same never-the-trunk guarantee as dol-write.sh.
+if [ ! -f "$WORKTREE/.git" ]; then
+  echo "REFUSED: $WORKTREE is not a linked git worktree (trunk is reserved for the user)" >&2
+  exit 1
+fi
+
+mkdir -p "$OUT_DIR"
+TS="$(date +%Y%m%dT%H%M%S)"
+NAME="$(basename "$WORKTREE")"
+JOURNAL="$OUT_DIR/builder-$NAME-$TS.json"
+META="$OUT_DIR/builder-$NAME-$TS.meta.json"
+
+START="$(date +%s)"
+set +e
+(cd "$WORKTREE" && vibe -p "$(cat "$PROMPT_FILE")" --auto-approve \
+    --max-turns "$MAX_TURNS" --max-price "$MAX_PRICE" --output json) > "$JOURNAL" 2>"$JOURNAL.stderr"
+EXIT_CODE=$?
+set -e
+WALL=$(( $(date +%s) - START ))
+
+PROMPT_SHA="$(shasum -a 256 "$PROMPT_FILE" | cut -d' ' -f1)"
+cat > "$META" <<EOF
+{
+  "worktree": "$WORKTREE",
+  "prompt_file": "$PROMPT_FILE",
+  "prompt_sha256": "$PROMPT_SHA",
+  "max_turns": $MAX_TURNS,
+  "max_price": $MAX_PRICE,
+  "timestamp": "$TS",
+  "wall_clock_s": $WALL,
+  "exit_code": $EXIT_CODE,
+  "journal": "$JOURNAL"
+}
+EOF
+echo "$META"
+exit "$EXIT_CODE"
@@ -0,0 +1,51 @@
+# Harness portability run — 2026-07-18 (erp#63)
+
+Verifier-parity + builder-bench evidence for [erp#63](https://gitea.arcodange.lab/arcodange-org/erp/issues/63).
+Protocol per [`fleet/harness/README.md`](../../README.md): same assembled prompts
+(sha256-pinned in each transcript), temperature 0, responses judged **blind**
+(runtime identity stripped — `blinded-mapping.json` re-attaches it) by 2
+independent context-free judges per cell (strict + skeptic lens), unanimous on
+all 8 cells (`judging-verdicts-20260718.json`).
+
+## Verifier parity table
+
+| Runtime | locate-test | backlog audit | Latency (s) | Admitted to verifier duty |
+| --- | --- | --- | --- | --- |
+| Claude (`claude-fable-5`, context-free subagent) | PASS | PASS | 11 / 29 | baseline |
+| **Mistral** (`vibe -p`, `mistral-medium-3.5`) | PASS | PASS | 14 / 102 | **yes** |
+| **Ornith** (`leonsarmiento/Ornith-1.0-35B-5bit-mlx`, hermes `127.0.0.1:18080`) | PASS | PASS | 18 / 51 | **yes** |
+| MLX small (`mlx-community/Qwen2.5-7B-Instruct-4bit`) | FAIL (environment rules incomplete) | FAIL (Q1: anchored on the PRD-authoring PR-log row, missed the #38 ship) | 7 / 41 | no |
+
+Two non-Claude verifiers reach verdict parity with the Claude baseline on both
+tests → the cross-family verification rule is codified in the PRD qa-strategy
+(paired factory PR).
+
+**Deviation, stated honestly:** the issue names Qwen3 14B/30B-A3B or
+Mistral Small 3.x as the MLX candidate class; only Qwen2.5-7B-4bit and
+Llama-3.2-3B are resident on the M4 today. The 7B-4bit result is the honest
+small-model floor — the named larger candidates remain to be benched once
+pulled. Ornith (35B) covers the "hermes + local model" verifier claim.
+
+## Builder bench — erp#56 under the Mistral runtime
+
+One tightly-footered issue run end-to-end by `vibe -p` (`mistral-medium-3.5`)
+via [`vibe-builder.sh`](../../bin/vibe-builder.sh), unchanged Execution footer,
+caps `--max-turns 60 --max-price 3`:
+
+| Metric | Value |
+| --- | --- |
+| Completion | both file-side deliverables correct (annual fix + personal-card lane note), committed with a conventional message |
+| Code corrections by the operator | **0** |
+| Interventions | 1, environmental: relinking the shared skill `.env` into the worktree so the acceptance command could run; the builder correctly *reported* the missing credentials instead of fabricating output |
+| Wall-clock | 261 s |
+| Acceptance (`bin/arcodange bank match --since 2026-01-01`) | run post-relink from the builder's branch: **0 bank-UNKNOWN**, no stale gap annotation for the now-recorded AI subscriptions → merged as erp PR #68 |
+
+Full journal: `builder-known-patterns-fix-journal.json` (raw `vibe --output json`);
+prompt: `builder-bench-56-prompt.md`; caps + wall-clock: the `.meta.json`.
+
+## Files
+
+- `{locate,backlog}-<runtime>-<ts>.json` — the 8 verifier transcripts (prompt sha256, input sha256s, response, latency).
+- `blinded-mapping.json` — cell-id → runtime identity (judging was blind).
+- `judging-verdicts-20260718.json` — 16 judge verdicts (2 per cell), item-level notes.
+- `builder-*` — the erp#56 bench artifacts.
@@ -0,0 +1,20 @@
+{
+  "test": "backlog",
+  "runtime": "claude",
+  "model": "claude-fable-5 (context-free subagent)",
+  "endpoint": "Claude Code Agent tool; single Read of the pinned assembled prompt file, no other tools",
+  "timestamp": "20260718T1850",
+  "latency_s": null,
+  "prompt_sha256": "28b6bbaabb441f4b4c9d7bf7eb1236f14a1689716ee8c615506654a5ca1f8020",
+  "inputs": {
+    "AGENTS.md": {
+      "path": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/harness-portability/AGENTS.md",
+      "sha256": "a77d356e7804d2792b605bef2e9daba0233d93ef791675acda021d2dac9b02a6"
+    },
+    "STATUS.md": {
+      "path": "/private/tmp/claude-502/-Users-gabrielradureau-Work-Arcodange-erp/3f5e7a57-37fd-4fed-b967-3efdc6435a91/scratchpad/inputs/STATUS.md",
+      "sha256": "8a6fcbbd50fc60f3a557260662a478db98622a146c2a4fe03264fe1c76e7da9b"
+    }
+  },
+  "response": "1. **What shipped most recently?** erp#38 — the fleet scaffold — shipped 2026-07-15 via PR erp#62, settling decision D8. It delivered the `fleet/` tree: the atom registry (`fleet/README.md` with the `atom.yaml` contract schema), the 7 class prompt skeletons in `fleet/classes/`, and the worked example `invoice-extract`. This is the first execution-phase ship of the PRD (everything earlier — read skills, sandbox + promote, backups, bank reco, email ingest, telegram-gateway MVP — is the pre-PRD Foundation ledger). STATUS itself was last updated 2026-07-15 to record this. (Note: the backlog map also mentions \"erp#65 client-dossier ops (phase 1 ✅ 07-15)\", so a phase of #65 apparently landed the same day, but #38 is the ship the documents headline and date as the milestone event.)\n\n2. **What should be worked on next, and why.** Per the resume protocol (pick the top unblocked issue of the earliest open milestone, skipping issues whose only open step is a `[HUMAN]` gate), the answer is **erp#39 — golden set + injection fixtures**, which STATUS explicitly names \"**entry issue**\". Why this one: it sits in P1 (the earliest open milestone by the protocol's priority order, due 2026-10-09, currently 🟡 in progress); it was blocked by #38 and is \"now unblocked\" post-ship; and it is the eval half of the harness that everything downstream depends on — the graduation model requires golden-set evals for any atom to earn autonomy, and the QA/closure gates require golden-set regressions and quarantined injection fixtures, so nothing else in P1 (e.g. #40 invoice-extract atom) can be proven done without it. The documents also say an orchestrator may fan out independent lanes in parallel — #51 (Dolibarr FEC/BlockedLog verifications, \"startable today\") and #41–44 (write-skill side, disjoint directories) — but for one session, one lane, #39 is the pick. Note P2's erp#46 has the *hardest* deadline (2026-09-01) but its step 1 is a `[HUMAN]` Qonto-UI gate, which the protocol says to surface in the digest rather than stall on.\n\n3. **What I would verify before trusting these documents, and in what order.** The documents prescribe their own trust order — **live system > code/git log > STATUS > PRD leaves > agent memories/plans** — and warn that any page whose Last Updated predates the newest closed work is suspect. Concretely, in that order:\n   - **Live forge/system first:** the Gitea milestones and issues themselves (owner `arcodange-org`, repo `erp`) — confirm PR erp#62 is actually merged and #38 closed with its acceptance criteria evidenced; confirm #39's \"Blocked by\" footer is really clear; check whether anything shipped *after* 2026-07-15 (STATUS's Last Updated is three days stale relative to today), which could change the entry issue; confirm milestone due dates and open/closed states match the phase tracker.\n   - **Code/git:** that the claimed artifacts exist as described — `fleet/README.md`, `fleet/classes/` (7 skeletons), the `invoice-extract` worked example, `bin/arcodange`, `ops/sandbox/sandbox-lifecycle.sh` — and `git log` corroborates the #62 merge date.\n   - **STATUS internal consistency:** the tracker says \"#38 ✅\" yet the backlog map still lists issues not in the phase rows (#65, #67 under P4, #56/#57 \"Ops (post-replay)\" appear in the map but not the phase table) — reconcile against the live milestone contents; also verify the #65 \"phase 1 ✅ 07-15\" claim, which the headline ship note doesn't mention.\n   - **PRD leaves last:** that pages the ship should have touched (agent-catalog, task-inventory, roadmap) were truth-passed — a leaf whose Last Updated predates 2026-07-15 in the fleet area is suspect per the documents' own rule.\n   - I would *not* rely on memories/plans for any of this; both documents rank them last.\n\nOne caveat as a cold reader: I was given only these two documents, so every claim above (including \"#38 shipped\") is documentation, not observation — items in the first tier of the verification list are exactly the ones I could not check from here.\n"
+}
@@ -0,0 +1,20 @@
+{
+  "test": "backlog",
+  "runtime": "mistral",
+  "model": "vibe-active-model",
+  "endpoint": "vibe -p",
+  "timestamp": "20260718T194815",
+  "latency_s": 102,
+  "prompt_sha256": "89a3a8653ad858dc4e8066289b08ea1144aa5d568dd23108003cd384e9cd4de8",
+  "inputs": {
+    "AGENTS.md": {
+      "path": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/harness-portability/AGENTS.md",
+      "sha256": "a77d356e7804d2792b605bef2e9daba0233d93ef791675acda021d2dac9b02a6"
+    },
+    "STATUS.md": {
+      "path": "/private/tmp/claude-502/-Users-gabrielradureau-Work-Arcodange-erp/3f5e7a57-37fd-4fed-b967-3efdc6435a91/scratchpad/inputs/STATUS.md",
+      "sha256": "8a6fcbbd50fc60f3a557260662a478db98622a146c2a4fe03264fe1c76e7da9b"
+    }
+  },
+  "response": "1. **erp#38 (fleet scaffold)**, shipped 2026-07-15.\n\n2. **erp#39 (golden set + injection fixtures)** — it is the first cold-start entry point listed in STATUS.md, explicitly marked as unblocked post-#38, and aligns with the resume protocol to pick the top unblocked issue of the earliest open milestone (P1).\n\n3. Verify in trust order: **live system → code/git log → STATUS.md → PRD leaves → agent memories/plans**."
+}
@@ -0,0 +1,20 @@
+{
+  "test": "backlog",
+  "runtime": "mlx",
+  "model": "mlx-community/Qwen2.5-7B-Instruct-4bit",
+  "endpoint": "http://127.0.0.1:18080/v1",
+  "timestamp": "20260718T185141",
+  "latency_s": 18,
+  "prompt_sha256": "89a3a8653ad858dc4e8066289b08ea1144aa5d568dd23108003cd384e9cd4de8",
+  "inputs": {
+    "AGENTS.md": {
+      "path": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/harness-portability/AGENTS.md",
+      "sha256": "a77d356e7804d2792b605bef2e9daba0233d93ef791675acda021d2dac9b02a6"
+    },
+    "STATUS.md": {
+      "path": "/private/tmp/claude-502/-Users-gabrielradureau-Work-Arcodange-erp/3f5e7a57-37fd-4fed-b967-3efdc6435a91/scratchpad/inputs/STATUS.md",
+      "sha256": "8a6fcbbd50fc60f3a557260662a478db98622a146c2a4fe03264fe1c76e7da9b"
+    }
+  },
+  "response": "1. **What shipped most recently?**\n   - The most recent shipment was on 2026-07-11, as documented in the PR log. Specifically, PRD authored: hub + task inventory + agent architecture + model fleet + challenges + POC plan + QA strategy.\n\n2. **What should be worked on next, and why that item?**\n   - Based on the phase tracker and backlog map, the next item to work on should be **erp#39**. This is because it is the top unblocked issue of the earliest open milestone (P1 flagship). The issue body should carry everything needed to start, and since it is unblocked, it is ready to be picked up for work.\n\n3. **What would you verify before trusting these documents, and in what order?**\n   - Before trusting these documents, the following verifications should be performed in the order listed:\n     1. **Verify the live system**: Check the current state of the system to ensure it matches the documentation.\n     2. **Verify the code and git log**: Check the codebase and the commit history to ensure they match the documentation.\n     3. **Verify the current STATUS document**: Check the `STATUS.md` document to ensure it matches the current state of the system and code.\n     4. **Verify the PRD leaves**: Check the PRD documentation to ensure it matches the current state of the system, code, and `STATUS.md`.\n     5. **Verify the agent memories and plans**: Check any agent memories or plans to ensure they match the current state of the system, code, and `STATUS.md`."
+}
@@ -0,0 +1,20 @@
+{
+  "test": "backlog",
+  "runtime": "ornith",
+  "model": "leonsarmiento/Ornith-1.0-35B-5bit-mlx",
+  "endpoint": "http://127.0.0.1:18080/v1",
+  "timestamp": "20260718T185050",
+  "latency_s": 51,
+  "prompt_sha256": "28b6bbaabb441f4b4c9d7bf7eb1236f14a1689716ee8c615506654a5ca1f8020",
+  "inputs": {
+    "AGENTS.md": {
+      "path": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/harness-portability/AGENTS.md",
+      "sha256": "a77d356e7804d2792b605bef2e9daba0233d93ef791675acda021d2dac9b02a6"
+    },
+    "STATUS.md": {
+      "path": "/private/tmp/claude-502/-Users-gabrielradureau-Work-Arcodange-erp/3f5e7a57-37fd-4fed-b967-3efdc6435a91/scratchpad/inputs/STATUS.md",
+      "sha256": "8a6fcbbd50fc60f3a557260662a478db98622a146c2a4fe03264fe1c76e7da9b"
+    }
+  },
+  "response": "\n\nBased strictly on the two provided documents:\n\n### 1. What shipped most recently?\n**`erp#38` (fleet scaffold)** shipped on **2026-07-15** via PR `erp#62`. It settled D8 and is the only milestone row marked `✅` in the phase tracker. The status header explicitly calls it out: *\"execution started — erp#38 shipped, D8 settled (2026-07-15)\"*.\n\n### 2. What should be worked on next, and why?\n**`erp#39` (golden set + injection fixtures)** is the designated next item.\n\n**Why:**\n- **Milestone ordering:** The resume protocol mandates picking the top unblocked issue of the *earliest open milestone*. P1 (due 2026-10-09) is the earliest open, and #38 is the only completed P1 issue.\n- **Explicit cold-start label:** The backlog map states: *\"Cold-start entry points as of 2026-07-15 (post-#38): **erp#39** (golden set — the eval half of the harness, now unblocked)\"*.\n- **Unblocked & independent:** It has no open `[HUMAN]` gates blocking it, and the arbitration rule explicitly allows it to run in parallel with `#51` and `#41–44` because they *\"touch disjoint directories\"*. However, #39 is the canonical P1 successor and the eval harness it builds is a prerequisite for the autonomy ladder and graduation protocol described in `AGENTS.md`.\n\n*(Note: Per the arbitration rule, a session could legitimately fan out `#39 ∥ #51 ∥ #41–44` in parallel, but #39 is the primary next pick.)*\n\n### 3. What would you verify before trusting these documents, and in what order?\nThe documents explicitly define a **trust order** for versionable claims. I would audit in that exact sequence, mapping each tier to concrete verification actions:\n\n| Trust Tier | What to verify | How (per doc context) |\n|------------|----------------|------------------------|\n| **1. Live system** | Actual state of the ERP, sandbox, CLI, and issue #38's deliverables | Hit `erp.arcodange.lab` and `erp-sandbox.arcodange.lab`; run `bin/arcodange` read commands; confirm the fleet scaffold actually exists and functions; verify `ai_agent`/`ai_agent_sandbox` keys behave as described (read-only prod, host-guarded sandbox). |\n| **2. Code / Git log** | Merge state, branch reality, directory layout, milestone closure | Check Gitea for PR `erp#62` (merged? closed?). Inspect `fleet/`, `fleet/classes/`, `atom.yaml` registry, and the P1 milestone state. Confirm `erp#38` is actually closed and its deliverables match the issue body. |\n| **3. STATUS.md** | Internal consistency, date accuracy, backlog map fidelity | Verify `Last Updated: 2026-07-15` matches the newest closed milestone. Cross-check the phase tracker row for P1 against Gitea. Confirm the backlog map's open/closed flags match actual issue states. |\n| **4. PRD leaves** | Cross-document consistency | Read the linked PRD files (`compliance.md`, `agent-architecture.md`, `agent-catalog.md`, `qa-strategy.md`, `roadmap.md`, `poc-plan.md`). Check for stale claims, broken anchors, or contradictions with STATUS (e.g., `not yet` claims that should now be `✅`). |\n| **5. Memories / plans** | Lowest priority | Treat any session memory or plan as untrusted until tiers 1–4 confirm it. |\n\n**Additional doc-surface checks** (from the closure protocol, §6):\n- Run a **fresh-reader smoke test**: a context-free reader of only `STATUS.md` + repo `AGENTS.md` must"
+}
@@ -0,0 +1,50 @@
+{
+  "cell-01": {
+    "file": "backlog-claude-baseline-20260718.json",
+    "runtime": "claude",
+    "model": "claude-fable-5 (context-free subagent)",
+    "latency_s": null
+  },
+  "cell-02": {
+    "file": "backlog-mistral-20260718T194815.json",
+    "runtime": "mistral",
+    "model": "vibe-active-model",
+    "latency_s": 102
+  },
+  "cell-03": {
+    "file": "backlog-mlx-Qwen2.5-7B-Instruct-4bit-20260718T185141.json",
+    "runtime": "mlx",
+    "model": "mlx-community/Qwen2.5-7B-Instruct-4bit",
+    "latency_s": 18
+  },
+  "cell-04": {
+    "file": "backlog-ornith-20260718T185050.json",
+    "runtime": "ornith",
+    "model": "leonsarmiento/Ornith-1.0-35B-5bit-mlx",
+    "latency_s": 51
+  },
+  "cell-05": {
+    "file": "locate-claude-baseline-20260718.json",
+    "runtime": "claude",
+    "model": "claude-fable-5 (context-free subagent)",
+    "latency_s": null
+  },
+  "cell-06": {
+    "file": "locate-mistral-20260718T194754.json",
+    "runtime": "mistral",
+    "model": "vibe-active-model",
+    "latency_s": 14
+  },
+  "cell-07": {
+    "file": "locate-mlx-Qwen2.5-7B-Instruct-4bit-20260718T184831.json",
+    "runtime": "mlx",
+    "model": "mlx-community/Qwen2.5-7B-Instruct-4bit",
+    "latency_s": 7
+  },
+  "cell-08": {
+    "file": "locate-ornith-20260718T185032.json",
+    "runtime": "ornith",
+    "model": "leonsarmiento/Ornith-1.0-35B-5bit-mlx",
+    "latency_s": 18
+  }
+}
@@ -0,0 +1,30 @@
+You are a builder agent working ONE issue of the Arcodange erp backlog, end-to-end, unattended.
+
+Ground rules:
+- Your working directory is an isolated git worktree of the erp repo, on branch `arcodange/known-patterns-fix`. Work ONLY inside it.
+- Read AGENTS.md at the repo root first (orientation + operating rules), then the SKILL.md of anything you touch. A change that makes a documented claim false updates that doc in the same commit.
+- Commit your changes on the current branch with a conventional-commit message (e.g. `fix(bank-reco): ...`). Do NOT push. Do NOT open a PR. Do NOT post to Gitea.
+- Skill credential .env files are gitignored and absent from this worktree; if a command needs one, the operator's copies live at /Users/gabrielradureau/Work/Arcodange/erp/.claude/skills/<skill>/.env (read-only APIs). If a credentialed command cannot run, report that honestly in EVIDENCE instead of fabricating output.
+- End your final message with a section titled `EVIDENCE:` containing (1) what changed and why, (2) the output of `git log --oneline -3` and `git diff HEAD~1 --stat`, (3) the output (or honest failure) of the acceptance command.
+
+The issue you are executing — erp#56 (verbatim):
+---
+Title: fix(bank-reco): known-patterns.json — Mistral is an ANNUAL subscription, not monthly
+
+Small correction from the 2026-07-11 regularization (source: the actual invoice PDF `MSTRL-API-814045-001`, 2026-04-02, "Abonnement annuel - Le Chat Pro - Annual", 143,90 HT / 172,68 TTC).
+
+`.claude/skills/arcodange-bank-reco/known-patterns.json` currently documents the MISTRAL.AI card debit as "Récurrent mensuel" — it is **annual** (next expected ~2027-04). This mis-set expectation cost investigation time ("where are the May/June Mistral debits?" — there are none).
+
+While in there:
+- Anthropic (CLAUDE.AI SUBSCRIPTION) pattern: April was on the Qonto card, but the **May/June receipts (#2344-8391, #2997-4837 in books@) have no Qonto debit** — probably the personal card (fk_account=3, API-invisible). Document the pattern as "payment rail moved — personal-card lane, see #57".
+- Both AI subscriptions are now RECORDED supplier invoices (post-replay), so these patterns may graduate from "known bank-only gap" to "matched" — re-check what `bank match` reports after the prod replay and prune accordingly.
+
+**Acceptance:** `bank match --since 2026-01-01` after the prod replay shows no stale "intentional gap" annotations for entries that are now properly recorded.
+
+---
+**Execution** — Blocked by: the **prod replay** of the 2026-07-11 pack (the re-check depends on it; the annual-vs-monthly text fix itself can land anytime) · Blocks: none · Human gates: none.
+**Start:** `claude --worktree known-patterns-fix` from the erp repo root; edit `.claude/skills/arcodange-bank-reco/known-patterns.json`, then (post-replay) run `bank match --since 2026-01-01`.
+**Done means:** corrected patterns + post-replay match output posted here → PR merged with `Closes #56`.
+---
+
+Note on the "Blocked by" line: the prod replay of the 2026-07-11 pack was executed on 2026-07-15 (erp PR #64), so the re-check is unblocked. `bank match` is run as `bin/arcodange bank match --since 2026-01-01`.
@@ -0,0 +1,11 @@
+{
+  "worktree": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/known-patterns-fix",
+  "prompt_file": "/private/tmp/claude-502/-Users-gabrielradureau-Work-Arcodange-erp/3f5e7a57-37fd-4fed-b967-3efdc6435a91/scratchpad/bench-56-prompt.md",
+  "prompt_sha256": "97b46fa984c9b63a8b95a578d2eec2b44c177d045df30b6fe09250824798de5a",
+  "max_turns": 60,
+  "max_price": 3,
+  "timestamp": "20260718T184946",
+  "wall_clock_s": 261,
+  "exit_code": 0,
+  "journal": "/private/tmp/claude-502/-Users-gabrielradureau-Work-Arcodange-erp/3f5e7a57-37fd-4fed-b967-3efdc6435a91/scratchpad/harness-runs/builder-known-patterns-fix-20260718T184946.json"
+}
@@ -0,0 +1,378 @@
+[
+  {
+    "cell": "cell-01",
+    "judges": [
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "Q1 — shipped: names the most recent ✅ item STATUS records",
+            "correct": true,
+            "note": "Correctly identifies erp#38 fleet scaffold, 2026-07-15, PR erp#62, D8 settled — not an older phase or open issue; even flags the secondary #65 phase-1 mention."
+          },
+          {
+            "item": "Q2 — next: applies resume protocol (top unblocked issue, earliest open milestone, skip [HUMAN]-gated)",
+            "correct": true,
+            "note": "Picks erp#39 (STATUS's named entry issue, unblocked post-#38) and explicitly surfaces P2's erp#46 [HUMAN] Qonto-UI gate rather than stalling on it, despite its earlier 2026-09-01 due date."
+          },
+          {
+            "item": "Q3 — trust: states trust order in the right direction and proposes Last-Updated / live-repo checks",
+            "correct": true,
+            "note": "States live system > code/git log > STATUS > PRD leaves > memories verbatim, orders concrete verification steps accordingly, and checks Last Updated staleness against today."
+          }
+        ],
+        "notes": "Substantively correct on all three rubric questions. No trust-order inversion, no [HUMAN]-gated issue proposed as next without flagging the gate, no shipped-work claims the documents do not support. The only nit — framing P1 as earliest by \"priority order\" rather than strictly by due date — does not invert the protocol, since the earlier-due P2 milestone's sole issue (#46) is the [HUMAN]-gated one the protocol says to skip-and-surface, which the response does explicitly. Per the pass rule, minor framing differences that do not invert the protocol do not fail."
+      },
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "Q1 — shipped",
+            "correct": true,
+            "note": "Names erp#38 fleet scaffold, 2026-07-15, PR erp#62, D8 settled — exactly what STATUS records; correctly separates pre-PRD Foundation ledger and even surfaces the #65 phase-1 same-day note; no unsupported ship claims (fleet/ contents match the AGENTS.md map row)."
+          },
+          {
+            "item": "Q2 — next",
+            "correct": true,
+            "note": "Picks erp#39 (STATUS's explicit 'entry issue'), notes it was unblocked by #38, and explicitly surfaces — not stalls on — P2 erp#46's earlier 2026-09-01 deadline as [HUMAN]-gated; its aside calling P1 'earliest by priority order' (P2 is earlier by due date) is a minor mislabel that does not invert the protocol since the [HUMAN]-gate skip is applied correctly and the outcome matches."
+          },
+          {
+            "item": "Q3 — trust",
+            "correct": true,
+            "note": "States the exact trust order (live system > code/git log > STATUS > PRD leaves > memories), works through it in that order concretely (verify #62 merged, artifacts exist, Last Updated staleness, leaf truth-pass), and flags real STATUS internal inconsistencies (#65/#67/#56/#57 in map but not phase table)."
+          }
+        ],
+        "notes": "Skeptic pass could not refute: no protocol inversion, no [HUMAN]-gated issue proposed as next, no fabricated ships. The one attackable point — describing P1 as \"the earliest open milestone by the protocol's priority order\" when the rubric frames ordering by due date (P2 due 2026-09-01 < P1 due 2026-10-09) — is defused by the response itself, which explicitly notes P2's harder deadline and that #46's only open step is a [HUMAN] Qonto-UI gate to be surfaced in the digest, i.e. it applies the skip rule and lands on the correct answer. \"Three days stale relative to today\" assumes a current date not in the documents, but this is a harmless aside, not a rubric item. All three questions substantively correct → PASS per the pass rule."
+      }
+    ]
+  },
+  {
+    "cell": "cell-02",
+    "judges": [
+      {
+        "items": [
+          {
+            "item": "Q1 — shipped: names the most recent ✅ item STATUS records",
+            "correct": true,
+            "note": "erp#38 fleet scaffold, 2026-07-15 — matches STATUS phase tracker and backlog map exactly; not an older phase or open issue."
+          },
+          {
+            "item": "Q2 — next: applies resume protocol, skips [HUMAN]-gated issues",
+            "correct": true,
+            "note": "Picks erp#39, STATUS's explicit entry issue, as top unblocked post-#38; never proposes gated erp#46; 'earliest milestone = P1' is a minor imprecision (P2 is earlier but gated) that does not invert the protocol."
+          },
+          {
+            "item": "Q3 — trust: correct trust order + verification proposal",
+            "correct": true,
+            "note": "States live system > code/git log > STATUS > PRD leaves > memories in the right direction, framed as what to verify in order; Last Updated unmentioned but verifying against the live system satisfies the rubric's disjunctive second half."
+          }
+        ],
+        "verdict": "PASS",
+        "notes": "All three answers are substantively correct per the rubric. The response is terse but names the exact ground-truth items: erp#38 as most recent ship, erp#39 as next (the STATUS-designated entry issue, with resume-protocol reasoning), and the trust order stated in the correct direction. None of the enumerated fail conditions apply: no trust-order inversion, no blocked/[HUMAN]-gated issue proposed as next, no unsupported shipped-work claim. The only imperfections — calling P1 the earliest open milestone (P2 is earlier by due date but its entry step is operator-gated, hence correctly skipped) and not explicitly mentioning Last Updated — are minor omissions that do not invert the protocol, which the pass rule explicitly tolerates."
+      },
+      {
+        "items": [
+          {
+            "item": "Q1 — shipped: names most recent ✅ item in STATUS",
+            "correct": true,
+            "note": "erp#38 fleet scaffold, 2026-07-15 — exactly what STATUS records as the latest shipped item (PR erp#62); no unsupported claims."
+          },
+          {
+            "item": "Q2 — next: resume protocol applied, unblocked issue, [HUMAN] gates not stalled on",
+            "correct": true,
+            "note": "erp#39 is STATUS's explicit entry issue, unblocked post-#38; does not propose the [HUMAN]-gated #46; 'earliest milestone (P1)' matches AGENTS.md's own ordering, and not enumerating the #46 skip is a minor omission, not an inversion."
+          },
+          {
+            "item": "Q3 — trust: correct trust order + verification proposal",
+            "correct": true,
+            "note": "States live system > code/git log > STATUS > PRD leaves > memories verbatim, right direction; 'verify in trust order: live system first' proposes checking against the live system; omitting Last Updated check is minor, no inversion."
+          }
+        ],
+        "verdict": "PASS",
+        "notes": "Skeptic pass could not refute: all three answers are substantively correct per the rubric. Attack vectors tried and rejected: (1) P2 has an earlier due date than P1, but its sole issue #46 is [HUMAN]-gated so #39 is the correct next item either way, and AGENTS.md itself lists P1 first in the resume sequence; (2) the response does not flag #46's gate, but the rubric's fail condition is proposing a gated issue as next, which did not occur; (3) no Last Updated check mentioned, but the trust order is stated correctly and un-inverted, and the rubric treats such omissions as minor. Terseness is explicitly not a defect."
+      }
+    ]
+  },
+  {
+    "cell": "cell-03",
+    "judges": [
+      {
+        "verdict": "FAIL",
+        "items": [
+          {
+            "item": "Q1 — shipped most recently",
+            "correct": false,
+            "note": "Names the 2026-07-11 PRD authoring (PR log) as most recent; STATUS records erp#38 fleet scaffold shipped 2026-07-15 (PR erp#62) — the rubric explicitly excludes an older phase/item."
+          },
+          {
+            "item": "Q2 — what next and why",
+            "correct": true,
+            "note": "Picks erp#39 as top unblocked issue of the earliest open milestone (P1), matching STATUS's entry issue; omitting the [HUMAN]-gate skip rule is a minor omission since the chosen issue is not gated and no gated issue was proposed."
+          },
+          {
+            "item": "Q3 — trust verification order",
+            "correct": true,
+            "note": "States live system > code/git log > STATUS > PRD leaves > agent memories/plans in the correct direction and proposes verifying claims against the live system; no inversion."
+          }
+        ],
+        "notes": "Pass rule requires substantive correctness on all three questions. Q1 is substantively wrong: the response anchors on the PR-log entry (2026-07-11 PRD authored) and misses the phase tracker's most recent shipped item, erp#38 fleet scaffold ✅ 2026-07-15 (D8 settled, PR erp#62), which the rubric names as the expected answer and explicitly rules out \"an older phase\". Q2 and Q3 pass cleanly, but one failed question forces the overall FAIL."
+      },
+      {
+        "items": [
+          {
+            "item": "Q1 — shipped most recently",
+            "correct": false,
+            "note": "Cites 2026-07-11 PRD authoring (factory#21) as most recent; STATUS records erp#38 fleet scaffold shipped 2026-07-15 (PR erp#62) in three places — this is the 'older phase' error the rubric disqualifies."
+          },
+          {
+            "item": "Q2 — what to work on next",
+            "correct": true,
+            "note": "Names erp#39, the STATUS-designated entry issue, correctly as top unblocked issue of the earliest open milestone (P1); omits restating the [HUMAN]-gate skip rule but does not propose a gated/blocked issue, so no protocol inversion."
+          },
+          {
+            "item": "Q3 — trust/verification order",
+            "correct": true,
+            "note": "States the exact trust order live system > code/git log > STATUS > PRD leaves > memories in the right direction and proposes verifying claims against the live system; omits Last Updated but the rubric accepts either check."
+          }
+        ],
+        "verdict": "FAIL",
+        "notes": "Pass rule requires substantive correctness on all three questions. Q1 fails: the response treats the PR-log entry (2026-07-11 PRD authoring) as the latest shipment while ignoring the STATUS banner, phase tracker, and backlog map all recording erp#38 (fleet scaffold, 2026-07-15, PR erp#62) as the most recent shipped item — precisely the stale-claim failure the cold-reader test exists to catch. Q2 and Q3 are substantively correct, but one wrong answer forces overall FAIL."
+      }
+    ]
+  },
+  {
+    "cell": "cell-04",
+    "judges": [
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "Q1 — shipped: names the most recent ✅ item STATUS records",
+            "correct": true,
+            "note": "Correctly identifies erp#38 fleet scaffold, 2026-07-15, PR erp#62, D8 settled — matches STATUS header and P1 row; no unsupported shipped claims."
+          },
+          {
+            "item": "Q2 — next: applies resume protocol, skips [HUMAN]-gated issues",
+            "correct": true,
+            "note": "Picks erp#39 as top unblocked issue of earliest open milestone (P1, due 2026-10-09), cites the entry-issue designation, notes parallel lanes #51/#41-44, and does not stall on or propose the [HUMAN]-gated #46."
+          },
+          {
+            "item": "Q3 — trust: correct trust order + Last Updated / live-repo verification",
+            "correct": true,
+            "note": "States live system > code/git log > STATUS > PRD leaves > memories in the right direction, checks Last Updated against newest closed milestone, and verifies claims against live Gitea/repo; truncation only cuts a bonus section after the required content."
+          }
+        ],
+        "notes": "All three questions substantively correct per the rubric. Q1: erp#38/PR erp#62/2026-07-15 exactly matches STATUS. Q2: erp#39 via the resume protocol, [HUMAN] gate (#46) correctly avoided. Q3: full five-tier trust order in the correct direction with concrete verification steps (Last Updated stamp, PR merge state, fleet/ directory). The response ends mid-sentence in a supplementary \"additional doc-surface checks\" section, but this is a minor omission that does not invert any protocol or drop a required half — per the pass rule, PASS."
+      },
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "Q1 — shipped most recently",
+            "correct": true,
+            "note": "Names erp#38 fleet scaffold, 2026-07-15, PR erp#62, D8 settled — matches STATUS exactly; minor overclaim ('only ✅ row', Phase 0 is also ✅) does not cite unsupported work."
+          },
+          {
+            "item": "Q2 — what next and why",
+            "correct": true,
+            "note": "Correctly picks erp#39 (STATUS's explicit entry issue), confirms it unblocked with no [HUMAN] gate; flaw noted — wrongly calls P1 the earliest open milestone (P2 due 2026-09-01 is earlier) and never surfaces #46's gate, but no fail trigger: no gated issue proposed, no protocol inversion."
+          },
+          {
+            "item": "Q3 — trust order + verification",
+            "correct": true,
+            "note": "States live system > code/git log > STATUS > PRD leaves > memories in the correct direction, proposes checking Last Updated and verifying against Gitea/live repo; response truncates only in bonus material after the required content."
+          }
+        ],
+        "notes": "Skeptic lens applied: the strongest attack is Q2's reasoning error (claims P1 is the earliest open milestone when P2 is due 2026-09-01, and skips #46 silently rather than surfacing its [HUMAN] gate). This is a factual slip in the justification, but the rubric's fail conditions are (a) inverting the trust order, (b) proposing a blocked/[HUMAN]-gated issue as next without flagging the gate, (c) citing unsupported shipped work — none apply: the pick (#39) is the ground-truth answer, gate-awareness is shown, and the trust order is stated correctly. Q1 and Q3 are cleanly correct. Mid-sentence truncation occurs only in an optional add-on section. PASS per the rubric's pass rule."
+      }
+    ]
+  },
+  {
+    "cell": "cell-05",
+    "judges": [
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "atom registry",
+            "correct": true,
+            "note": "Response states `fleet/README.md`, exactly matching ground truth."
+          },
+          {
+            "item": "class skeletons",
+            "correct": true,
+            "note": "Response states `fleet/classes/` and mentions the 7 prompt skeletons, matching ground truth."
+          },
+          {
+            "item": "environment rules",
+            "correct": true,
+            "note": "Both required halves present: the AGENTS.md 'Operating rules for agents' section plus `.claude/skills/dolibarr-sandbox-write/SKILL.md`."
+          }
+        ],
+        "notes": "All three locations are path-level correct per the rubric table. The environment-rules answer includes both mandated halves (operating rules section + sandbox-write SKILL.md). Pass rule (\"all three locations correct\") is satisfied with no wrongness or protocol inversion; verdict PASS."
+      },
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "atom registry",
+            "correct": true,
+            "note": "States fleet/README.md — exact path-level match to ground truth."
+          },
+          {
+            "item": "class skeletons",
+            "correct": true,
+            "note": "States fleet/classes/ and mentions the 7 skeletons — matches ground truth."
+          },
+          {
+            "item": "environment rules",
+            "correct": true,
+            "note": "Both required halves present: the AGENTS.md 'Operating rules for agents' section AND .claude/skills/dolibarr-sandbox-write/SKILL.md."
+          }
+        ],
+        "notes": "Skeptic review found no wrong paths, no protocol inversion, no missing halves. All three locations match the rubric table exactly; surrounding prose is faithful paraphrase of AGENTS.md, not invention. Pass rule (all three correct, both environment-rules halves) is fully satisfied."
+      }
+    ]
+  },
+  {
+    "cell": "cell-06",
+    "judges": [
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "atom registry",
+            "correct": true,
+            "note": "States `fleet/README.md`, exact match with ground truth."
+          },
+          {
+            "item": "class skeletons",
+            "correct": true,
+            "note": "States `fleet/classes/`, path-level match; '(7 skeleton files)' gloss not required by pass rule."
+          },
+          {
+            "item": "environment rules",
+            "correct": true,
+            "note": "Both required halves present: the operating-rules section (anchor #operating-rules-for-agents) + `.claude/skills/dolibarr-sandbox-write/SKILL.md`."
+          }
+        ],
+        "notes": "All three locations correct at path level, and the environment-rules answer includes both mandated halves, satisfying the pass rule exactly. The response is terse but terseness is not a defect; content mirrors AGENTS.md lines 36-38 accurately."
+      },
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "atom registry",
+            "correct": true,
+            "note": "States `fleet/README.md`, exactly matching ground truth."
+          },
+          {
+            "item": "class skeletons",
+            "correct": true,
+            "note": "States `fleet/classes/`, correct at path level (7-file count not required by pass rule)."
+          },
+          {
+            "item": "environment rules",
+            "correct": true,
+            "note": "Both required halves present: the operating-rules section (anchor #operating-rules-for-agents in AGENTS.md) + `.claude/skills/dolibarr-sandbox-write/SKILL.md`."
+          }
+        ],
+        "notes": "Skeptic lens found no refutation: all three locations match the ground-truth table path-for-path, and the environment-rules answer includes both mandatory halves. Terseness is explicitly not a defect per instructions. Pass rule (all three correct) is satisfied."
+      }
+    ]
+  },
+  {
+    "cell": "cell-07",
+    "judges": [
+      {
+        "verdict": "FAIL",
+        "items": [
+          {
+            "item": "atom registry",
+            "correct": true,
+            "note": "Response states `fleet/README.md`, matching ground truth exactly."
+          },
+          {
+            "item": "class skeletons",
+            "correct": true,
+            "note": "Response states `fleet/classes/` (7 skeletons), matching ground truth exactly."
+          },
+          {
+            "item": "environment rules",
+            "correct": false,
+            "note": "SKILL.md half is present, but the operating-rules half is mislocated: the response claims the agent operating rules are 'detailed in `fleet/README.md`' instead of the AGENTS.md operating-rules section, so the required first half never appears correctly."
+          }
+        ],
+        "notes": "Pass rule requires all three locations correct, and for environment rules both halves must appear. The response nails items 1 and 2, but for item 3 it substitutes `fleet/README.md` for the AGENTS.md operating-rules section — a wrong path, not a wording variance — so the environment-rules answer is only half right. Per the rule as written, one incorrect item forces FAIL."
+      },
+      {
+        "verdict": "FAIL",
+        "items": [
+          {
+            "item": "atom registry",
+            "correct": true,
+            "note": "States `fleet/README.md`, matching ground truth exactly."
+          },
+          {
+            "item": "class skeletons",
+            "correct": true,
+            "note": "States `fleet/classes/` with the 7 prompt skeletons, matching ground truth."
+          },
+          {
+            "item": "environment rules",
+            "correct": false,
+            "note": "Second half (`.claude/skills/dolibarr-sandbox-write/SKILL.md`) is present, but the first half is wrong: the response claims the operating rules live in `fleet/README.md` instead of the AGENTS.md operating-rules section — the required AGENTS.md half never appears and is actively misattributed."
+          }
+        ],
+        "notes": "The response nails the two easy items but botches the composite third. Ground truth requires BOTH halves for environment rules: (a) the AGENTS.md operating rules section and (b) the sandbox-write SKILL.md. The response substitutes `fleet/README.md` for the operating-rules half (\"The operating rules for agents are detailed in fleet/README.md\"), which is not a wording variance but a wrong path — AGENTS.md is never named as the home of the operating rules. Per the pass rule (all three locations correct, both environment-rules halves required), this is a FAIL."
+      }
+    ]
+  },
+  {
+    "cell": "cell-08",
+    "judges": [
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "atom registry",
+            "correct": true,
+            "note": "Response states `fleet/README.md`, matching ground truth exactly."
+          },
+          {
+            "item": "class skeletons",
+            "correct": true,
+            "note": "Response states `fleet/classes/`, matching ground truth at path level."
+          },
+          {
+            "item": "environment rules",
+            "correct": true,
+            "note": "Both required halves present: the '## Operating rules for agents' section of AGENTS.md plus `.claude/skills/dolibarr-sandbox-write/SKILL.md`."
+          }
+        ],
+        "notes": "All three ground-truth locations are correct at path level and the environment-rules answer includes both required halves, so the pass rule (all three correct) is satisfied. The response is concise but completeness is not penalized under the rubric."
+      },
+      {
+        "verdict": "PASS",
+        "items": [
+          {
+            "item": "atom registry -> fleet/README.md",
+            "correct": true,
+            "note": "Response states `fleet/README.md` exactly, matching the ground truth path."
+          },
+          {
+            "item": "class skeletons -> fleet/classes/",
+            "correct": true,
+            "note": "Response states `fleet/classes/`, matching the ground truth path (the '7 files' detail is not required at path level)."
+          },
+          {
+            "item": "environment rules -> AGENTS.md operating-rules section + .claude/skills/dolibarr-sandbox-write/SKILL.md",
+            "correct": true,
+            "note": "Both required halves appear: the '## Operating rules for agents' section in this file plus `.claude/skills/dolibarr-sandbox-write/SKILL.md`."
+          }
+        ],
+        "notes": "Skeptic pass found nothing to refute: all three locations are path-correct, the two-half requirement for environment rules is satisfied, and the response contains no wrong paths, fabricated locations, or protocol inversions. Terse but complete; per the pass rule (all three correct) the verdict is PASS."
+      }
+    ]
+  }
+]
@@ -0,0 +1,16 @@
+{
+  "test": "locate",
+  "runtime": "claude",
+  "model": "claude-fable-5 (context-free subagent)",
+  "endpoint": "Claude Code Agent tool; single Read of the pinned assembled prompt file, no other tools",
+  "timestamp": "20260718T1850",
+  "latency_s": null,
+  "prompt_sha256": "6cb373e229d4d58965e5e5b718400178afbbf83f02eed59f2fab5895df1edc0f",
+  "inputs": {
+    "AGENTS.md": {
+      "path": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/harness-portability/AGENTS.md",
+      "sha256": "a77d356e7804d2792b605bef2e9daba0233d93ef791675acda021d2dac9b02a6"
+    }
+  },
+  "response": "Per AGENTS.md:\n\n- **Atom registry**: `fleet/README.md` — defines what an atom is, the `atom.yaml` contract schema field by field, and the `fleet/` layout. An atom absent from the registry does not run.\n- **Class skeletons**: `fleet/classes/` — the 7 prompt skeletons per the PRD agent catalog; every atom's `prompt.md` extends exactly one, and prompts carry no business rules (those live in `fleet/profile/` + validators).\n- **Environment rules**: the \"Operating rules for agents\" section of AGENTS.md itself, plus `.claude/skills/dolibarr-sandbox-write/SKILL.md` (the host-guarded sandbox write path and its promote gate).\n"
+}
@@ -0,0 +1,16 @@
+{
+  "test": "locate",
+  "runtime": "mistral",
+  "model": "vibe-active-model",
+  "endpoint": "vibe -p",
+  "timestamp": "20260718T194754",
+  "latency_s": 14,
+  "prompt_sha256": "b22405e7d0db8915c4aae6eddbc2a003f600be41f605572356fe79a7c1f51c00",
+  "inputs": {
+    "AGENTS.md": {
+      "path": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/harness-portability/AGENTS.md",
+      "sha256": "a77d356e7804d2792b605bef2e9daba0233d93ef791675acda021d2dac9b02a6"
+    }
+  },
+  "response": "- **Atom registry**: `fleet/README.md`\n- **Class skeletons**: `fleet/classes/`\n- **Environment rules**: the [operating rules](#operating-rules-for-agents) section + `.claude/skills/dolibarr-sandbox-write/SKILL.md`"
+}
@@ -0,0 +1,16 @@
+{
+  "test": "locate",
+  "runtime": "mlx",
+  "model": "mlx-community/Qwen2.5-7B-Instruct-4bit",
+  "endpoint": "http://127.0.0.1:18080/v1",
+  "timestamp": "20260718T184831",
+  "latency_s": 7,
+  "prompt_sha256": "6cb373e229d4d58965e5e5b718400178afbbf83f02eed59f2fab5895df1edc0f",
+  "inputs": {
+    "AGENTS.md": {
+      "path": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/harness-portability/AGENTS.md",
+      "sha256": "69206359ba2bea471e7f325684465124f0dae6183bf509ffda5bfae9b596dd15"
+    }
+  },
+  "response": "Based on the `AGENTS.md` file provided, here is where the atom registry, class skeletons, and environment rules live:\n\n- **Atom Registry**: \n  - Path: `fleet/README.md`\n  - Description: Describes what an atom is, the `atom.yaml` contract schema, and the `fleet/` layout. An atom absent from the registry does not run.\n\n- **Class Skeletons**: \n  - Path: `fleet/classes/`\n  - Description: Contains the 7 prompt skeletons per the PRD agent catalog. Every atom's `prompt.md` extends exactly one of these skeletons, and prompts carry no business rules (those live in `fleet/profile/` + validators).\n\n- **Environment Rules**: \n  - Path: `fleet/README.md` (for the atom registry) and `.claude/skills/dolibarr-sandbox-write/SKILL.md` (for the sandbox write path and its promote gate).\n  - Description: The operating rules for agents are detailed in `fleet/README.md`, and the specific rules for the sandbox write path are detailed in `.claude/skills/dolibarr-sandbox-write/SKILL.md`."
+}
@@ -0,0 +1,16 @@
+{
+  "test": "locate",
+  "runtime": "ornith",
+  "model": "leonsarmiento/Ornith-1.0-35B-5bit-mlx",
+  "endpoint": "http://127.0.0.1:18080/v1",
+  "timestamp": "20260718T185032",
+  "latency_s": 18,
+  "prompt_sha256": "6cb373e229d4d58965e5e5b718400178afbbf83f02eed59f2fab5895df1edc0f",
+  "inputs": {
+    "AGENTS.md": {
+      "path": "/Users/gabrielradureau/Work/Arcodange/erp/.claude/worktrees/harness-portability/AGENTS.md",
+      "sha256": "69206359ba2bea471e7f325684465124f0dae6183bf509ffda5bfae9b596dd15"
+    }
+  },
+  "response": "\n\nBased on `AGENTS.md` alone:\n\n- **Atom registry**: `fleet/README.md` (defines what an atom is, the `atom.yaml` contract schema, and the `fleet/` layout)\n- **Class skeletons**: `fleet/classes/`\n- **Environment rules**: The `## Operating rules for agents` section in this file, plus `.claude/skills/dolibarr-sandbox-write/SKILL.md` (the host-guarded sandbox write path and promote gate)"
+}
@@ -0,0 +1,44 @@
+# Verifier test — cold-reader backlog audit
+
+The fresh-reader smoke test from the PRD
+[closure protocol](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/STATUS.md),
+run standalone: a cold reader with only the two orientation documents must
+reconstruct where the work stands. Used for verifier admission (erp#63).
+
+## Inputs
+
+| Document | Pinned source |
+| --- | --- |
+| `AGENTS.md` | the erp repo root, at the commit under test |
+| `STATUS.md` | `vibe/PRD/ai-back-office/STATUS.md` from the **factory** repo, origin/main at run time |
+
+The runtime gets both documents embedded in the prompt and **nothing else**.
+
+## Prompt (verbatim)
+
+> You are a cold reader auditing the Arcodange AI back-office backlog. Using ONLY
+> the two documents below — no other knowledge, no tools — answer:
+> 1. What shipped most recently?
+> 2. What should be worked on next, and why that item?
+> 3. What would you verify before trusting these documents, and in what order?
+
+`bin/run-verifier.sh backlog <runtime> --status-file <path>` assembles this
+prompt with both documents appended under `--- AGENTS.md ---` and
+`--- STATUS.md ---` delimiters.
+
+## Rubric
+
+| Question | Substantively correct means |
+| --- | --- |
+| 1 — shipped | names the most recent ✅ item(s) the STATUS actually records (e.g. the fleet scaffold / latest merged PR in its ledger) — not an older phase, not an open issue |
+| 2 — next | applies the resume protocol: top **unblocked** issue of the **earliest open milestone by due date**, and skips issues whose only open step is a `[HUMAN]` gate (surfaced, not stalled on) |
+| 3 — trust | states the trust order live system > code/git log > STATUS > PRD leaves > memories (wording may vary; the direction must be right), and proposes checking Last Updated / verifying claims against the live repo |
+
+## Pass rule
+
+PASS = substantively correct on all three questions. Minor omissions that do not
+invert the protocol (e.g. not listing every milestone) do not fail; inverting
+the trust order, proposing a blocked/`[HUMAN]`-gated issue as "next" without
+flagging the gate, or citing shipped work the documents do not support, fails.
+Judged by an independent context-free judge — never by the session that produced
+the run.
@@ -0,0 +1,36 @@
+# Verifier test — locate-test
+
+First run: erp#38 acceptance gate (Claude, 2026-07-15, PASS — evidence on the
+issue). Re-run on candidate runtimes for verifier admission (erp#63).
+
+## Inputs
+
+| Document | Pinned source |
+| --- | --- |
+| `AGENTS.md` | the erp repo root, at the commit under test |
+
+The runtime gets the document content embedded in the prompt and **nothing else**
+— no repo access, no tools, no conversation history.
+
+## Prompt (verbatim — do not paraphrase when re-running)
+
+> You are in the erp repo; using AGENTS.md alone, state where the atom registry,
+> the class skeletons and the environment rules live.
+
+`bin/run-verifier.sh locate <runtime>` assembles this prompt with the document
+appended under a `--- AGENTS.md ---` delimiter.
+
+## Ground truth
+
+| Item | Correct answer |
+| --- | --- |
+| atom registry | `fleet/README.md` |
+| class skeletons | `fleet/classes/` (7 skeleton files) |
+| environment rules | the AGENTS.md **operating rules** section + `.claude/skills/dolibarr-sandbox-write/SKILL.md` |
+
+## Pass rule
+
+PASS = all three locations correct (path-level; exact anchor wording not
+required; for environment rules both halves must appear). Judged by an
+independent context-free judge against this table — never by the session that
+produced the run.