feat(fleet): multi-runtime harness — verifier tests + capped builder shell

The harness layer (builder sessions, cold verifiers, evidence flow) gets a
committable home, per the PRD model-fleet § harness portability and erp#63:

- fleet/harness/verifier/: the two canonical verifier tests (locate-test,
  cold-reader backlog audit) with pinned inputs, verbatim prompts, ground
  truth and pass rules — judged context-free, never self-graded.
- fleet/harness/bin/run-verifier.sh: runs a test against any OpenAI-style
  local endpoint (Ornith/MLX) or vibe -p (Mistral); emits sha256-pinned
  JSON transcripts.
- fleet/harness/bin/vibe-builder.sh: the bounded shell for scoped builders
  and recurring tasks — refuses the trunk (linked-worktree guard), hard
  --max-turns/--max-price caps, full JSON journal per run.
- fleet/README.md layout + AGENTS.md Fleet section updated in the same
  change (same-change freshness rule).

Part of erp#63 (harness portability spike, D2).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
2026-07-18 18:52:53 +02:00
co-authored by Claude Fable 5
parent fe6909a2a6
commit f2a60817e2
7 changed files with 349 additions and 1 deletions
+44
View File
@@ -0,0 +1,44 @@
# Verifier test — cold-reader backlog audit
The fresh-reader smoke test from the PRD
[closure protocol](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/STATUS.md),
run standalone: a cold reader with only the two orientation documents must
reconstruct where the work stands. Used for verifier admission (erp#63).
## Inputs
| Document | Pinned source |
| --- | --- |
| `AGENTS.md` | the erp repo root, at the commit under test |
| `STATUS.md` | `vibe/PRD/ai-back-office/STATUS.md` from the **factory** repo, origin/main at run time |
The runtime gets both documents embedded in the prompt and **nothing else**.
## Prompt (verbatim)
> You are a cold reader auditing the Arcodange AI back-office backlog. Using ONLY
> the two documents below — no other knowledge, no tools — answer:
> 1. What shipped most recently?
> 2. What should be worked on next, and why that item?
> 3. What would you verify before trusting these documents, and in what order?
`bin/run-verifier.sh backlog <runtime> --status-file <path>` assembles this
prompt with both documents appended under `--- AGENTS.md ---` and
`--- STATUS.md ---` delimiters.
## Rubric
| Question | Substantively correct means |
| --- | --- |
| 1 — shipped | names the most recent ✅ item(s) the STATUS actually records (e.g. the fleet scaffold / latest merged PR in its ledger) — not an older phase, not an open issue |
| 2 — next | applies the resume protocol: top **unblocked** issue of the **earliest open milestone by due date**, and skips issues whose only open step is a `[HUMAN]` gate (surfaced, not stalled on) |
| 3 — trust | states the trust order live system > code/git log > STATUS > PRD leaves > memories (wording may vary; the direction must be right), and proposes checking Last Updated / verifying claims against the live repo |
## Pass rule
PASS = substantively correct on all three questions. Minor omissions that do not
invert the protocol (e.g. not listing every milestone) do not fail; inverting
the trust order, proposing a blocked/`[HUMAN]`-gated issue as "next" without
flagging the gate, or citing shipped work the documents do not support, fails.
Judged by an independent context-free judge — never by the session that produced
the run.