Files

T

arcodangeandClaude Fable 5 f2a60817e2 feat(fleet): multi-runtime harness — verifier tests + capped builder shell

The harness layer (builder sessions, cold verifiers, evidence flow) gets a
committable home, per the PRD model-fleet § harness portability and erp#63:

- fleet/harness/verifier/: the two canonical verifier tests (locate-test,
  cold-reader backlog audit) with pinned inputs, verbatim prompts, ground
  truth and pass rules — judged context-free, never self-graded.
- fleet/harness/bin/run-verifier.sh: runs a test against any OpenAI-style
  local endpoint (Ornith/MLX) or vibe -p (Mistral); emits sha256-pinned
  JSON transcripts.
- fleet/harness/bin/vibe-builder.sh: the bounded shell for scoped builders
  and recurring tasks — refuses the trunk (linked-worktree guard), hard
  --max-turns/--max-price caps, full JSON journal per run.
- fleet/README.md layout + AGENTS.md Fleet section updated in the same
  change (same-change freshness rule).

Part of erp#63 (harness portability spike, D2).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh

2026-07-18 18:52:53 +02:00

2.2 KiB

Raw Permalink Blame History

Verifier test — cold-reader backlog audit

The fresh-reader smoke test from the PRD closure protocol, run standalone: a cold reader with only the two orientation documents must reconstruct where the work stands. Used for verifier admission (erp#63).

Inputs

Document	Pinned source
`AGENTS.md`	the erp repo root, at the commit under test
`STATUS.md`	`vibe/PRD/ai-back-office/STATUS.md` from the factory repo, origin/main at run time

The runtime gets both documents embedded in the prompt and nothing else.

Prompt (verbatim)

You are a cold reader auditing the Arcodange AI back-office backlog. Using ONLY the two documents below — no other knowledge, no tools — answer:

What shipped most recently?

What should be worked on next, and why that item?

What would you verify before trusting these documents, and in what order?

bin/run-verifier.sh backlog <runtime> --status-file <path> assembles this prompt with both documents appended under --- AGENTS.md --- and --- STATUS.md --- delimiters.

Rubric

Question	Substantively correct means
1 — shipped	names the most recent ✅ item(s) the STATUS actually records (e.g. the fleet scaffold / latest merged PR in its ledger) — not an older phase, not an open issue
2 — next	applies the resume protocol: top unblocked issue of the earliest open milestone by due date, and skips issues whose only open step is a `[HUMAN]` gate (surfaced, not stalled on)
3 — trust	states the trust order live system > code/git log > STATUS > PRD leaves > memories (wording may vary; the direction must be right), and proposes checking Last Updated / verifying claims against the live repo

Pass rule

PASS = substantively correct on all three questions. Minor omissions that do not invert the protocol (e.g. not listing every milestone) do not fail; inverting the trust order, proposing a blocked/[HUMAN]-gated issue as "next" without flagging the gate, or citing shipped work the documents do not support, fails. Judged by an independent context-free judge — never by the session that produced the run.

2.2 KiB Raw Permalink Blame History