Files

T

arcodangeandClaude Fable 5 f2a60817e2 feat(fleet): multi-runtime harness — verifier tests + capped builder shell

The harness layer (builder sessions, cold verifiers, evidence flow) gets a
committable home, per the PRD model-fleet § harness portability and erp#63:

- fleet/harness/verifier/: the two canonical verifier tests (locate-test,
  cold-reader backlog audit) with pinned inputs, verbatim prompts, ground
  truth and pass rules — judged context-free, never self-graded.
- fleet/harness/bin/run-verifier.sh: runs a test against any OpenAI-style
  local endpoint (Ornith/MLX) or vibe -p (Mistral); emits sha256-pinned
  JSON transcripts.
- fleet/harness/bin/vibe-builder.sh: the bounded shell for scoped builders
  and recurring tasks — refuses the trunk (linked-worktree guard), hard
  --max-turns/--max-price caps, full JSON journal per run.
- fleet/README.md layout + AGENTS.md Fleet section updated in the same
  change (same-change freshness rule).

Part of erp#63 (harness portability spike, D2).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh

2026-07-18 18:52:53 +02:00

1.3 KiB

Raw Blame History

Verifier test — locate-test

First run: erp#38 acceptance gate (Claude, 2026-07-15, PASS — evidence on the issue). Re-run on candidate runtimes for verifier admission (erp#63).

Inputs

Document	Pinned source
`AGENTS.md`	the erp repo root, at the commit under test

The runtime gets the document content embedded in the prompt and nothing else — no repo access, no tools, no conversation history.

Prompt (verbatim — do not paraphrase when re-running)

You are in the erp repo; using AGENTS.md alone, state where the atom registry, the class skeletons and the environment rules live.

bin/run-verifier.sh locate <runtime> assembles this prompt with the document appended under a --- AGENTS.md --- delimiter.

Ground truth

Item	Correct answer
atom registry	`fleet/README.md`
class skeletons	`fleet/classes/` (7 skeleton files)
environment rules	the AGENTS.md operating rules section + `.claude/skills/dolibarr-sandbox-write/SKILL.md`

Pass rule

PASS = all three locations correct (path-level; exact anchor wording not required; for environment rules both halves must appear). Judged by an independent context-free judge against this table — never by the session that produced the run.

1.3 KiB Raw Blame History