arcodangeandClaude Fable 5
f2a60817e2
The harness layer (builder sessions, cold verifiers, evidence flow) gets a committable home, per the PRD model-fleet § harness portability and erp#63: - fleet/harness/verifier/: the two canonical verifier tests (locate-test, cold-reader backlog audit) with pinned inputs, verbatim prompts, ground truth and pass rules — judged context-free, never self-graded. - fleet/harness/bin/run-verifier.sh: runs a test against any OpenAI-style local endpoint (Ornith/MLX) or vibe -p (Mistral); emits sha256-pinned JSON transcripts. - fleet/harness/bin/vibe-builder.sh: the bounded shell for scoped builders and recurring tasks — refuses the trunk (linked-worktree guard), hard --max-turns/--max-price caps, full JSON journal per run. - fleet/README.md layout + AGENTS.md Fleet section updated in the same change (same-change freshness rule). Part of erp#63 (harness portability spike, D2). Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
37 lines
1.3 KiB
Markdown
37 lines
1.3 KiB
Markdown
# Verifier test — locate-test
|
|
|
|
First run: erp#38 acceptance gate (Claude, 2026-07-15, PASS — evidence on the
|
|
issue). Re-run on candidate runtimes for verifier admission (erp#63).
|
|
|
|
## Inputs
|
|
|
|
| Document | Pinned source |
|
|
| --- | --- |
|
|
| `AGENTS.md` | the erp repo root, at the commit under test |
|
|
|
|
The runtime gets the document content embedded in the prompt and **nothing else**
|
|
— no repo access, no tools, no conversation history.
|
|
|
|
## Prompt (verbatim — do not paraphrase when re-running)
|
|
|
|
> You are in the erp repo; using AGENTS.md alone, state where the atom registry,
|
|
> the class skeletons and the environment rules live.
|
|
|
|
`bin/run-verifier.sh locate <runtime>` assembles this prompt with the document
|
|
appended under a `--- AGENTS.md ---` delimiter.
|
|
|
|
## Ground truth
|
|
|
|
| Item | Correct answer |
|
|
| --- | --- |
|
|
| atom registry | `fleet/README.md` |
|
|
| class skeletons | `fleet/classes/` (7 skeleton files) |
|
|
| environment rules | the AGENTS.md **operating rules** section + `.claude/skills/dolibarr-sandbox-write/SKILL.md` |
|
|
|
|
## Pass rule
|
|
|
|
PASS = all three locations correct (path-level; exact anchor wording not
|
|
required; for environment rules both halves must appear). Judged by an
|
|
independent context-free judge against this table — never by the session that
|
|
produced the run.
|