The harness layer (builder sessions, cold verifiers, evidence flow) gets a committable, multi-runtime home — per the PRD model-fleet › harness portability (operator direction 2026-07-15: Claude is an escalation tier, not a hard dependency).
Ships
fleet/harness/verifier/ — the two canonical verifier tests (locate-test, cold-reader backlog audit): verbatim prompts, pinned inputs, ground truth, pass rules.
fleet/harness/bin/run-verifier.sh — runs a test against any OpenAI-style local endpoint (Ornith/MLX) or vibe -p (Mistral); sha256-pinned JSON transcripts.
fleet/harness/bin/vibe-builder.sh — the bounded shell for scoped builders and recurring tasks: linked-worktree guard (refuses the trunk, same structural pattern as dol-write.sh), hard --max-turns/--max-price caps, full JSON journal.
fleet/harness/runs/2026-07-18/ — the erp#63 evidence: 8 verifier transcripts, blind-judging verdicts (16/16 unanimous), the erp#56 builder-bench journal.
Evidence highlights (full table in the #63 evidence comment): Mistral (mistral-medium-3.5 via vibe -p) and Ornith 35B (hermes MLX) reach verdict parity with the Claude baseline on both verifier tests → admitted to verifier duty; Qwen2.5-7B-4bit fails both (honest floor). Builder bench: erp#56 completed end-to-end by the Mistral runtime (0 code corrections, 261 s) → PR #68.
Paired factory PR codifies the cross-family verification rule in the PRD qa-strategy.
Closes #63.
The harness layer (builder sessions, cold verifiers, evidence flow) gets a committable, multi-runtime home — per the PRD [model-fleet › harness portability](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/model-fleet.md#harness-portability) (operator direction 2026-07-15: Claude is an escalation tier, not a hard dependency).
**Ships**
- `fleet/harness/verifier/` — the two canonical verifier tests (locate-test, cold-reader backlog audit): verbatim prompts, pinned inputs, ground truth, pass rules.
- `fleet/harness/bin/run-verifier.sh` — runs a test against any OpenAI-style local endpoint (Ornith/MLX) or `vibe -p` (Mistral); sha256-pinned JSON transcripts.
- `fleet/harness/bin/vibe-builder.sh` — the bounded shell for scoped builders and recurring tasks: linked-worktree guard (refuses the trunk, same structural pattern as `dol-write.sh`), hard `--max-turns`/`--max-price` caps, full JSON journal.
- `fleet/harness/runs/2026-07-18/` — the erp#63 evidence: 8 verifier transcripts, blind-judging verdicts (16/16 unanimous), the erp#56 builder-bench journal.
- Same-change doc updates: `fleet/README.md` layout + `AGENTS.md` Fleet section.
**Evidence highlights** (full table in the [#63 evidence comment](https://gitea.arcodange.lab/arcodange-org/erp/issues/63#issuecomment-2939)): Mistral (`mistral-medium-3.5` via `vibe -p`) and Ornith 35B (hermes MLX) reach verdict parity with the Claude baseline on both verifier tests → admitted to verifier duty; Qwen2.5-7B-4bit fails both (honest floor). Builder bench: erp#56 completed end-to-end by the Mistral runtime (0 code corrections, 261 s) → PR #68.
Paired factory PR codifies the cross-family verification rule in the PRD qa-strategy.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
The harness layer (builder sessions, cold verifiers, evidence flow) gets a
committable home, per the PRD model-fleet § harness portability and erp#63:
- fleet/harness/verifier/: the two canonical verifier tests (locate-test,
cold-reader backlog audit) with pinned inputs, verbatim prompts, ground
truth and pass rules — judged context-free, never self-graded.
- fleet/harness/bin/run-verifier.sh: runs a test against any OpenAI-style
local endpoint (Ornith/MLX) or vibe -p (Mistral); emits sha256-pinned
JSON transcripts.
- fleet/harness/bin/vibe-builder.sh: the bounded shell for scoped builders
and recurring tasks — refuses the trunk (linked-worktree guard), hard
--max-turns/--max-price caps, full JSON journal per run.
- fleet/README.md layout + AGENTS.md Fleet section updated in the same
change (same-change freshness rule).
Part of erp#63 (harness portability spike, D2).
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
- runs/2026-07-18/: the 8 sha256-pinned verifier transcripts (4 runtimes ×
2 tests), blind-judging verdicts (2 independent judges per cell, unanimous),
the erp#56 builder-bench journal + prompt + caps, and the evidence README
with the parity table.
- run-verifier.sh: mistral runtime drops the tool-filter flag (--enabled-tools
with a no-match pattern hangs vibe 2.21.0); plain -p with --max-turns 1.
Verdicts: Mistral (vibe -p, mistral-medium-3.5) and Ornith 35B (hermes MLX)
reach verdict parity with the Claude baseline on both tests → admitted to
verifier duty. Qwen2.5-7B-4bit fails both → the honest small-model floor.
Builder bench: erp#56 completed by the Mistral runtime, 0 code corrections,
261 s, acceptance run clean (0 bank-UNKNOWN) → merged as PR #68.
Closes#63 (with the paired factory qa-strategy PR).
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #63.
The harness layer (builder sessions, cold verifiers, evidence flow) gets a committable, multi-runtime home — per the PRD model-fleet › harness portability (operator direction 2026-07-15: Claude is an escalation tier, not a hard dependency).
Ships
fleet/harness/verifier/— the two canonical verifier tests (locate-test, cold-reader backlog audit): verbatim prompts, pinned inputs, ground truth, pass rules.fleet/harness/bin/run-verifier.sh— runs a test against any OpenAI-style local endpoint (Ornith/MLX) orvibe -p(Mistral); sha256-pinned JSON transcripts.fleet/harness/bin/vibe-builder.sh— the bounded shell for scoped builders and recurring tasks: linked-worktree guard (refuses the trunk, same structural pattern asdol-write.sh), hard--max-turns/--max-pricecaps, full JSON journal.fleet/harness/runs/2026-07-18/— the erp#63 evidence: 8 verifier transcripts, blind-judging verdicts (16/16 unanimous), the erp#56 builder-bench journal.fleet/README.mdlayout +AGENTS.mdFleet section.Evidence highlights (full table in the #63 evidence comment): Mistral (
mistral-medium-3.5viavibe -p) and Ornith 35B (hermes MLX) reach verdict parity with the Claude baseline on both verifier tests → admitted to verifier duty; Qwen2.5-7B-4bit fails both (honest floor). Builder bench: erp#56 completed end-to-end by the Mistral runtime (0 code corrections, 261 s) → PR #68.Paired factory PR codifies the cross-family verification rule in the PRD qa-strategy.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh