chore(fleet): erp#63 evidence — verifier parity + builder bench transcripts
- runs/2026-07-18/: the 8 sha256-pinned verifier transcripts (4 runtimes × 2 tests), blind-judging verdicts (2 independent judges per cell, unanimous), the erp#56 builder-bench journal + prompt + caps, and the evidence README with the parity table. - run-verifier.sh: mistral runtime drops the tool-filter flag (--enabled-tools with a no-match pattern hangs vibe 2.21.0); plain -p with --max-turns 1. Verdicts: Mistral (vibe -p, mistral-medium-3.5) and Ornith 35B (hermes MLX) reach verdict parity with the Claude baseline on both tests → admitted to verifier duty. Qwen2.5-7B-4bit fails both → the honest small-model floor. Builder bench: erp#56 completed by the Mistral runtime, 0 code corrections, 261 s, acceptance run clean (0 bank-UNKNOWN) → merged as PR #68. Closes #63 (with the paired factory qa-strategy PR). Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
@@ -98,8 +98,10 @@ PY
|
||||
)"
|
||||
;;
|
||||
mistral)
|
||||
# Plain -p, no tool filtering: --enabled-tools with a no-match pattern hangs
|
||||
# vibe 2.21.0. --max-turns 1 makes tool use moot for a pure-answer test.
|
||||
MODEL="vibe-active-model"
|
||||
RESPONSE="$(vibe -p "$PROMPT" --max-turns 1 --enabled-tools __none__ --output text)"
|
||||
RESPONSE="$(vibe -p "$PROMPT" --max-turns 1 --output text)"
|
||||
;;
|
||||
esac
|
||||
|
||||
|
||||
Reference in New Issue
Block a user