feat(fleet): invoice-extract atom — dual extraction + validators + provenance (erp#40)
Implementation of the T02 atom over the erp#39 golden set: - validators.py: instruction-pattern + multi-IBAN pre-screens (0 hard false positives on the 16 real docs; all 6 injection fixtures quarantined BEFORE any model call), the atom.yaml invariants, and literal provenance anchoring with locale-aware locate (FR/EN months incl. abbreviations, NBSP-tolerant amounts, line-wrap + column-interleave fragment anchoring for refs). - extract.py: single-leg runner (MLX endpoint / vibe -p), zero credentials, zero action tools; reasoning-channel aware. - dual_run.py: model_policy in code — dual legs, exact critical-field agreement; disagreement, single-valid-leg or both-invalid → escalations/ for the Claude tier (resolutions go back through validators.check). Eval (eval/2026-07-19/, full transcripts + journals committed): - critical-field accuracy 100 % (bar 98 %) — MET - injection suite 6/6 quarantined — zero leaks - overall field accuracy 94.9 % (known gaps: supplier ids often null, period_covered format) — non-blocking, noted for the next version - 9/16 documents escalated to the Claude tier (Mistral API timeouts, small local model on receipts, one BIC-glued IBAN, derived-ratio rates) — consistent with the A1 autonomy level recorded in atom.yaml Runtimes this run: m4-local = Qwen2.5-7B-4bit (MLX), mistral = vibe -p (mistral-medium-3.5) — provisional pending erp#45; journals are the routing-bench raw material. Closes erp#40 (PR to follow once arcodange/golden-set is pushed — this branch stacks on it). Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
@@ -4,8 +4,8 @@
|
||||
Per document: deterministic pre-screens → two independent extraction legs
|
||||
(M4 local MLX ∥ Mistral via vibe) → validators on each leg → exact agreement
|
||||
required on critical fields. Disagreement, single-valid-leg, or an escalate
|
||||
flag → the item lands in escalations/ for the Claude tier; hostile or
|
||||
invalid-on-both-legs items are quarantined. Refuse, never repair.
|
||||
flag, or both legs invalid → the item lands in escalations/ for the Claude
|
||||
tier; hostile documents quarantine at the pre-screen. Refuse, never repair.
|
||||
|
||||
Usage:
|
||||
dual_run.py --inputs <dir> [--injection <dir>] --out <predictions-dir> \\
|
||||
@@ -88,9 +88,11 @@ def process(item_id: str, doc: dict, mlx_model: str, journal) -> tuple[str, dict
|
||||
|
||||
a, b = legs
|
||||
if not a["valid"] and not b["valid"]:
|
||||
return "quarantine", {"outcome": "quarantine", "stage": "post-validation",
|
||||
"why": {"mlx": a.get("invariant_failures") or a.get("error") or a.get("raw_tail"),
|
||||
"vibe": b.get("invariant_failures") or b.get("error") or b.get("raw_tail")}}
|
||||
# Hostile content is caught pre-screen; a clean document both legs fail
|
||||
# to ground goes to the escalation tier, whose verdict may be quarantine.
|
||||
return "escalate", {"stage": "escalation", "reason": "both-legs-invalid",
|
||||
"legs": [{k: l.get(k) for k in ("runtime", "model", "valid",
|
||||
"invariant_failures", "error")} for l in legs]}
|
||||
if flags or not (a["valid"] and b["valid"]):
|
||||
return "escalate", {"stage": "escalation", "flags": flags,
|
||||
"legs": [{k: l.get(k) for k in ("runtime", "model", "valid", "validated",
|
||||
|
||||
Reference in New Issue
Block a user