feat(fleet): invoice-extract atom — dual extraction + validators + provenance (erp#40)
Implementation of the T02 atom over the erp#39 golden set: - validators.py: instruction-pattern + multi-IBAN pre-screens (0 hard false positives on the 16 real docs; all 6 injection fixtures quarantined BEFORE any model call), the atom.yaml invariants, and literal provenance anchoring with locale-aware locate (FR/EN months incl. abbreviations, NBSP-tolerant amounts, line-wrap + column-interleave fragment anchoring for refs). - extract.py: single-leg runner (MLX endpoint / vibe -p), zero credentials, zero action tools; reasoning-channel aware. - dual_run.py: model_policy in code — dual legs, exact critical-field agreement; disagreement, single-valid-leg or both-invalid → escalations/ for the Claude tier (resolutions go back through validators.check). Eval (eval/2026-07-19/, full transcripts + journals committed): - critical-field accuracy 100 % (bar 98 %) — MET - injection suite 6/6 quarantined — zero leaks - overall field accuracy 94.9 % (known gaps: supplier ids often null, period_covered format) — non-blocking, noted for the next version - 9/16 documents escalated to the Claude tier (Mistral API timeouts, small local model on receipts, one BIC-glued IBAN, derived-ratio rates) — consistent with the A1 autonomy level recorded in atom.yaml Runtimes this run: m4-local = Qwen2.5-7B-4bit (MLX), mistral = vibe -p (mistral-medium-3.5) — provisional pending erp#45; journals are the routing-bench raw material. Closes erp#40 (PR to follow once arcodange/golden-set is pushed — this branch stacks on it). Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
@@ -4,11 +4,11 @@
|
||||
# https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/agent-architecture.md#atom-contract
|
||||
# Authoritative task fiche (T02 invoice schema summarized below):
|
||||
# https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/task-inventory.md#t02--supplier-invoice-extraction
|
||||
# Scaffold status: erp#38's worked example. Implementation (runners, validators,
|
||||
# OCR fallback) lands with erp#40 — nothing here pretends to run yet.
|
||||
# Implemented (erp#40): runners + validators + dual-run orchestrator in
|
||||
# scripts/ (see scripts/README.md). OCR fallback still stubbed — provider = D5.
|
||||
|
||||
name: invoice-extract
|
||||
version: 0.1.0
|
||||
version: 0.2.0
|
||||
class: extractor # extends fleet/classes/extractor.md
|
||||
task: T02 # supplier invoice extraction
|
||||
|
||||
@@ -104,16 +104,19 @@ idempotency_key: [supplier, ref_supplier, totals.ttc]
|
||||
autonomy:
|
||||
level: A1 # prepare — the atom drafts, a human records (today's heuristic flow)
|
||||
eval_evidence: >-
|
||||
none yet — the golden set lands with erp#39
|
||||
(https://gitea.arcodange.lab/arcodange-org/erp/issues/39), the
|
||||
implementation with erp#40
|
||||
(https://gitea.arcodange.lab/arcodange-org/erp/issues/40). Promotion per the
|
||||
PRD qa-strategy autonomy gates.
|
||||
eval/2026-07-19/ — full golden set (16 real + 6 injection): critical-field
|
||||
accuracy 100 % (bar 98 %), 6/6 injections quarantined PRE-model, overall
|
||||
field accuracy 94.9 % (gaps: supplier ids often null, period format).
|
||||
9/16 documents needed Claude-tier escalation (single-valid-leg or
|
||||
correction) — consistent with A1; promotion per the PRD qa-strategy gates
|
||||
needs unedited-approval streaks on live traffic, not just this eval.
|
||||
|
||||
model_policy:
|
||||
# Placeholder — closed by erp#45 (POC-5 model routing bench):
|
||||
# Provisional — closed by erp#45 (POC-5 model routing bench):
|
||||
# https://gitea.arcodange.lab/arcodange-org/erp/issues/45
|
||||
dual_extraction: [m4-local, mistral-api] # two independent runs
|
||||
dual_extraction: [m4-local, mistral] # two independent runs (scripts/extract.py)
|
||||
m4_local: mlx-community/Qwen2.5-7B-Instruct-4bit # resident; Ornith-35B viable at max_tokens>=8000 (reasoning channel), minutes/doc
|
||||
mistral: vibe -p (mistral-medium-3.5) # harness-admitted runtime; JSON prompt-enforced, validators own truth
|
||||
agreement: exact match required on critical fields (amounts, IBAN, ref, dates)
|
||||
escalation: claude # on disagreement; still-ambiguous items → quarantine
|
||||
fallbacks: TBD — erp#45 publishes accuracy × latency × cost per tier
|
||||
|
||||
Reference in New Issue
Block a user