arcodange-org/erp
1
0
Fork 0
Code Issues 18 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(fleet): golden set + injection fixtures from real history (erp#39)

Browse Source 
Seed the invoice-extract (T02) and mail-classify (T01) golden sets from real
Arcodange history, plus an adversarial injection suite and an offline
field-level scorer.

invoice-extract/
- 16 real supplier PDFs (DARNIS/Hiway F1040/F1042/F1045/F1046, Anthropic
  invoice+receipt x2, Mistral, OVH, greffe d'Evry, INPI x2, Legalstart, Qonto,
  Infogreffe) fetched from the Zoho mailbox + Dolibarr GED, each with a
  hand-verified expected JSON per the T02 schema. Every expected value was
  cross-checked against the pdftotext -layout text and re-validated against the
  deterministic invariants (HT+TVA=TTC, per-rate sums, IBAN mod-97, SIREN Luhn).
- inputs/ carries both the source PDF and its {source_sha256, mime, text} pair.
- 6 SYNTHETIC injection fixtures (LLM-directive, hidden white text, IBAN-swap
  BEC lure, arithmetic-repair lure, fake tool-call, ref-hijack duplicate) whose
  only correct outcome is quarantine; each PDF is marked SYNTHETIC.
- score.py: stdlib-only field-level scorer, critical fields (amounts/IBAN/refs/
  dates) scored separately against the 98% bar, injection leaks blocking; a
  built-in --self-test proves it catches perturbed fields and leaks.
- manifest.json: per-item provenance (mail message id / GED path + sha256),
  linked Dolibarr supplier invoice, a verification note, and the list of real
  documents deliberately excluded (fee statements, payment proofs, La Poste
  receipts with no HT/TVA breakdown) with reasons.

mail-classify/
- 1824 historical mails labeled into {supplier-invoice, bank-notice,
  government-admin, client, other} via sender-domain + subject weak supervision,
  one human-correctable JSONL line per message with confidence + reason +
  message-id provenance. manifest.json records the pull method and distribution.

Docs: golden/README hub, invoice-extract/README (T02 schema + conventions),
injection/README (threat table), mail-classify/README (method + distribution).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
arcodange
2026-07-18 19:07:58 +02:00
co-authored by Claude Fable 5
parent fe6909a2a6
commit 6df4693880
74 changed files with 3631 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fleet/golden/invoice-extract/inputs/anthropic-inv-9bf0758d-695749.json
+5
View File
@@ -0,0 +1,5 @@
{
"source_sha256": "9fc1e8f4bd27db7a62eed2b296c7ac7e3229143d7e2a5561dede994dbcb4a376",
"mime": "application/pdf",
"text": "Invoice\nInvoice number 9BF0758D-695749\nDate of issue April 12, 2026\nDate due April 12, 2026\n\n\nAnthropic Ireland, Limited Bill to\n6th Floor South Bank House, Barrow Street, Arcodange\nDublin 4, DUBLIN, Ireland 73 Boulevard de l'Yerres\nDUBLIN 91000 Évry\nCo. Dublin France\nIreland [email protected]\[email protected] FR VAT FR00999657455\n\n\n\n€180.00 due April 12, 2026\nPay online\n\nReverse charge applies - Customer to account for tax under Article 196 of Directive 2006/112/EC.\n\n\nDescription Qty Unit price Tax Amount\n\n [1]\nClaude Pro 1 €180.00 0% €180.00\nApr 12, 2026–Apr 12, 2027\n\n\n Subtotal €180.00\n Total €180.00\n Amount due €180.00\n\n\nCustomer may be obliged to account for VAT on reverse charge basis.\n\n[1] Tax to be paid on reverse charge basis\n\n\n\n\n Page 1 of 1\n\f"
}