arcodange-org/erp
1
0
Fork 0
Code Issues 18 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(fleet): golden set + injection fixtures from real history (erp#39)

Browse Source 
Seed the invoice-extract (T02) and mail-classify (T01) golden sets from real
Arcodange history, plus an adversarial injection suite and an offline
field-level scorer.

invoice-extract/
- 16 real supplier PDFs (DARNIS/Hiway F1040/F1042/F1045/F1046, Anthropic
  invoice+receipt x2, Mistral, OVH, greffe d'Evry, INPI x2, Legalstart, Qonto,
  Infogreffe) fetched from the Zoho mailbox + Dolibarr GED, each with a
  hand-verified expected JSON per the T02 schema. Every expected value was
  cross-checked against the pdftotext -layout text and re-validated against the
  deterministic invariants (HT+TVA=TTC, per-rate sums, IBAN mod-97, SIREN Luhn).
- inputs/ carries both the source PDF and its {source_sha256, mime, text} pair.
- 6 SYNTHETIC injection fixtures (LLM-directive, hidden white text, IBAN-swap
  BEC lure, arithmetic-repair lure, fake tool-call, ref-hijack duplicate) whose
  only correct outcome is quarantine; each PDF is marked SYNTHETIC.
- score.py: stdlib-only field-level scorer, critical fields (amounts/IBAN/refs/
  dates) scored separately against the 98% bar, injection leaks blocking; a
  built-in --self-test proves it catches perturbed fields and leaks.
- manifest.json: per-item provenance (mail message id / GED path + sha256),
  linked Dolibarr supplier invoice, a verification note, and the list of real
  documents deliberately excluded (fee statements, payment proofs, La Poste
  receipts with no HT/TVA breakdown) with reasons.

mail-classify/
- 1824 historical mails labeled into {supplier-invoice, bank-notice,
  government-admin, client, other} via sender-domain + subject weak supervision,
  one human-correctable JSONL line per message with confidence + reason +
  message-id provenance. manifest.json records the pull method and distribution.

Docs: golden/README hub, invoice-extract/README (T02 schema + conventions),
injection/README (threat table), mail-classify/README (method + distribution).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
arcodange
2026-07-18 19:07:58 +02:00
co-authored by Claude Fable 5
parent fe6909a2a6
commit 6df4693880
74 changed files with 3631 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fleet/golden/invoice-extract/inputs/mistral-mstrl-api-814045-001.json
+5
View File
@@ -0,0 +1,5 @@
{
"source_sha256": "892afe9d7e8f96f1debad9685f7b7ec88a826c53ab8993751ce7803c5bd7ba3f",
"mime": "application/pdf",
"text": "Facture\nNº de facture MSTRL-API-814045-001 Organization Gabriel Radureau\nDate d'émission 2 avr. 2026\nDélai de paiement 0 jours\n\n\n\nDe Facturé à\nMistral AI SAS Arcodange\n952 418 325 73 boulevard de l'yerres\n15, rue des Halles 91000 Evry Courcouronnes\n75001, Paris France\nFrance [email protected]\[email protected] ID fiscal: FR00999657455\nID fiscal: FR95952418325\n\n\n\n172,68 €\nDate d'échéance le 2 avr. 2026\n\n\n\nDétails de Le Chat Pro - Annual\n\nFrais de conso. du 2 avr. 2026 au 1 avr. 2027 Unités Prix unitaire Taux de taxe Montant\n\n\nAbonnement annuel - Le Chat Pro - Annual 1.0 143,90 € 20.0% 143,90 €\n\n\n Sous total (HT) 143,90 €\n\n\n EU FR Standard (20.0% sur 143,90 €) 28,78 €\n\n\n Sous total (TTC) 172,68 €\n\n\n Total 172,68 €\n\n\n\nMistral AI - SAS au capital de 19,189.04€ - 952 418 325 R.C.S. Paris\nFor any inquiry, please refer to https://help.mistral.ai/en/\n\f"
}