arcodangeandClaude Fable 5
6df4693880
Seed the invoice-extract (T02) and mail-classify (T01) golden sets from real
Arcodange history, plus an adversarial injection suite and an offline
field-level scorer.
invoice-extract/
- 16 real supplier PDFs (DARNIS/Hiway F1040/F1042/F1045/F1046, Anthropic
invoice+receipt x2, Mistral, OVH, greffe d'Evry, INPI x2, Legalstart, Qonto,
Infogreffe) fetched from the Zoho mailbox + Dolibarr GED, each with a
hand-verified expected JSON per the T02 schema. Every expected value was
cross-checked against the pdftotext -layout text and re-validated against the
deterministic invariants (HT+TVA=TTC, per-rate sums, IBAN mod-97, SIREN Luhn).
- inputs/ carries both the source PDF and its {source_sha256, mime, text} pair.
- 6 SYNTHETIC injection fixtures (LLM-directive, hidden white text, IBAN-swap
BEC lure, arithmetic-repair lure, fake tool-call, ref-hijack duplicate) whose
only correct outcome is quarantine; each PDF is marked SYNTHETIC.
- score.py: stdlib-only field-level scorer, critical fields (amounts/IBAN/refs/
dates) scored separately against the 98% bar, injection leaks blocking; a
built-in --self-test proves it catches perturbed fields and leaks.
- manifest.json: per-item provenance (mail message id / GED path + sha256),
linked Dolibarr supplier invoice, a verification note, and the list of real
documents deliberately excluded (fee statements, payment proofs, La Poste
receipts with no HT/TVA breakdown) with reasons.
mail-classify/
- 1824 historical mails labeled into {supplier-invoice, bank-notice,
government-admin, client, other} via sender-domain + subject weak supervision,
one human-correctable JSONL line per message with confidence + reason +
message-id provenance. manifest.json records the pull method and distribution.
Docs: golden/README hub, invoice-extract/README (T02 schema + conventions),
injection/README (threat table), mail-classify/README (method + distribution).
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
52 lines
1.7 KiB
JSON
52 lines
1.7 KiB
JSON
{
|
|
"set": "mail-classify",
|
|
"issue": "erp#39",
|
|
"generated_at": "2026-07-18",
|
|
"task": "T01 mailbox triage & routing",
|
|
"classes": [
|
|
"supplier-invoice",
|
|
"bank-notice",
|
|
"government-admin",
|
|
"client",
|
|
"other"
|
|
],
|
|
"source": {
|
|
"kind": "zoho-mail",
|
|
"account": "[email protected]",
|
|
"account_id": "7685847000000002002",
|
|
"pull": "read-only Zoho Mail OAuth API via arcodange-email-ingest; all 14 non-empty folders paged in full",
|
|
"folders_scanned": {
|
|
"/Inbox": 86,
|
|
"/Inbox/abonnements": 4,
|
|
"/Inbox/analytics": 10,
|
|
"/Inbox/bonjour": 463,
|
|
"/Inbox/books": 35,
|
|
"/Inbox/bureaux": 21,
|
|
"/Inbox/contact": 116,
|
|
"/Inbox/helloworld": 102,
|
|
"/Inbox/jobs": 17,
|
|
"/Newsletter": 431,
|
|
"/Notification": 127,
|
|
"/clients/KissMetrics": 412
|
|
}
|
|
},
|
|
"provenance": {
|
|
"method": "per-message source id is the Zoho `message_id` (+ `folder`) recorded on each JSONL line — the anchor back to the mailbox. Mail messages are not downloaded as files, so there is no per-message file hash; instead the label file itself is content-hashed for integrity.",
|
|
"labels_file": "labels.jsonl",
|
|
"labels_sha256": "53ea2fcb276efb5c13d91d0f2ed0e49c7ccaad5e473463d763b9f75eec19c500",
|
|
"count": 1824
|
|
},
|
|
"labeling": {
|
|
"type": "machine-seeded weak supervision (sender-domain + subject heuristics), human-correctable in place",
|
|
"correction": "edit `label`, set `corrected_by`; keep `machine_labeled:true`",
|
|
"bias": "precision-favoring on the actionable classes; human pass raises recall"
|
|
},
|
|
"distribution": {
|
|
"other": 1742,
|
|
"government-admin": 34,
|
|
"client": 30,
|
|
"bank-notice": 10,
|
|
"supplier-invoice": 8
|
|
}
|
|
}
|