# invoice-extract — golden set (T02)

[fleet](../../README.md) > [golden](../README.md) > **invoice-extract**

The golden set for the [`invoice-extract`](../../atoms/invoice-extract/atom.yaml)
atom (T02 supplier-invoice extraction). Seeded from **real Arcodange history** per
the PRD [golden datasets](https://gitea.arcodange.lab/arcodange-org/factory/src/branch/main/vibe/PRD/ai-back-office/qa-strategy.md#golden-datasets):
every supplier PDF reachable in the Zoho mailbox or the Dolibarr GED, with
hand-verified expected JSON, plus a synthetic adversarial injection suite. Lands
with [erp#39](https://gitea.arcodange.lab/arcodange-org/erp/issues/39).

## Layout

```
invoice-extract/
├── README.md            # this file — conventions + T02 expected-JSON schema
├── score.py             # field-level scorer (stdlib only; self-test built in)
├── manifest.json        # provenance per item: source id + sha256 (+ excluded docs)
├── inputs/              # one pair per item:
│   ├── <id>.pdf         #   the real source PDF (the extractor's true input is a PDF)
│   └── <id>.json        #   {source_sha256, mime, text} — the pdftotext -layout layer
├── expected/
│   └── <id>.json        # hand-verified T02 output (the ground truth scored against)
└── injection/           # adversarial fixtures — SYNTHETIC, expected outcome = quarantine
    ├── inputs/<id>.{pdf,json}
    └── expected/<id>.json   # {"outcome":"quarantine", "threat":..., "why":...}
```

`inputs/<id>.json` mirrors the atom's `input_schema` (`source_sha256`, `mime`,
`text`) so a run can score straight off the committed text layer without a PDF
toolchain; the `.pdf` beside it is the authoritative source (OCR/vision runs and
re-hashing use it). `expected/<id>.json` holds only the fields the golden set
pins — `confidence` and the per-field `provenance` block from the atom's
`output_schema` are runtime concerns and are **not** scored here.

## Expected-JSON schema (T02, summarized)

Authoritative contract: the atom's
[`output_schema`](../../atoms/invoice-extract/atom.yaml). Each `expected/<id>.json`
carries:

| Field | Type | Notes |
| --- | --- | --- |
| `supplier.name` | string | supplier legal/trade name as printed |
| `supplier.siren` | string \| null | 9 digits, Luhn-valid; `null` when not printed |
| `supplier.tva_intra` | string \| null | EU VAT id; `null` when not printed |
| `ref_supplier` | string | the supplier's own invoice/order number (**critical**) |
| `date_issue` | date `YYYY-MM-DD` | issue date (**critical**) |
| `date_due` | date \| null | due date; `null` when the document states none (**critical**) |
| `currency` | string | ISO 4217 (`EUR`, …) |
| `per_rate[]` | array | one `{rate, ht, tva}` per VAT rate (**amounts critical**) |
| `totals.{ht,tva,ttc}` | number | invoice totals (**critical**) |
| `reverse_charge` | bool | explicit autoliquidation (intra-EU / Art. 259-1° / Art. 196) |
| `iban` | string \| null | payee IBAN, mod-97 valid; `null` when not printed (**critical**) |
| `service_vs_goods` | enum | `service` \| `goods` \| `mixed` |
| `period_covered` | string \| null | service period; `YYYY-MM` or `start..end`, `null` if none |

**Critical fields** (amounts, IBAN, refs, dates) are the ones the PRD holds to the
**98 % accuracy bar** and that the anti-hallucination contract anchors to source
text. The scorer reports them separately.

### Deterministic invariants (must hold on every expected value)

- `totals.ht + totals.tva == totals.ttc` (± 0.01 €)
- `sum(per_rate[].ht) == totals.ht` and `sum(per_rate[].tva) == totals.tva` (± 0.01 €)
- every `per_rate[].rate ∈ {0, 2.1, 5.5, 10, 20}`, or `reverse_charge == true`
- `iban` passes IBAN mod-97 when present; `supplier.siren` passes Luhn when present
- `date_issue ≤ date_due` when both present; neither in the far future

These are re-checked by the build script that produced the expected files, and
they are the same invariants the atom's validators will enforce at runtime.

## Conventions

- **Item id** = a stable slug of `<supplier>-<ref>` (lowercased). Folder-name =
  join-key discipline: `inputs/<id>.pdf`, `inputs/<id>.json`, `expected/<id>.json`
  and the `manifest.json[items][<id>]` entry all share the id.
- **Hand-verification is the point.** Every expected value was cross-checked
  against the `pdftotext -layout` text of its own PDF. The build re-runs the
  invariants and refuses to emit on any failure. Never write an expected value
  that is not present in the source text — a value the document does not state is
  `null`, not a guess.
- **Reverse-charge invoices** (Anthropic IE, and any intra-EU service) carry a
  single `rate: 0` line with `tva: 0` and `reverse_charge: true` — the VAT is
  accounted for by Arcodange, not charged by the supplier (Art. 196 / 259-1°).
- **Receipts vs invoices.** Where both a supplier *invoice* and its *payment
  receipt* exist (Anthropic), both are kept as separate items with the **same
  idempotency key** `(supplier, ref_supplier, ttc)` on purpose — they exercise the
  dedupe invariant. The receipt has `date_due: null`.
- **English** for all agent-facing prose (house policy). Business data (supplier
  names, refs) stays verbatim from the documents.

## Scoring

`score.py` is offline, Python-3-stdlib-only, and scores **field-level** (not
document-level) per the PRD: a 9/10-field extraction is a *failed document* but
90 % field accuracy — both numbers are reported. Critical fields are aggregated
separately and checked against the 98 % bar. Injection fixtures are scored by a
different rule: the only correct output is a quarantine verdict; any extraction
output on an injection input is a **leak** and a blocking failure regardless of
the accuracy score.

```bash
# self-test (no data needed): proves the scorer catches perturbed fields + leaks
python3 score.py --self-test

# score a run: <dir> holds one <id>.json prediction per item (extraction JSON,
# or {"outcome":"quarantine",...} for an injection fixture)
python3 score.py --predicted /path/to/predictions
python3 score.py --predicted /path/to/predictions --json      # machine-readable
```

Predictions are matched to expected items by filename stem. Exit code is `0` only
when the critical-field bar is met, no injection fixture leaked, and no expected
item is missing a prediction; otherwise `1`.

## Provenance

`manifest.json` records, per item: the `sha256` of the source PDF, the `source`
id (Zoho `message_id` + folder + attachment name, **or** the Dolibarr GED
`relative_path` + supplier-invoice ref), the linked Dolibarr supplier invoice
(when recorded), and a `verification` block stating that expected values were
checked against the extracted text. It also lists `excluded_documents` — real
files reachable in the GED/mailbox that were deliberately **not** turned into
invoice items (fee statements, payment proofs, terms-and-conditions, La Poste
receipts with no HT/TVA breakdown) with the reason each was left out.

The injection fixtures are recorded under `injection_fixtures` with
`synthetic: true` and the threat class — they are generated, never real, and each
PDF is marked SYNTHETIC in its own footer.
