diff --git a/.claude/skills/dolibarr-sandbox-write/SKILL.md b/.claude/skills/dolibarr-sandbox-write/SKILL.md index 067ee9b..57fae34 100644 --- a/.claude/skills/dolibarr-sandbox-write/SKILL.md +++ b/.claude/skills/dolibarr-sandbox-write/SKILL.md @@ -8,8 +8,13 @@ description: >- with a before/after read-back diff), add contacts idempotently (dedupe by email then name), customer + supplier invoices with product/service lines and the supplier's own reference, validate them, and record règlements (payments). Every - write goes through dol-write.sh, which REFUSES any host that is not the sandbox — - the structural guarantee (ADR-0003) that this skill can never mutate production. + create is IDEMPOTENT (erp#44): before any POST it dedupes against the target + (thirdparty by exact name, supplier invoice by socid+ref_supplier, customer + invoice by socid+date+total+line fingerprint, payment by invoice+amount+ + normalized transaction id; credit notes excepted, a follow-up), so replaying + a manifest — even one that failed mid-run — is a no-op, never a duplicate. Every write goes through dol-write.sh, + which REFUSES any host that is not the sandbox — the structural guarantee + (ADR-0003) that this skill can never mutate production. Use when the user asks to "create a thirdparty / supplier / client fiche", "compléter / mettre à jour la fiche client", "add a contact to a thirdparty", "saisir une facture", "record an invoice with lines", "enregistrer un règlement / @@ -77,7 +82,15 @@ echo '{"name":"OVH","role":"supplier","siret":"..."}' | scripts/third `role`: `client` | `supplier` | `both`. Codes auto-assign from the mask (`CL{0000}` / `FO{0000}`) via the `-1` sentinel; pass `client_code`/`supplier_code` to override. Optional: `country_id` (default 1=FR), `siret`, `tva_intra`, -`address`, `zip`, `town`, `email`, `phone`, `idprof1`. Emits the new id. +`address`, `zip`, `town`, `email`, `phone`, `idprof1`. + +**Idempotent (erp#44): dedupe by exact name.** Before any POST the name is looked +up on the target with the same semantics as promote's `#thirdparty:name=` lookup: +one match whose roles cover the requested `role` → `{"id": , +"deduped": true}`, no write; **2+ matches abort** (ambiguous — never guess); a +match **missing the requested role aborts** too (refuse-never-repair: reusing a +client fiche as a supplier would skip the code mask and hide a data problem). +Otherwise it creates and emits `{"id": , "deduped": false}`. ### 2 · Invoice (facture) — `scripts/invoice-create.sh` @@ -95,7 +108,26 @@ echo '{"socid":7,"kind":"supplier","ref_supplier":"INV-2026-042","validate":true `desc, qty, price_ht, tva, type` (product|service) and optional `product_id` (`fk_product`) to link a catalogue product. Totals + TVA are computed by Dolibarr. `validate:true` turns the draft (`PROV…`) into a final numbered invoice; omit it -to leave a draft. Emits `{id, ref, ref_supplier, total_ht, total_ttc, statut}`. +to leave a draft. Emits `{id, ref, ref_supplier, total_ht, total_ttc, statut, +deduped}`. + +**Idempotent (erp#44).** Before any POST the thirdparty's invoices of the same +kind are listed and deduped: + +- **supplier with `ref_supplier`** → by **(socid, ref_supplier)** — the + supplier's own ref is the natural key. Same key with a *different* total + **aborts** (a conflict to resolve, never a dedupe). +- **customer** (or supplier without `ref_supplier`) → by **(socid, date, + total_ttc ±0.02, line fingerprint)** — the multiset of (desc, qty, subprice, + tva_tx), desc HTML-unescaped/whitespace-collapsed since Dolibarr returns + stored text HTML-encoded. Credit notes (`type=2`) are never candidates; + 2+ matches abort. + +A hit emits the **existing** invoice with `"deduped": true` — and if +`validate:true` was asked while the match is still a draft (run 1 died between +create and validate), it is **validated now**, so a replayed manifest converges +instead of stalling on a half-done op. A listing failure other than 404 aborts +(POSTing blind would mint duplicates — the exact erp#44 failure mode). ### 3 · Payment (règlement) — `scripts/payment-record.sh` @@ -129,10 +161,24 @@ an explicit error — never truncated silently. `arcodange-bank-reco`'s bank-mat normalizes feed ids the same way, so short-form nums keep reconciling by id (historical long-form nums too). Pass the raw feed id; the script does the rest. -Emits **`{id, bank_transaction_id, transaction_id}`**. `bank_transaction_id` is the -Dolibarr bank line (`llx_bank.fk_bank_line`) the payment created — the id the -reconciliation (`arcodange-bank-reco`) keys on. `transaction_id` echoes the -**normalized** num actually stored. Both ends are captured at write time. +**Idempotent (erp#44), composing with the normalization above.** Before any POST +the invoice's payment list is fetched and deduped by **(invoice, amount, +normalized transaction_id)**: a row whose stored num — normalized the same way, +so historical long-form Qonto nums still match — equals the normalized +`transaction_id` is a replay. Supplier payments also require the amounts to +agree (±0.005; same tx with a *different* amount **aborts** as a conflict); +customer payments settle the full remaining, so the tx id alone is the key. A +hit emits `{"id": null, "bank_transaction_id": , +"transaction_id": , "deduped": true}` without posting (`id` is null +by honesty — Dolibarr's payment list does not expose the paiement rowid; the +bank line is the stable handle). **Without a `transaction_id` there is no dedupe +key — a replay WILL double-pay**, one more reason it is always passed. + +Emits **`{id, bank_transaction_id, transaction_id, deduped}`**. +`bank_transaction_id` is the Dolibarr bank line (`llx_bank.fk_bank_line`) the +payment created — the id the reconciliation (`arcodange-bank-reco`) keys on. +`transaction_id` echoes the **normalized** num actually stored. Both ends are +captured at write time. ### 4 · Credit note (avoir) — `scripts/creditnote-create.sh` @@ -212,6 +258,16 @@ dependent ops wire up on the target. `--target sandbox` writes via `dol-write.sh `ARCO_PROMOTE_CONFIRM` is set exactly. Pair it with `dolibarr-data-snapshot` (prod before/after) to confirm only the intended records changed. +**Replay is safe (erp#44).** Every op script dedupes before POSTing, so a +manifest that failed mid-run — the 2026-07-11 rehearsal's manifest B, whose op 1 +had already created the DARNIS invoice — can simply be re-applied: already-done +ops answer `deduped=true`, the rest execute. `promote-apply` marks each op +`created` or `deduped=true` inline and totals them in the summary line +(`OK — promote complete (1 created, 2 deduped)`), so an all-`deduped` second run +is visible proof of a no-op. Live acceptance: `tests/replay-idempotency.sh` +double-applies a self-contained manifest (thirdparty + supplier invoice + +payment) on the sandbox and asserts run 2 dedupes all ops with zero new rows. + A manifest value can reference another entity two ways, both resolved against the **target** so the same file is portable sandbox↔prod: @@ -253,9 +309,21 @@ sandbox KissMetrics on `--target sandbox` and the prod one on `--target prod`. anything else (`code_client`, roles, payment conditions…) *before* any request: a typo'd field must fail loudly, not silently mutate the ledger. Offline proof in `tests/run-tests.sh`. -- **Contacts dedupe by (socid, email) then (socid, lastname+firstname)**, both - case-insensitive — `contact-create.sh` answers `{"id": …, "deduped": true}` - instead of minting a duplicate, so replaying a manifest is always safe. +- **Every create dedupes before POSTing (erp#44)** — thirdparty by exact name + (ambiguous or role-mismatch aborts), supplier invoice by (socid, ref_supplier) + (total mismatch aborts), customer invoice by (socid, date, total, line + fingerprint), payment by (invoice, amount, normalized tx id), contact by + (socid, email) then (socid, lastname+firstname) — each answering + `{"deduped": true}` instead of minting a duplicate, so replaying a manifest is + always safe. Two holes: a **payment without a `transaction_id`** has no + dedupe key and WILL double-pay on a replay, and **`creditnote-create.sh` does + not dedupe yet** (supplier-avoir parity follow-up) — do not replay a manifest + containing a creditnote op past a mid-run failure. Offline proof: + `tests/run-tests.sh`; live double-apply proof: `tests/replay-idempotency.sh`. +- **A dedupe lookup that fails (non-404) aborts the op** — the scripts refuse to + POST blind, because assuming "no match" on a 403/timeout is precisely how + duplicates get minted (cf. the `voir_tous` ACL trap in the `dolibarr` skill: + missing permissions masquerade as empty lists). - **`poste`, not `soc2`.** The job-title field on a Dolibarr contact is `poste`; `soc2` (seen in WIP operator payloads) is not a Dolibarr field and the API would drop it silently — `contact-create.sh` refuses it with a pointer to diff --git a/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/rehearsal-runbook.md b/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/rehearsal-runbook.md index d8439c0..0ea527f 100644 --- a/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/rehearsal-runbook.md +++ b/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/rehearsal-runbook.md @@ -27,6 +27,7 @@ Agent sandbox : armé (droit 251 accordé — PR de pérennisation en cours par 2. Les ids Qonto (67 c) dépassent `num_payment` varchar(50) → **forme canonique courte = suffixe UUID** (38 c) ; PR de normalisation bank-match/payment-record en cours par sous-agent. 3. Flag `paye=1` fantôme sur 2 brouillons → remis à 0 avant règlement. 4. `invoice-create` n'est pas idempotent (le re-run d'un manifeste dupliquerait) → ne jamais rejouer un manifeste partiellement appliqué ; pour prod, tout part de zéro donc manifeste B complet OK. + *Addendum (erp#44, 2026-07-18) : résolu — les trois scripts d'écriture dédupliquent désormais AVANT tout POST (fournisseur par `socid`+`ref_supplier`, client par empreinte date/total/lignes, règlement par tx normalisé, tiers par nom exact) ; le re-run du manifeste B sur la sandbox est prouvé no-op (5/5 `deduped`, zéro ligne nouvelle — `tests/replay-idempotency.sh`).* ## ➡️ À toi — replay prod (10 min, ta clé, jamais stockée) diff --git a/.claude/skills/dolibarr-sandbox-write/scripts/invoice-create.sh b/.claude/skills/dolibarr-sandbox-write/scripts/invoice-create.sh index ad80396..18b4a5d 100755 --- a/.claude/skills/dolibarr-sandbox-write/scripts/invoice-create.sh +++ b/.claude/skills/dolibarr-sandbox-write/scripts/invoice-create.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Create a customer or supplier invoice (facture) with product/service lines in -# the SANDBOX, optionally validating it. +# the SANDBOX, optionally validating it — IDEMPOTENT (erp#44): replaying the +# same create returns the existing invoice instead of minting a duplicate. # # Input: a JSON object on stdin (or a file path in $1): # socid (required) thirdparty id @@ -10,7 +11,28 @@ # validate true|false (default false = leave draft) # lines: [ { desc, qty, price_ht, tva, type: "product"|"service", product_id? } ] # -# Emits {id, ref, ref_supplier, total_ht, total_ttc, statut} on stdout. +# Idempotency (erp#44 — Learning #4 of the 2026-07-11 rehearsal, where a manifest +# that failed mid-run could not be replayed because op 1 would have re-created +# the DARNIS invoice). BEFORE any POST, list the thirdparty's invoices of the +# same kind and dedupe: +# - supplier kind WITH ref_supplier : by (socid, ref_supplier) — the supplier's +# own ref is the natural key. A match whose total_ttc DIFFERS from the +# requested lines ABORTS (same key, different content = a data conflict to +# resolve by a human, not a dedupe). +# - customer kind (or supplier without ref_supplier) : by (socid, date, +# total_ttc ±0.02, line fingerprint) where the fingerprint is the multiset of +# (desc, qty, subprice, tva_tx) per line — desc HTML-unescaped and +# whitespace-collapsed, since Dolibarr returns stored descriptions +# HTML-encoded. +# - 2+ matches ABORT (ambiguous — the target already has duplicates; never +# guess). Credit notes (type=2) are never dedupe candidates. +# On a match: if `validate:true` was requested and the match is still a DRAFT +# (statut=0 — e.g. run 1 died between create and validate), it is validated now, +# so a replayed manifest CONVERGES instead of leaving a half-done op behind. +# A listing failure other than 404 ABORTS — POSTing blind would mint duplicates. +# +# Emits {id, ref, ref_supplier, total_ht, total_ttc, statut, deduped} on stdout +# — deduped:true means no invoice was created (the id is the pre-existing one). set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}" @@ -18,8 +40,8 @@ W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}" SRC="${1:-}" if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi -PYF="$(mktemp -t dolpy.XXXXXX)"; trap 'rm -f "${PYF}"' EXIT -cat > "${PYF}" <<'PY' +TMPD="$(mktemp -d -t invcre.XXXXXX)"; trap 'rm -rf "${TMPD}"' EXIT +cat > "${TMPD}/map.py" <<'PY' import json, sys, datetime d = json.loads(sys.stdin.read()) if not d.get("socid"): @@ -27,8 +49,8 @@ if not d.get("socid"): supplier = d.get("kind", "customer").lower() in ("supplier", "fournisseur") endpoint = "/supplierinvoices" if supplier else "/invoices" ds = d.get("date") -epoch = int((datetime.datetime.strptime(ds, "%Y-%m-%d") if ds - else datetime.datetime.now()).timestamp()) +dt = datetime.datetime.strptime(ds, "%Y-%m-%d") if ds else datetime.datetime.now() +epoch = int(dt.timestamp()) lines = [] for ln in d.get("lines", []): is_product = ln.get("type", "service").lower() in ("product", "produit") @@ -45,16 +67,127 @@ for ln in d.get("lines", []): body = {"socid": d["socid"], "date": epoch, "type": 0, "lines": lines} if supplier and d.get("ref_supplier"): body["ref_supplier"] = d["ref_supplier"] +# Expected total_ttc, Dolibarr-style (per-line rounding), for the dedupe key. +total_ttc = round(sum(round(float(L["qty"]) * float(L["subprice"]), 2) + * (1 + float(L["tva_tx"]) / 100.0) for L in lines), 2) +dedupe = {"supplier": supplier, "socid": str(d["socid"]), + "ref_supplier": str(d.get("ref_supplier") or "") if supplier else "", + "date": dt.strftime("%Y-%m-%d"), "total_ttc": total_ttc, + "lines": [[L["desc"], L["qty"], L["subprice"], L["tva_tx"]] for L in lines]} print(endpoint) print(json.dumps(body)) print("1" if d.get("validate") else "0") +print(json.dumps(dedupe)) PY -MAPPED="$(printf '%s' "${INPUT}" | python3 "${PYF}")" +MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")" ENDPOINT="$(sed -n 1p <<<"${MAPPED}")" BODY="$(sed -n 2p <<<"${MAPPED}")" VALIDATE="$(sed -n 3p <<<"${MAPPED}")" +DEDUPE="$(sed -n 4p <<<"${MAPPED}")" +printf '%s' "${DEDUPE}" > "${TMPD}/dedupe.json" +# Final read-back: shared by the create and dedupe paths. +emit_summary() { # $1 = invoice id, $2 = deduped true|false + "${W}" GET "${ENDPOINT}/$1" | DEDUPED="$2" python3 -c "import json,sys,os +d=json.load(sys.stdin) +o={k:d.get(k) for k in ('id','ref','ref_supplier','total_ht','total_ttc','statut')} +o['deduped']=os.environ['DEDUPED']=='true' +print(json.dumps(o))" +} + +# --- Dedupe lookup (erp#44): list this thirdparty's invoices of the same kind --- +SOCID="$(python3 -c "import json,sys; print(json.load(sys.stdin)['socid'])" < "${TMPD}/dedupe.json")" +set +e +"${W}" GET "${ENDPOINT}?thirdparty_ids=${SOCID}&limit=500" \ + > "${TMPD}/list.json" 2> "${TMPD}/list.err" +rc=$? +set -e +if [[ ${rc} -ne 0 ]]; then + if grep -q "HTTP 404" "${TMPD}/list.err"; then + printf '[]' > "${TMPD}/list.json" # empty lists can answer 404 (dolibarr skill gotcha) + else + cat "${TMPD}/list.err" >&2 + echo "invoice-create.sh: could not list ${ENDPOINT} for socid ${SOCID} — refusing to POST blind (dedupe impossible)" >&2 + exit 1 + fi +fi + +cat > "${TMPD}/match.py" <<'PY' +import json, sys, html, datetime +ded = json.load(open(sys.argv[1])) +try: + rows = json.load(open(sys.argv[2])) +except Exception: + rows = [] +rows = [r for r in (rows if isinstance(rows, list) else []) + if str(r.get("type", "0")) != "2"] # credit notes never dedupe a create + +def norm_desc(s): + return " ".join(html.unescape(str(s or "")).split()) +def fnum(v, nd): + try: return round(float(v), nd) + except Exception: return None +def fingerprint(lines): + return sorted((norm_desc(l[0]), fnum(l[1], 3), fnum(l[2], 2), fnum(l[3], 3)) + for l in lines) + +TOL = 0.02 +want_fp = fingerprint(ded["lines"]) +hits = [] +if ded["supplier"] and ded["ref_supplier"]: + key = ded["ref_supplier"].strip() + hits = [r for r in rows if str(r.get("ref_supplier") or "").strip() == key] + if len(hits) == 1 and ded["lines"]: + got = fnum(hits[0].get("total_ttc"), 2) + if got is not None and abs(got - ded["total_ttc"]) > TOL: + sys.exit("invoice-create.sh: ABORT — supplier invoice with ref_supplier " + "%r already exists on socid %s (id %s) but its total_ttc %.2f " + "differs from the requested %.2f. Same key, different content " + "is a conflict to resolve, not a dedupe." % + (key, ded["socid"], hits[0].get("id"), got, ded["total_ttc"])) +else: + for r in rows: + try: + rdate = datetime.datetime.fromtimestamp(int(r.get("date"))).strftime("%Y-%m-%d") + except Exception: + continue + if rdate != ded["date"]: + continue + got = fnum(r.get("total_ttc"), 2) + if got is None or abs(got - ded["total_ttc"]) > TOL: + continue + rl = r.get("lines") + if not isinstance(rl, list): + sys.exit("invoice-create.sh: ABORT — candidate invoice id %s matches " + "(date, total) but carries no inline lines to fingerprint; " + "refusing to guess" % r.get("id")) + if fingerprint([[l.get("desc"), l.get("qty"), l.get("subprice"), l.get("tva_tx")] + for l in rl]) == want_fp: + hits.append(r) +if len(hits) > 1: + sys.exit("invoice-create.sh: ABORT — dedupe key matches %d invoices on socid %s " + "(ids %s): the target already holds duplicates; refusing to guess." + % (len(hits), ded["socid"], ", ".join(str(h.get("id")) for h in hits))) +if hits: + print(json.dumps({"id": int(hits[0]["id"]), "statut": str(hits[0].get("statut"))})) +PY +MATCH="$(python3 "${TMPD}/match.py" "${TMPD}/dedupe.json" "${TMPD}/list.json")" + +if [[ -n "${MATCH}" ]]; then + ID="$(python3 -c "import json,sys; print(json.load(sys.stdin)['id'])" <<<"${MATCH}")" + STATUT="$(python3 -c "import json,sys; print(json.load(sys.stdin)['statut'])" <<<"${MATCH}")" + echo "invoice-create.sh: invoice already exists on socid ${SOCID} (id ${ID}) — deduped, no POST" >&2 + # Converge an interrupted run: validate the matched draft if validation was asked. + if [[ "${VALIDATE}" == "1" && "${STATUT}" == "0" ]]; then + echo "invoice-create.sh: matched invoice ${ID} is still a draft — validating it now (completes the interrupted op)" >&2 + "${W}" POST "${ENDPOINT}/${ID}/validate" '{}' >/dev/null + fi + emit_summary "${ID}" true + exit 0 +fi + +# --- Create (no match) ---------------------------------------------------------- ID="$("${W}" POST "${ENDPOINT}" "${BODY}")" if [[ ! "${ID}" =~ ^[0-9]+$ ]]; then echo "invoice-create.sh: create did not return an id: ${ID}" >&2 @@ -63,6 +196,4 @@ fi if [[ "${VALIDATE}" == "1" ]]; then "${W}" POST "${ENDPOINT}/${ID}/validate" '{}' >/dev/null fi -"${W}" GET "${ENDPOINT}/${ID}" | python3 -c "import json,sys -d=json.load(sys.stdin) -print(json.dumps({k:d.get(k) for k in ('id','ref','ref_supplier','total_ht','total_ttc','statut')}))" +emit_summary "${ID}" false diff --git a/.claude/skills/dolibarr-sandbox-write/scripts/payment-record.sh b/.claude/skills/dolibarr-sandbox-write/scripts/payment-record.sh index fdd1dcc..e1c9cec 100755 --- a/.claude/skills/dolibarr-sandbox-write/scripts/payment-record.sh +++ b/.claude/skills/dolibarr-sandbox-write/scripts/payment-record.sh @@ -1,5 +1,7 @@ #!/usr/bin/env bash -# Record a payment (règlement) on a validated invoice in the SANDBOX. +# Record a payment (règlement) on a validated invoice in the SANDBOX — +# IDEMPOTENT (erp#44): replaying the same règlement is a no-op, not a double +# payment. # # Input: a JSON object on stdin (or a file path in $1): # invoice_id (required) the invoice to pay @@ -21,12 +23,28 @@ # normalization is REFUSED with an error — never truncated silently. # comment (optional) # +# Idempotency (erp#44, composing with the erp#37 normalization): BEFORE any POST, +# the invoice's payment list is fetched and deduped by +# (invoice, amount, normalized transaction_id): +# - a row whose num — normalized the same way (historical rows may still carry +# long-form Qonto ids) — equals the normalized transaction_id is a REPLAY: +# for supplier payments the amounts must also agree (±0.005; same tx id with +# a DIFFERENT amount ABORTS as a data conflict); customer payments settle the +# full remaining so the tx id alone is the key. +# - a dedupe hit emits {"id": null, "bank_transaction_id": , +# "transaction_id": , "deduped": true} and exits 0 WITHOUT +# posting (the payments list does not expose the paiement rowid — id is null +# by honesty, the bank line is the stable handle reconciliation keys on). +# - the SAME normalized tx appearing on 2+ rows of this invoice ABORTS (the +# target already holds duplicates; never guess). +# - WITHOUT a transaction_id there is NO dedupe key — the payment posts with a +# warning (as before) and a replay WILL duplicate it. Always pass the tx id. +# - a listing failure other than 404 ABORTS: paying blind would double-pay. +# # The invoice must be VALIDATED first (invoice-create.sh ... "validate":true). -# Emits {id, bank_transaction_id, transaction_id} on stdout — `transaction_id` is -# the NORMALIZED num actually stored (what bank-match keys on). `bank_transaction_id` -# is the Dolibarr bank line (llx_bank.fk_bank_line) the payment created — the id -# bank reconciliation (arcodange-bank-reco) keys on to link this règlement to a -# statement line. Recording without a transaction_id warns (it won't auto-reconcile). +# Emits {id, bank_transaction_id, transaction_id, deduped} on stdout — +# `transaction_id` is the NORMALIZED num actually stored (what bank-match keys +# on), `bank_transaction_id` the Dolibarr bank line (llx_bank.fk_bank_line). set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}" @@ -34,9 +52,9 @@ W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}" SRC="${1:-}" if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi -PYF="$(mktemp -t dolpy.XXXXXX)"; PYF2="$(mktemp -t dolpy2.XXXXXX)" -trap 'rm -f "${PYF}" "${PYF2}"' EXIT -cat > "${PYF}" <<'PY' +TMPD="$(mktemp -d -t payrec.XXXXXX)" +trap 'rm -rf "${TMPD}"' EXIT +cat > "${TMPD}/map.py" <<'PY' import json, sys, datetime, re d = json.loads(sys.stdin.read()) if not d.get("invoice_id"): @@ -55,7 +73,7 @@ epoch = int((datetime.datetime.strptime(ds, "%Y-%m-%d") if ds inv = d["invoice_id"] # transaction_id is the first-class bank-feed tx id; num is the back-compat alias. raw_tx = str(d.get("transaction_id") or d.get("num") or "") -# Normalize to the canonical short form. Dolibarr stores num_payment in +# Normalize to the canonical short form (erp#37). Dolibarr stores num_payment in # varchar(50) (llx_paiement.num_paiement / llx_paiementfourn.num_paiement) and # a Qonto id (~67 chars, ---transaction-) blows past it — # HTTP 400 "value too long for type character varying(50)". Strip everything @@ -73,7 +91,8 @@ if len(tx) > 50: % (tx, len(tx))) if not tx: sys.stderr.write("payment-record.sh: WARNING — no transaction_id given; this " - "règlement won't auto-reconcile to the bank feed\n") + "règlement won't auto-reconcile to the bank feed AND cannot " + "be deduped on a replay (erp#44) — a re-run will double-pay\n") if supplier: if d.get("amount") is None: sys.exit("payment-record.sh: supplier payments require an 'amount'") @@ -90,12 +109,49 @@ else: print(endpoint) print(json.dumps(body)) print(tx) +print(json.dumps({"supplier": supplier, + "amount": (float(d["amount"]) if supplier else None)})) +PY + +# Dedupe a replay against the invoice's existing payments (erp#44). +cat > "${TMPD}/dedupe.py" <<'PY' +import json, sys, os, re +ded = json.loads(os.environ["DEDUPE"]) +tx = os.environ.get("TX", "") +try: + rows = json.load(sys.stdin) +except Exception: + rows = [] +rows = rows if isinstance(rows, list) else [] +norm = lambda s: re.sub(r'^.*transaction-', '', str(s or "")) # erp#37, both sides +hits = [r for r in rows if tx and norm(r.get("num")) == tx] +if not hits: + sys.exit(0) # no match -> proceed to POST +if len(hits) > 1: + sys.exit("payment-record.sh: ABORT — transaction_id %r already appears on %d " + "payments of this invoice; the target holds duplicates, refusing to " + "guess" % (tx, len(hits))) +r = hits[0] +if ded["supplier"] and ded["amount"] is not None: + try: + got = float(r.get("amount")) + except (TypeError, ValueError): + got = None + if got is None or abs(got - ded["amount"]) > 0.005: + sys.exit("payment-record.sh: ABORT — transaction_id %r is already recorded " + "on this invoice with amount %s, but %s was requested. Same tx, " + "different amount is a conflict to resolve, not a dedupe." + % (tx, r.get("amount"), ded["amount"])) +btx = r.get("fk_bank_line") +print(json.dumps({"id": None, + "bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx, + "transaction_id": tx, "deduped": True})) PY # Correlate the created payment back to its bank transaction line. The payments # list carries fk_bank_line but not the paiement rowid, so match on the provided # transaction_id (the external bank ref), else fall back to the most recent line. -cat > "${PYF2}" <<'PY' +cat > "${TMPD}/correlate.py" <<'PY' import json, sys, os rows = json.load(sys.stdin); rows = rows if isinstance(rows, list) else [] tx = os.environ.get("TX", ""); pid = int(os.environ["PAYID"]) @@ -111,13 +167,38 @@ btx = (pick or {}).get("fk_bank_line") # the JSON always reports the canonical short form even on the recency fallback. print(json.dumps({"id": pid, "bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx, - "transaction_id": (pick or {}).get("num", "") or tx})) + "transaction_id": (pick or {}).get("num", "") or tx, + "deduped": False})) PY -MAPPED="$(printf '%s' "${INPUT}" | python3 "${PYF}")" +MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")" ENDPOINT="$(sed -n 1p <<<"${MAPPED}")" BODY="$(sed -n 2p <<<"${MAPPED}")" TX="$(sed -n 3p <<<"${MAPPED}")" +DEDUPE="$(sed -n 4p <<<"${MAPPED}")" + +# --- Pre-POST dedupe (erp#44) — only possible when a transaction_id was given --- +if [[ -n "${TX}" ]]; then + set +e + "${W}" GET "${ENDPOINT}" > "${TMPD}/payments.json" 2> "${TMPD}/list.err" + rc=$? + set -e + if [[ ${rc} -ne 0 ]]; then + if grep -q "HTTP 404" "${TMPD}/list.err"; then + printf '[]' > "${TMPD}/payments.json" # empty lists can answer 404 (dolibarr skill gotcha) + else + cat "${TMPD}/list.err" >&2 + echo "payment-record.sh: could not list ${ENDPOINT} — refusing to pay blind (dedupe impossible)" >&2 + exit 1 + fi + fi + MATCH="$(DEDUPE="${DEDUPE}" TX="${TX}" python3 "${TMPD}/dedupe.py" < "${TMPD}/payments.json")" + if [[ -n "${MATCH}" ]]; then + echo "payment-record.sh: transaction_id ${TX} already recorded on this invoice — deduped, no POST" >&2 + printf '%s\n' "${MATCH}" + exit 0 + fi +fi PAYID="$("${W}" POST "${ENDPOINT}" "${BODY}")" if [[ ! "${PAYID}" =~ ^[0-9]+$ ]]; then @@ -126,4 +207,4 @@ if [[ ! "${PAYID}" =~ ^[0-9]+$ ]]; then fi # Same path serves the GET list; resolve fk_bank_line and emit the enriched record. -"${W}" GET "${ENDPOINT}" | PAYID="${PAYID}" TX="${TX}" python3 "${PYF2}" +"${W}" GET "${ENDPOINT}" | PAYID="${PAYID}" TX="${TX}" python3 "${TMPD}/correlate.py" diff --git a/.claude/skills/dolibarr-sandbox-write/scripts/promote-apply.sh b/.claude/skills/dolibarr-sandbox-write/scripts/promote-apply.sh index 55d34a4..84c8440 100755 --- a/.claude/skills/dolibarr-sandbox-write/scripts/promote-apply.sh +++ b/.claude/skills/dolibarr-sandbox-write/scripts/promote-apply.sh @@ -40,6 +40,7 @@ OP_SCRIPT = {"thirdparty": "thirdparty-create.sh", "invoice": "invoice-create.sh "creditnote": "creditnote-create.sh", "payment": "payment-record.sh", "thirdparty_update": "thirdparty-update.sh", "contact": "contact-create.sh"} refmap = {} +n_created = n_deduped = 0 import urllib.parse DOL_WRITE = os.environ.get("DOL_WRITE") # GET wrapper for the chosen target @@ -117,16 +118,28 @@ for i, op in enumerate(ops, 1): ref = op.get("ref") if ref and rid is not None: refmap[ref] = int(rid) if str(rid).isdigit() else rid - # Surface the idempotency evidence inline: contact dedupes and - # thirdparty-update read-back diffs are the proof a re-apply is a no-op. + # Surface the idempotency evidence inline: every op script reports + # deduped=true/false (erp#44), and thirdparty-update reports its read-back + # diff — together the proof a re-apply is a no-op. extra = "" if isinstance(parsed, dict): if parsed.get("deduped"): + n_deduped += 1 extra += " deduped=true (already on target — no write)" + elif "deduped" in parsed: + n_created += 1 + extra += " created" ch = parsed.get("changed") if isinstance(ch, dict): extra += " changed=%d%s" % (len(ch), (" [%s]" % ", ".join(sorted(ch))) if ch else " (no-op)") - print(" [%d/%d] %-17s %-8s -> id=%s%s" % (i, len(ops), t, ("@" + ref) if ref else "", rid, extra)) -print("OK — promote complete. ref -> id: %s" % json.dumps(refmap)) + print(" [%d/%d] %-17s %-8s -> id=%s%s" % (i, len(ops), t, ("@" + ref) if ref else "", + rid if rid is not None else "-", extra)) +counts = [] +if n_created: + counts.append("%d created" % n_created) +if n_deduped: + counts.append("%d deduped" % n_deduped) +print("OK — promote complete%s. ref -> id: %s" + % ((" (%s)" % ", ".join(counts)) if counts else "", json.dumps(refmap))) PY diff --git a/.claude/skills/dolibarr-sandbox-write/scripts/promote-plan.sh b/.claude/skills/dolibarr-sandbox-write/scripts/promote-plan.sh index e76868f..22299e3 100755 --- a/.claude/skills/dolibarr-sandbox-write/scripts/promote-plan.sh +++ b/.claude/skills/dolibarr-sandbox-write/scripts/promote-plan.sh @@ -17,11 +17,17 @@ for i, op in enumerate(ops, 1): t = op["op"]; inp = op.get("input", {}); ref = op.get("ref") print(" %d. %s%s" % (i, t, (" => @%s" % ref) if ref else "")) if t == "thirdparty": - print(" name=%r role=%s%s" % (inp.get("name"), inp.get("role", "client"), + print(" name=%r role=%s%s (idempotent: dedupe by exact name; ambiguous aborts)" + % (inp.get("name"), inp.get("role", "client"), (" tva=%s" % inp["tva_intra"]) if inp.get("tva_intra") else "")) elif t == "invoice": - print(" socid=%s kind=%s%s validate=%s" % (inp.get("socid"), inp.get("kind", "customer"), - (" ref_supplier=%s" % inp["ref_supplier"]) if inp.get("ref_supplier") else "", bool(inp.get("validate")))) + supplier = str(inp.get("kind", "customer")).lower() in ("supplier", "fournisseur") + dk = ("socid+ref_supplier" if supplier and inp.get("ref_supplier") + else "socid+date+total+lines") + print(" socid=%s kind=%s%s validate=%s (idempotent: dedupe by %s)" + % (inp.get("socid"), inp.get("kind", "customer"), + (" ref_supplier=%s" % inp["ref_supplier"]) if inp.get("ref_supplier") else "", + bool(inp.get("validate")), dk)) for ln in inp.get("lines", []): print(" - %r qty=%s pu_ht=%s tva=%s%% [%s]" % (ln.get("desc", ""), ln.get("qty", 1), ln.get("price_ht", ln.get("subprice")), ln.get("tva", ln.get("tva_tx", 20)), ln.get("type", "service"))) @@ -34,7 +40,8 @@ for i, op in enumerate(ops, 1): txid = inp.get("transaction_id") or inp.get("num") print(" invoice=%s mode=%s account=%s %s%s" % (inp.get("invoice_id"), inp.get("mode", "VIR"), inp.get("account_id"), ("amount=%s" % inp["amount"]) if inp.get("amount") else "(full)", - (" tx=%s" % txid) if txid else " tx=MISSING")) + (" tx=%s (idempotent: dedupe by invoice+amount+tx)" % txid) if txid + else " tx=MISSING (no dedupe key — a replay WILL double-pay)")) elif t == "thirdparty_update": flds = inp.get("fields") or {} print(" socid=%s update %d dossier field(s): %s" % (inp.get("socid"), len(flds), diff --git a/.claude/skills/dolibarr-sandbox-write/scripts/thirdparty-create.sh b/.claude/skills/dolibarr-sandbox-write/scripts/thirdparty-create.sh index 85d1b40..d811e6a 100755 --- a/.claude/skills/dolibarr-sandbox-write/scripts/thirdparty-create.sh +++ b/.claude/skills/dolibarr-sandbox-write/scripts/thirdparty-create.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash -# Create a client and/or supplier thirdparty (fiche tiers) in the SANDBOX. +# Create a client and/or supplier thirdparty (fiche tiers) in the SANDBOX — +# IDEMPOTENT (erp#44): replaying the same create is a no-op, not a duplicate. # # Input: a JSON object on stdin (or a file path in $1). Fields: # name (required) @@ -8,8 +9,21 @@ # client_code / supplier_code default "-1" = auto-generate via the code mask # siret, tva_intra, address, zip, town, email, phone, idprof1 (optional) # -# Emits the new thirdparty id on stdout. All writes go through dol-write.sh, -# which refuses any host that is not the sandbox. +# Idempotency (erp#44): BEFORE any POST, look the name up on the target with the +# same semantics as promote-apply's `#thirdparty:name=...` lookup +# (GET /thirdparties?sqlfilters=(t.nom:=:'name'), limit 2): +# - 0 matches (the API answers HTTP 404, not []) -> create +# - 1 match whose roles cover the requested role -> {"id": , "deduped": true} +# - 1 match MISSING the requested role -> ABORT (refuse-never-repair: +# silently reusing a client fiche as a supplier would skip the code mask and +# hide a data problem — fix the fiche deliberately, not as a create side effect) +# - 2+ matches -> ABORT (ambiguous, never guess) +# Any other listing failure ABORTS: assuming "no match" on e.g. a 403 would mint +# duplicates — the exact failure mode this dedupe exists to prevent. +# +# Emits {"id": N, "deduped": false} after a create, {"id": N, "deduped": true} +# after a dedupe hit. All writes go through dol-write.sh, which refuses any host +# that is not the sandbox. # # Examples: # echo '{"name":"KissMetrics","role":"client","tva_intra":"US.."}' | thirdparty-create.sh @@ -21,9 +35,11 @@ W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}" SRC="${1:-}" if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi -PYF="$(mktemp -t dolpy.XXXXXX)"; trap 'rm -f "${PYF}"' EXIT -cat > "${PYF}" <<'PY' -import json, sys +TMPD="$(mktemp -d -t tpcre.XXXXXX)"; trap 'rm -rf "${TMPD}"' EXIT + +# --- 1. Validate + map the POST body (before ANY request) ---------------------- +cat > "${TMPD}/map.py" <<'PY' +import json, sys, urllib.parse d = json.loads(sys.stdin.read()) if not d.get("name"): sys.exit("thirdparty-create.sh: 'name' is required") @@ -43,8 +59,74 @@ body = { for k in ("siret", "tva_intra", "address", "zip", "town", "email", "phone", "idprof1"): if d.get(k): body[k] = d[k] +# Line 1: POST body. Line 2: the dedupe lookup path (exact name, promote-apply +# `#thirdparty:name=` semantics — SQL-escape ' by doubling, then URL-encode). print(json.dumps(body)) +flt = "(t.nom:=:'%s')" % str(d["name"]).replace("'", "''") +print("/thirdparties?limit=2&sqlfilters=" + urllib.parse.quote(flt)) PY +MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")" +BODY="$(sed -n 1p <<<"${MAPPED}")" +LOOKUP="$(sed -n 2p <<<"${MAPPED}")" +printf '%s' "${BODY}" > "${TMPD}/body.json" -BODY="$(printf '%s' "${INPUT}" | python3 "${PYF}")" -"${W}" POST /thirdparties "${BODY}" +# --- 2. Dedupe by exact name against the target (erp#44) ----------------------- +set +e +"${W}" GET "${LOOKUP}" > "${TMPD}/list.json" 2> "${TMPD}/list.err" +rc=$? +set -e +if [[ ${rc} -ne 0 ]]; then + if grep -q "HTTP 404" "${TMPD}/list.err"; then + printf '[]' > "${TMPD}/list.json" # no-match answers 404, not [] (dolibarr skill gotcha) + else + cat "${TMPD}/list.err" >&2 + echo "thirdparty-create.sh: could not look up name on the target — refusing to POST blind (dedupe impossible)" >&2 + exit 1 + fi +fi + +cat > "${TMPD}/match.py" <<'PY' +import json, sys +body = json.load(open(sys.argv[1])) +try: + rows = json.load(open(sys.argv[2])) +except Exception: + rows = [] +rows = rows if isinstance(rows, list) else [] +if len(rows) > 1: + sys.exit("thirdparty-create.sh: ABORT — name %r matches %d thirdparties on the " + "target (ids %s); ambiguous, refusing to guess (same rule as the " + "promote '#thirdparty:name=' lookup)" + % (body["name"], len(rows), ", ".join(str(r.get("id")) for r in rows))) +if rows: + r = rows[0] + want_client = body["client"] == "1" + want_supp = body["fournisseur"] == "1" + # Dolibarr: client '1'=customer '2'=prospect '3'=both; fournisseur '1'=yes. + has_client = str(r.get("client") or "0") in ("1", "2", "3") + has_supp = str(r.get("fournisseur") or "0") == "1" + missing = [] + if want_client and not has_client: missing.append("client") + if want_supp and not has_supp: missing.append("supplier") + if missing: + sys.exit("thirdparty-create.sh: ABORT — %r already exists (id %s) but " + "without the requested role(s): %s. Refusing to dedupe onto a " + "fiche that can't carry the ops that follow, and refusing to " + "mutate its roles as a create side effect — fix the fiche " + "deliberately first." % (body["name"], r.get("id"), ", ".join(missing))) + print(json.dumps({"id": int(r["id"]), "deduped": True})) +PY +MATCH="$(python3 "${TMPD}/match.py" "${TMPD}/body.json" "${TMPD}/list.json")" +if [[ -n "${MATCH}" ]]; then + echo "thirdparty-create.sh: thirdparty already exists — deduped, no POST" >&2 + printf '%s\n' "${MATCH}" + exit 0 +fi + +# --- 3. POST the new thirdparty ------------------------------------------------- +NEWID="$("${W}" POST /thirdparties "${BODY}")" +if [[ ! "${NEWID}" =~ ^[0-9]+$ ]]; then + echo "thirdparty-create.sh: create did not return an id: ${NEWID}" >&2 + exit 1 +fi +printf '{"id": %s, "deduped": false}\n' "${NEWID}" diff --git a/.claude/skills/dolibarr-sandbox-write/tests/replay-idempotency.sh b/.claude/skills/dolibarr-sandbox-write/tests/replay-idempotency.sh new file mode 100755 index 0000000..dd1ecf8 --- /dev/null +++ b/.claude/skills/dolibarr-sandbox-write/tests/replay-idempotency.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# erp#44 acceptance — LIVE double-apply on the SANDBOX: applying the same +# manifest twice must be a no-op the second time (all ops deduped, zero new +# rows). This is the replay that was IMPOSSIBLE after the 2026-07-11 rehearsal's +# manifest B failed mid-run (Learning #4: re-applying would have duplicated the +# DARNIS invoice). +# +# What it does (writes go ONLY through the host-guarded dol-write.sh): +# 1. builds a small self-contained manifest with a unique-per-run fixture: +# one supplier thirdparty + one validated supplier invoice (Qonto-style +# transaction id, so the erp#37 normalization is exercised too) + one payment +# 2. applies it → expects 3 created, no dedupe +# 3. applies it AGAIN → expects 3 deduped, zero new rows (verified by +# row-counting thirdparties / invoices / payments via the API) +# +# Sandbox etiquette: the fixture rows stay behind (the sandbox is disposable; +# a checkpoint refresh reclaims them). Run from anywhere: +# .claude/skills/dolibarr-sandbox-write/tests/replay-idempotency.sh [evidence-dir] +set -euo pipefail +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +SCRIPTS="${SCRIPT_DIR}/../scripts" +W="${SCRIPTS}/dol-write.sh" # deliberately NOT $DOL_WRITE — this test is live +unset DOL_WRITE || true +EV="${1:-$(mktemp -d -t idem44.XXXXXX)}" +mkdir -p "${EV}" + +fail() { echo "FAIL: $*" >&2; exit 1; } + +STAMP="$(date +%Y%m%d-%H%M%S)" +NAME="IDEM44 Replay Fixture ${STAMP}" +REFSUP="IDEM44-${STAMP}" +TX="arcodange-idem44-transaction-${STAMP}-e2e-replay" # Qonto-style long form +TX_SHORT="${STAMP}-e2e-replay" # its normalized num +cat > "${EV}/manifest.json" < 0) + local out + if out="$("${W}" GET "$1" 2>/dev/null)"; then + python3 -c "import json,sys; r=json.load(sys.stdin); print(len(r) if isinstance(r,list) else 0)" <<<"${out}" + else + echo 0 + fi +} +FLT="$(python3 -c "import urllib.parse,sys; print(urllib.parse.quote(\"(t.nom:=:'%s')\" % sys.argv[1]))" "${NAME}")" +tp_count() { count_rows "/thirdparties?limit=100&sqlfilters=${FLT}"; } +inv_count() { count_rows "/supplierinvoices?thirdparty_ids=$1&limit=500"; } +pay_count() { count_rows "/supplierinvoices/$1/payments"; } + +echo "== erp#44 replay-idempotency — fixture ${STAMP} (evidence: ${EV}) ==" +[[ "$(tp_count)" == "0" ]] || fail "fixture name already exists on the sandbox (clock collision?)" + +echo; echo "-- RUN 1: expect 3 created ------------------------------------------------" +"${SCRIPTS}/promote-apply.sh" "${EV}/manifest.json" --target sandbox \ + | tee "${EV}/run1.out" +grep -q 'deduped' "${EV}/run1.out" && fail "run 1: nothing may dedupe on a fresh fixture" +[[ "$(grep -c ' created' "${EV}/run1.out")" == "3" ]] || fail "run 1: expected 3 created ops" +grep -q '(3 created)' "${EV}/run1.out" || fail "run 1: summary must say (3 created)" + +TPID="$(python3 -c "import json,sys,re +m=re.search(r'ref -> id: (\{.*\})', open(sys.argv[1]).read()); print(json.loads(m.group(1))['tp'])" "${EV}/run1.out")" +INVID="$(python3 -c "import json,sys,re +m=re.search(r'ref -> id: (\{.*\})', open(sys.argv[1]).read()); print(json.loads(m.group(1))['inv'])" "${EV}/run1.out")" + +TP1="$(tp_count)"; INV1="$(inv_count "${TPID}")"; PAY1="$(pay_count "${INVID}")" +echo "row counts after run 1: thirdparties=${TP1} invoices=${INV1} payments=${PAY1}" | tee "${EV}/counts-run1.txt" +[[ "${TP1}" == "1" && "${INV1}" == "1" && "${PAY1}" == "1" ]] || fail "run 1 must have created exactly 1 of each" + +echo; echo "-- RUN 2 (same manifest): expect 3 deduped, zero new rows -----------------" +"${SCRIPTS}/promote-apply.sh" "${EV}/manifest.json" --target sandbox \ + | tee "${EV}/run2.out" +[[ "$(grep -c 'deduped=true' "${EV}/run2.out")" == "3" ]] || fail "run 2: all 3 ops must dedupe" +grep -q ' created' "${EV}/run2.out" && fail "run 2: nothing may be created on a replay" +grep -q '(3 deduped)' "${EV}/run2.out" || fail "run 2: summary must say (3 deduped)" + +TP2="$(tp_count)"; INV2="$(inv_count "${TPID}")"; PAY2="$(pay_count "${INVID}")" +echo "row counts after run 2: thirdparties=${TP2} invoices=${INV2} payments=${PAY2}" | tee "${EV}/counts-run2.txt" +[[ "${TP2}" == "${TP1}" && "${INV2}" == "${INV1}" && "${PAY2}" == "${PAY1}" ]] \ + || fail "run 2 must add ZERO rows (run1: ${TP1}/${INV1}/${PAY1}, run2: ${TP2}/${INV2}/${PAY2})" + +# The payment's stored num must be the erp#37 canonical short form. +"${W}" GET "/supplierinvoices/${INVID}/payments" > "${EV}/payments.json" +grep -q "\"${TX_SHORT}\"" "${EV}/payments.json" \ + || fail "stored num must be the normalized short form ${TX_SHORT}" + +echo +echo "PASS: replay is a no-op — run 1 created 3 rows (tp=${TPID}, inv=${INVID}), run 2 deduped all 3, row counts unchanged (${TP2}/${INV2}/${PAY2})." +echo "Evidence in ${EV}: manifest.json run1.out run2.out counts-run*.txt payments.json" diff --git a/.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh b/.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh index 543a9a9..519673b 100755 --- a/.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh +++ b/.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh @@ -16,11 +16,25 @@ # 6. happy path: thirdparty_update + contact through promote-apply # --target sandbox (stubbed); a second apply is a proven no-op # (changed=0 for the fiche, deduped=true for the contact). +# idempotency keys (erp#44): +# 7. invoice-create supplier dedupe by (socid, ref_supplier) → no POST, +# deduped:true; same key + different total ABORTS as a conflict. +# 8. invoice-create customer dedupe by (socid, date, total, line fingerprint) +# → no POST; a different desc misses and creates. +# 9. payment-record dedupe by (invoice, amount, normalized tx) → no POST; +# same tx + different amount ABORTS; a historical LONG-form stored num +# still dedupes (composes with the erp#37 normalization). +# 10. thirdparty-create dedupe by exact name → no POST; ambiguous ABORTS; +# an existing fiche missing the requested role ABORTS; a miss creates. +# 11. a deduped DRAFT with validate:true is validated on replay (converges an +# op that died between create and validate). set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" PR="${SCRIPT_DIR}/../scripts/payment-record.sh" TU="${SCRIPT_DIR}/../scripts/thirdparty-update.sh" CC="${SCRIPT_DIR}/../scripts/contact-create.sh" +IC="${SCRIPT_DIR}/../scripts/invoice-create.sh" +TC="${SCRIPT_DIR}/../scripts/thirdparty-create.sh" PA="${SCRIPT_DIR}/../scripts/promote-apply.sh" PP="${SCRIPT_DIR}/../scripts/promote-plan.sh" STUB="${SCRIPT_DIR}/stub-dol-write.sh" @@ -30,6 +44,8 @@ fail() { echo "FAIL: $*" >&2; exit 1; } bash -n "${PR}" || fail "bash -n payment-record.sh" bash -n "${TU}" || fail "bash -n thirdparty-update.sh" bash -n "${CC}" || fail "bash -n contact-create.sh" +bash -n "${IC}" || fail "bash -n invoice-create.sh" +bash -n "${TC}" || fail "bash -n thirdparty-create.sh" bash -n "${PA}" || fail "bash -n promote-apply.sh" bash -n "${PP}" || fail "bash -n promote-plan.sh" bash -n "${STUB}" || fail "bash -n stub-dol-write.sh" @@ -139,4 +155,175 @@ grep -q -- '-> id=88' <<<"${OUT2}" || fail "run 2: dedupe must return the [[ ! -f "${S6}/contact_post_body.json" ]] || fail "run 2: must NOT POST a duplicate contact" echo "OK: promote-apply happy path — run 1 applies (changed=2, contact id 88), run 2 is a no-op (changed=0, deduped)" +# ============================ erp#44 idempotency ============================== +EPOCH_0630="$(python3 -c "import datetime; print(int(datetime.datetime(2026,6,30).timestamp()))")" +EPOCH_0531="$(python3 -c "import datetime; print(int(datetime.datetime(2026,5,31).timestamp()))")" + +# --- Case 7: supplier invoice dedupe by (socid, ref_supplier) --- +S7="$(mktemp -d -t ictest7.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}"' EXIT +cat > "${S7}/supplierinvoices.json" < "${S7}/invoice_detail_5.json" +IN7='{"socid":7,"kind":"supplier","date":"2026-06-30","ref_supplier":"F1045","validate":true, + "lines":[{"desc":"Apport d'"'"'affaire – juin 2026","qty":1,"price_ht":214.70,"tva":20,"type":"service"}]}' +OUT="$(printf '%s' "${IN7}" | DOL_WRITE="${STUB}" STUB_STATE="${S7}" bash "${IC}" 2>/dev/null)" \ + || fail "si-dedupe: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o['id'] == '5' and o['deduped'] is True, o +" || fail "si-dedupe: must return existing id 5 with deduped:true, got: ${OUT}" +[[ ! -f "${S7}/post_body.json" ]] || fail "si-dedupe: must NOT POST when ref_supplier matches" +[[ ! -f "${S7}/validated_endpoint" ]] || fail "si-dedupe: an already-validated match must NOT be re-validated" +# 7b — same (socid, ref_supplier) but different total = conflict, never a dedupe +rc=0 +printf '%s' '{"socid":7,"kind":"supplier","date":"2026-06-30","ref_supplier":"F1045", + "lines":[{"desc":"X","qty":1,"price_ht":999,"tva":20,"type":"service"}]}' \ + | DOL_WRITE="${STUB}" STUB_STATE="${S7}" bash "${IC}" >/dev/null 2>"${S7}/stderr7b" || rc=$? +[[ "${rc}" -ne 0 ]] || fail "si-conflict: same ref_supplier + different total must abort" +grep -q 'conflict' "${S7}/stderr7b" || fail "si-conflict: error must say it is a conflict" +[[ ! -f "${S7}/post_body.json" ]] || fail "si-conflict: must NOT POST on a conflict" +echo "OK: supplier invoice dedupe — (socid, ref_supplier) hit returns id, no POST; total mismatch aborts" + +# --- Case 8: customer invoice dedupe by (socid, date, total, line fingerprint) --- +S8="$(mktemp -d -t ictest8.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}"' EXIT +cat > "${S8}/invoices.json" < "${S8}/invoice_detail_21.json" +OUT="$(printf '%s' '{"socid":1,"kind":"customer","date":"2026-05-31", + "lines":[{"desc":"Prestation mai","qty":1,"price_ht":1020,"tva":0,"type":"service"}]}' \ + | DOL_WRITE="${STUB}" STUB_STATE="${S8}" bash "${IC}" 2>/dev/null)" \ + || fail "ci-dedupe: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o['id'] == '21' and o['deduped'] is True, o +" || fail "ci-dedupe: must return existing id 21 with deduped:true, got: ${OUT}" +[[ ! -f "${S8}/post_body.json" ]] || fail "ci-dedupe: must NOT POST when the fingerprint matches" +# 8b — a different desc breaks the fingerprint: the invoice is CREATED +printf '%s' '{"id":"77","ref":"FAC004","ref_supplier":null,"total_ht":"1020.00000000","total_ttc":"1020.00000000","statut":"0"}' \ + > "${S8}/invoice_detail_77.json" +OUT="$(printf '%s' '{"socid":1,"kind":"customer","date":"2026-05-31", + "lines":[{"desc":"Prestation juin","qty":1,"price_ht":1020,"tva":0,"type":"service"}]}' \ + | DOL_WRITE="${STUB}" STUB_STATE="${S8}" bash "${IC}" 2>/dev/null)" \ + || fail "ci-miss: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o['id'] == '77' and o['deduped'] is False, o +" || fail "ci-miss: a fingerprint miss must create (deduped:false), got: ${OUT}" +[[ -f "${S8}/post_body.json" ]] || fail "ci-miss: the create path must POST" +echo "OK: customer invoice dedupe — fingerprint hit returns id, no POST; desc change misses and creates" + +# --- Case 9: payment dedupe by (invoice, amount, normalized tx) --- +S9="$(mktemp -d -t prtest9.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}"' EXIT +cat > "${S9}/payments.json" </dev/null)" \ + || fail "pay-dedupe: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o == {'id': None, 'bank_transaction_id': 556, 'transaction_id': '${SHORT}', 'deduped': True}, o +" || fail "pay-dedupe: must dedupe on the normalized tx, got: ${OUT}" +[[ ! -f "${S9}/post_body.json" ]] || fail "pay-dedupe: must NOT POST a duplicate payment" +# 9b — same tx, different amount = conflict +rc=0 +printf '{"invoice_id":13,"kind":"supplier","account_id":1,"amount":97,"transaction_id":"%s"}' "${LONG}" \ + | DOL_WRITE="${STUB}" STUB_STATE="${S9}" bash "${PR}" >/dev/null 2>"${S9}/stderr9b" || rc=$? +[[ "${rc}" -ne 0 ]] || fail "pay-conflict: same tx + different amount must abort" +grep -q 'conflict' "${S9}/stderr9b" || fail "pay-conflict: error must say it is a conflict" +[[ ! -f "${S9}/post_body.json" ]] || fail "pay-conflict: must NOT POST on a conflict" +# 9c — a HISTORICAL long-form stored num still dedupes (erp#37 composition) +cat > "${S9}/payments.json" </dev/null)" \ + || fail "pay-dedupe-longnum: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o['deduped'] is True and o['transaction_id'] == '${SHORT}', o +" || fail "pay-dedupe-longnum: historical long num must normalize and dedupe, got: ${OUT}" +[[ ! -f "${S9}/post_body.json" ]] || fail "pay-dedupe-longnum: must NOT POST" +echo "OK: payment dedupe — normalized tx hit is a no-op, amount mismatch aborts, long-form history still dedupes" + +# --- Case 10: thirdparty dedupe by exact name --- +S10="$(mktemp -d -t tctest.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}" "${S10}"' EXIT +printf '%s' '[{"id":"7","name":"Darnis Operations","client":"0","fournisseur":"1"}]' > "${S10}/thirdparties.json" +OUT="$(printf '%s' '{"name":"Darnis Operations","role":"supplier"}' \ + | DOL_WRITE="${STUB}" STUB_STATE="${S10}" bash "${TC}" 2>/dev/null)" \ + || fail "tp-dedupe: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o == {'id': 7, 'deduped': True}, o +" || fail "tp-dedupe: must return existing id 7 with deduped:true, got: ${OUT}" +[[ ! -f "${S10}/tp_post_body.json" ]] || fail "tp-dedupe: must NOT POST when the name matches" +# 10b — two matches = ambiguous, abort +printf '%s' '[{"id":"7","name":"Darnis Operations","client":"0","fournisseur":"1"}, + {"id":"8","name":"Darnis Operations","client":"0","fournisseur":"1"}]' > "${S10}/thirdparties.json" +rc=0 +printf '%s' '{"name":"Darnis Operations","role":"supplier"}' \ + | DOL_WRITE="${STUB}" STUB_STATE="${S10}" bash "${TC}" >/dev/null 2>"${S10}/stderr10b" || rc=$? +[[ "${rc}" -ne 0 ]] || fail "tp-ambiguous: 2 name matches must abort" +grep -qi 'ambiguous' "${S10}/stderr10b" || fail "tp-ambiguous: error must say ambiguous" +[[ ! -f "${S10}/tp_post_body.json" ]] || fail "tp-ambiguous: must NOT POST" +# 10c — existing fiche missing the requested role = abort (refuse-never-repair) +printf '%s' '[{"id":"7","name":"Darnis Operations","client":"0","fournisseur":"1"}]' > "${S10}/thirdparties.json" +rc=0 +printf '%s' '{"name":"Darnis Operations","role":"client"}' \ + | DOL_WRITE="${STUB}" STUB_STATE="${S10}" bash "${TC}" >/dev/null 2>"${S10}/stderr10c" || rc=$? +[[ "${rc}" -ne 0 ]] || fail "tp-role: role mismatch must abort" +grep -q 'client' "${S10}/stderr10c" || fail "tp-role: error must name the missing role" +[[ ! -f "${S10}/tp_post_body.json" ]] || fail "tp-role: must NOT POST" +# 10d — no match (stub answers the Dolibarr 404) → the create path POSTs +rm -f "${S10}/thirdparties.json" +OUT="$(printf '%s' '{"name":"Fresh Supplier","role":"supplier"}' \ + | DOL_WRITE="${STUB}" STUB_STATE="${S10}" bash "${TC}" 2>/dev/null)" \ + || fail "tp-miss: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o == {'id': 90, 'deduped': False}, o +" || fail "tp-miss: a miss must create (deduped:false), got: ${OUT}" +[[ -f "${S10}/tp_post_body.json" ]] || fail "tp-miss: the create path must POST" +echo "OK: thirdparty dedupe — exact-name hit, ambiguous abort, role-mismatch abort, miss creates" + +# --- Case 11: a deduped DRAFT with validate:true is validated (run converges) --- +S11="$(mktemp -d -t ictest11.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}" "${S10}" "${S11}"' EXIT +cat > "${S11}/supplierinvoices.json" < "${S11}/invoice_detail_5.json" +OUT="$(printf '%s' '{"socid":7,"kind":"supplier","date":"2026-06-30","ref_supplier":"F1045","validate":true, + "lines":[{"desc":"Apport","qty":1,"price_ht":214.70,"tva":20,"type":"service"}]}' \ + | DOL_WRITE="${STUB}" STUB_STATE="${S11}" bash "${IC}" 2>/dev/null)" \ + || fail "draft-converge: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o['id'] == '5' and o['deduped'] is True and o['statut'] == '1', o +" || fail "draft-converge: must dedupe AND report the validated statut, got: ${OUT}" +[[ ! -f "${S11}/post_body.json" ]] || fail "draft-converge: must NOT create a second invoice" +grep -q '/supplierinvoices/5/validate' "${S11}/validated_endpoint" \ + || fail "draft-converge: the matched draft must be validated" +echo "OK: draft convergence — replay validates the half-done invoice instead of duplicating it" + echo "OK: all offline tests passed" diff --git a/.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh b/.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh index 4550f4d..eaa3b48 100755 --- a/.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh +++ b/.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh @@ -3,21 +3,58 @@ # the DOL_WRITE env hook). Never talks to any host — sandbox and prod are both out # of reach by construction. Dispatch by endpoint: # +# GET …/payments → serves $STUB_STATE/payments.json when present; else +# (post-POST correlate) a list built from post_body.json; +# else [] — the live sandbox answers [] (200) when empty +# GET /thirdparties?… → name-lookup: serves thirdparties.json, else the +# Dolibarr empty behavior (HTTP 404 + non-zero) # GET /thirdparties/ → serves $STUB_STATE/thirdparty.json (or a canned # "before" fiche on first read) # PUT /thirdparties/ → records put_body.json/put_endpoint, merges the body # into thirdparty.json (so the read-after sees it) +# POST /thirdparties → records tp_post_body.json, echoes 90 +# GET /supplierinvoices?… → dedupe listing: serves supplierinvoices.json else [] +# GET /invoices?… → dedupe listing: serves invoices.json else [] +# GET /supplierinvoices/ · /invoices/ +# → serves invoice_detail_.json, else HTTP 404 # GET /contacts… → serves $STUB_STATE/contacts.json; mimics Dolibarr's # empty-list behavior (HTTP 404 + non-zero) when absent # POST /contacts… → records contact_post_body.json, appends the contact # (id 88) to contacts.json, echoes 88 -# POST → payment behavior: records post_body.json/post_endpoint, -# echoes 77 (unchanged from the erp#37 tests) +# POST …/validate → records validated_endpoint (proof validation ran) +# POST → payment/invoice behavior: records post_body.json/ +# post_endpoint, echoes 77 (unchanged from erp#37) # GET → payments list served back from post_body.json set -euo pipefail STATE="${STUB_STATE:?stub-dol-write.sh: STUB_STATE not set}" METHOD="$1"; ENDPOINT="$2"; BODY="${3:-}" case "${METHOD} ${ENDPOINT}" in + GET\ *"/payments"*) + if [[ -f "${STATE}/payments.json" ]]; then + cat "${STATE}/payments.json" + elif [[ -f "${STATE}/post_body.json" ]]; then + python3 - "${STATE}/post_body.json" <<'PY' +import json, sys +body = json.load(open(sys.argv[1])) +print(json.dumps([{"num": body.get("num_payment", ""), + "amount": body.get("amount", ""), + "date": "2026-06-20 12:00:00", + "fk_bank_line": "556"}])) +PY + else + printf '[]' # live sandbox: empty payment list is [] with HTTP 200 + fi + ;; + "GET /thirdparties?"*) + if [[ -f "${STATE}/thirdparties.json" ]]; then + cat "${STATE}/thirdparties.json" + else + # Dolibarr answers 404 (not []) when a thirdparty lookup matches nothing. + printf '%s' '{"error":{"code":404,"message":"Not Found: No third parties found"}}' + echo "stub-dol-write.sh: HTTP 404 on GET ${ENDPOINT}" >&2 + exit 1 + fi + ;; "GET /thirdparties/"*) if [[ -f "${STATE}/thirdparty.json" ]]; then cat "${STATE}/thirdparty.json" @@ -40,6 +77,26 @@ json.dump(cur, open(p, "w"), ensure_ascii=False) print(json.dumps(cur, ensure_ascii=False)) PY ;; + "POST /thirdparties") + printf '%s' "${BODY}" > "${STATE}/tp_post_body.json" + echo "90" + ;; + "GET /supplierinvoices?"*) + if [[ -f "${STATE}/supplierinvoices.json" ]]; then cat "${STATE}/supplierinvoices.json"; else printf '[]'; fi + ;; + "GET /invoices?"*) + if [[ -f "${STATE}/invoices.json" ]]; then cat "${STATE}/invoices.json"; else printf '[]'; fi + ;; + "GET /supplierinvoices/"*|"GET /invoices/"*) + ID="${ENDPOINT##*/}" + if [[ -f "${STATE}/invoice_detail_${ID}.json" ]]; then + cat "${STATE}/invoice_detail_${ID}.json" + else + printf '%s' '{"error":{"code":404,"message":"Not Found"}}' + echo "stub-dol-write.sh: HTTP 404 on GET ${ENDPOINT}" >&2 + exit 1 + fi + ;; "GET /contacts"*) if [[ -f "${STATE}/contacts.json" ]]; then cat "${STATE}/contacts.json" @@ -63,6 +120,10 @@ json.dump(rows, open(p, "w"), ensure_ascii=False) PY echo "88" ;; + POST\ *"/validate") + printf '%s\n' "${ENDPOINT}" > "${STATE}/validated_endpoint" + echo '{"success":1}' + ;; POST\ *) printf '%s' "${BODY}" > "${STATE}/post_body.json" printf '%s\n' "${ENDPOINT}" > "${STATE}/post_endpoint"