Files
erp/.claude/skills/dolibarr-sandbox-write/scripts/invoice-create.sh
T
arcodangeandClaude Fable 5 3840e74dcd feat(write-skill): idempotency keys — manifest replay is a no-op (erp#44)
Learning #4 of the 2026-07-11 rehearsal: manifest B failed mid-run and could
not be re-applied — op 1 (the DARNIS invoice) had already run and a replay
would have duplicated it. Every write op now dedupes BEFORE any POST:

- thirdparty-create.sh: by exact name (promote '#thirdparty:name=' semantics);
  ambiguous (2+) aborts; an existing fiche missing the requested role aborts
  (refuse-never-repair). Emits {"id", "deduped"} instead of a bare id.
- invoice-create.sh: supplier kind by (socid, ref_supplier) — same key with a
  different total aborts as a conflict; customer kind (or supplier without
  ref_supplier) by (socid, date, total_ttc ±0.02, line fingerprint) with descs
  HTML-unescaped. Credit notes are never candidates. A deduped DRAFT with
  validate:true is validated on replay, so an interrupted run converges.
- payment-record.sh: by (invoice, amount, normalized transaction_id), composing
  with the erp#37 varchar(50) normalization on BOTH sides so historical
  long-form nums still match; same tx + different amount aborts; without a tx
  id there is no dedupe key (warned). Dedupe answers id:null (the payments list
  exposes no paiement rowid) + the existing bank line.
- All three refuse to POST blind when the dedupe lookup fails with anything but
  the documented empty-list 404 (the voir_tous trap would otherwise mint dupes).
- promote-apply.sh: marks each op created / deduped=true inline and totals them
  in the summary — an all-deduped second run is visible proof of a no-op.
- promote-plan.sh: advertises each op's dedupe key (and flags tx=MISSING as
  'a replay WILL double-pay').

Proof:
- tests/run-tests.sh: 5 new offline cases (11 total) — dedupe hits POST
  nothing, conflicts/ambiguity abort pre-POST, long-form history dedupes,
  draft convergence validates; stub extended to serve the new lookups with the
  live-observed empty behaviors ([] for invoices/payments, 404 for tiers).
- tests/replay-idempotency.sh (new, live): double-applies a self-contained
  manifest on the sandbox — run 1 '3 created' (rows 1/1/1), run 2 '3 deduped'
  with row counts unchanged and the stored num in erp#37 short form.
- The historic manifest-B now replays on the sandbox as 5/5 deduped, zero new
  rows — the exact replay Learning #4 declared impossible.

SKILL.md updated in the same change (per-op dedupe keys, replay-safety section,
gotchas); the 2026-07-11 runbook's Learning #4 carries a dated resolution
addendum.

Closes erp#44.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
2026-07-18 23:46:05 +02:00

200 lines
9.0 KiB
Bash
Executable File

#!/usr/bin/env bash
# Create a customer or supplier invoice (facture) with product/service lines in
# the SANDBOX, optionally validating it — IDEMPOTENT (erp#44): replaying the
# same create returns the existing invoice instead of minting a duplicate.
#
# Input: a JSON object on stdin (or a file path in $1):
# socid (required) thirdparty id
# kind "customer" | "supplier" (default "customer")
# date "YYYY-MM-DD" (default today)
# ref_supplier supplier's own invoice ref (supplier invoices)
# validate true|false (default false = leave draft)
# lines: [ { desc, qty, price_ht, tva, type: "product"|"service", product_id? } ]
#
# Idempotency (erp#44 — Learning #4 of the 2026-07-11 rehearsal, where a manifest
# that failed mid-run could not be replayed because op 1 would have re-created
# the DARNIS invoice). BEFORE any POST, list the thirdparty's invoices of the
# same kind and dedupe:
# - supplier kind WITH ref_supplier : by (socid, ref_supplier) — the supplier's
# own ref is the natural key. A match whose total_ttc DIFFERS from the
# requested lines ABORTS (same key, different content = a data conflict to
# resolve by a human, not a dedupe).
# - customer kind (or supplier without ref_supplier) : by (socid, date,
# total_ttc ±0.02, line fingerprint) where the fingerprint is the multiset of
# (desc, qty, subprice, tva_tx) per line — desc HTML-unescaped and
# whitespace-collapsed, since Dolibarr returns stored descriptions
# HTML-encoded.
# - 2+ matches ABORT (ambiguous — the target already has duplicates; never
# guess). Credit notes (type=2) are never dedupe candidates.
# On a match: if `validate:true` was requested and the match is still a DRAFT
# (statut=0 — e.g. run 1 died between create and validate), it is validated now,
# so a replayed manifest CONVERGES instead of leaving a half-done op behind.
# A listing failure other than 404 ABORTS — POSTing blind would mint duplicates.
#
# Emits {id, ref, ref_supplier, total_ht, total_ttc, statut, deduped} on stdout
# — deduped:true means no invoice was created (the id is the pre-existing one).
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
SRC="${1:-}"
if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi
TMPD="$(mktemp -d -t invcre.XXXXXX)"; trap 'rm -rf "${TMPD}"' EXIT
cat > "${TMPD}/map.py" <<'PY'
import json, sys, datetime
d = json.loads(sys.stdin.read())
if not d.get("socid"):
sys.exit("invoice-create.sh: 'socid' is required")
supplier = d.get("kind", "customer").lower() in ("supplier", "fournisseur")
endpoint = "/supplierinvoices" if supplier else "/invoices"
ds = d.get("date")
dt = datetime.datetime.strptime(ds, "%Y-%m-%d") if ds else datetime.datetime.now()
epoch = int(dt.timestamp())
lines = []
for ln in d.get("lines", []):
is_product = ln.get("type", "service").lower() in ("product", "produit")
L = {
"desc": ln.get("desc", ""),
"subprice": str(ln.get("price_ht", ln.get("subprice", 0))),
"qty": str(ln.get("qty", 1)),
"tva_tx": str(ln.get("tva", ln.get("tva_tx", 20))),
"product_type": "0" if is_product else "1",
}
if ln.get("product_id"):
L["fk_product"] = str(ln["product_id"])
lines.append(L)
body = {"socid": d["socid"], "date": epoch, "type": 0, "lines": lines}
if supplier and d.get("ref_supplier"):
body["ref_supplier"] = d["ref_supplier"]
# Expected total_ttc, Dolibarr-style (per-line rounding), for the dedupe key.
total_ttc = round(sum(round(float(L["qty"]) * float(L["subprice"]), 2)
* (1 + float(L["tva_tx"]) / 100.0) for L in lines), 2)
dedupe = {"supplier": supplier, "socid": str(d["socid"]),
"ref_supplier": str(d.get("ref_supplier") or "") if supplier else "",
"date": dt.strftime("%Y-%m-%d"), "total_ttc": total_ttc,
"lines": [[L["desc"], L["qty"], L["subprice"], L["tva_tx"]] for L in lines]}
print(endpoint)
print(json.dumps(body))
print("1" if d.get("validate") else "0")
print(json.dumps(dedupe))
PY
MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")"
ENDPOINT="$(sed -n 1p <<<"${MAPPED}")"
BODY="$(sed -n 2p <<<"${MAPPED}")"
VALIDATE="$(sed -n 3p <<<"${MAPPED}")"
DEDUPE="$(sed -n 4p <<<"${MAPPED}")"
printf '%s' "${DEDUPE}" > "${TMPD}/dedupe.json"
# Final read-back: shared by the create and dedupe paths.
emit_summary() { # $1 = invoice id, $2 = deduped true|false
"${W}" GET "${ENDPOINT}/$1" | DEDUPED="$2" python3 -c "import json,sys,os
d=json.load(sys.stdin)
o={k:d.get(k) for k in ('id','ref','ref_supplier','total_ht','total_ttc','statut')}
o['deduped']=os.environ['DEDUPED']=='true'
print(json.dumps(o))"
}
# --- Dedupe lookup (erp#44): list this thirdparty's invoices of the same kind ---
SOCID="$(python3 -c "import json,sys; print(json.load(sys.stdin)['socid'])" < "${TMPD}/dedupe.json")"
set +e
"${W}" GET "${ENDPOINT}?thirdparty_ids=${SOCID}&limit=500" \
> "${TMPD}/list.json" 2> "${TMPD}/list.err"
rc=$?
set -e
if [[ ${rc} -ne 0 ]]; then
if grep -q "HTTP 404" "${TMPD}/list.err"; then
printf '[]' > "${TMPD}/list.json" # empty lists can answer 404 (dolibarr skill gotcha)
else
cat "${TMPD}/list.err" >&2
echo "invoice-create.sh: could not list ${ENDPOINT} for socid ${SOCID} — refusing to POST blind (dedupe impossible)" >&2
exit 1
fi
fi
cat > "${TMPD}/match.py" <<'PY'
import json, sys, html, datetime
ded = json.load(open(sys.argv[1]))
try:
rows = json.load(open(sys.argv[2]))
except Exception:
rows = []
rows = [r for r in (rows if isinstance(rows, list) else [])
if str(r.get("type", "0")) != "2"] # credit notes never dedupe a create
def norm_desc(s):
return " ".join(html.unescape(str(s or "")).split())
def fnum(v, nd):
try: return round(float(v), nd)
except Exception: return None
def fingerprint(lines):
return sorted((norm_desc(l[0]), fnum(l[1], 3), fnum(l[2], 2), fnum(l[3], 3))
for l in lines)
TOL = 0.02
want_fp = fingerprint(ded["lines"])
hits = []
if ded["supplier"] and ded["ref_supplier"]:
key = ded["ref_supplier"].strip()
hits = [r for r in rows if str(r.get("ref_supplier") or "").strip() == key]
if len(hits) == 1 and ded["lines"]:
got = fnum(hits[0].get("total_ttc"), 2)
if got is not None and abs(got - ded["total_ttc"]) > TOL:
sys.exit("invoice-create.sh: ABORT — supplier invoice with ref_supplier "
"%r already exists on socid %s (id %s) but its total_ttc %.2f "
"differs from the requested %.2f. Same key, different content "
"is a conflict to resolve, not a dedupe." %
(key, ded["socid"], hits[0].get("id"), got, ded["total_ttc"]))
else:
for r in rows:
try:
rdate = datetime.datetime.fromtimestamp(int(r.get("date"))).strftime("%Y-%m-%d")
except Exception:
continue
if rdate != ded["date"]:
continue
got = fnum(r.get("total_ttc"), 2)
if got is None or abs(got - ded["total_ttc"]) > TOL:
continue
rl = r.get("lines")
if not isinstance(rl, list):
sys.exit("invoice-create.sh: ABORT — candidate invoice id %s matches "
"(date, total) but carries no inline lines to fingerprint; "
"refusing to guess" % r.get("id"))
if fingerprint([[l.get("desc"), l.get("qty"), l.get("subprice"), l.get("tva_tx")]
for l in rl]) == want_fp:
hits.append(r)
if len(hits) > 1:
sys.exit("invoice-create.sh: ABORT — dedupe key matches %d invoices on socid %s "
"(ids %s): the target already holds duplicates; refusing to guess."
% (len(hits), ded["socid"], ", ".join(str(h.get("id")) for h in hits)))
if hits:
print(json.dumps({"id": int(hits[0]["id"]), "statut": str(hits[0].get("statut"))}))
PY
MATCH="$(python3 "${TMPD}/match.py" "${TMPD}/dedupe.json" "${TMPD}/list.json")"
if [[ -n "${MATCH}" ]]; then
ID="$(python3 -c "import json,sys; print(json.load(sys.stdin)['id'])" <<<"${MATCH}")"
STATUT="$(python3 -c "import json,sys; print(json.load(sys.stdin)['statut'])" <<<"${MATCH}")"
echo "invoice-create.sh: invoice already exists on socid ${SOCID} (id ${ID}) — deduped, no POST" >&2
# Converge an interrupted run: validate the matched draft if validation was asked.
if [[ "${VALIDATE}" == "1" && "${STATUT}" == "0" ]]; then
echo "invoice-create.sh: matched invoice ${ID} is still a draft — validating it now (completes the interrupted op)" >&2
"${W}" POST "${ENDPOINT}/${ID}/validate" '{}' >/dev/null
fi
emit_summary "${ID}" true
exit 0
fi
# --- Create (no match) ----------------------------------------------------------
ID="$("${W}" POST "${ENDPOINT}" "${BODY}")"
if [[ ! "${ID}" =~ ^[0-9]+$ ]]; then
echo "invoice-create.sh: create did not return an id: ${ID}" >&2
exit 1
fi
if [[ "${VALIDATE}" == "1" ]]; then
"${W}" POST "${ENDPOINT}/${ID}/validate" '{}' >/dev/null
fi
emit_summary "${ID}" false