Files
arcodangeandClaude Fable 5 3840e74dcd feat(write-skill): idempotency keys — manifest replay is a no-op (erp#44)
Learning #4 of the 2026-07-11 rehearsal: manifest B failed mid-run and could
not be re-applied — op 1 (the DARNIS invoice) had already run and a replay
would have duplicated it. Every write op now dedupes BEFORE any POST:

- thirdparty-create.sh: by exact name (promote '#thirdparty:name=' semantics);
  ambiguous (2+) aborts; an existing fiche missing the requested role aborts
  (refuse-never-repair). Emits {"id", "deduped"} instead of a bare id.
- invoice-create.sh: supplier kind by (socid, ref_supplier) — same key with a
  different total aborts as a conflict; customer kind (or supplier without
  ref_supplier) by (socid, date, total_ttc ±0.02, line fingerprint) with descs
  HTML-unescaped. Credit notes are never candidates. A deduped DRAFT with
  validate:true is validated on replay, so an interrupted run converges.
- payment-record.sh: by (invoice, amount, normalized transaction_id), composing
  with the erp#37 varchar(50) normalization on BOTH sides so historical
  long-form nums still match; same tx + different amount aborts; without a tx
  id there is no dedupe key (warned). Dedupe answers id:null (the payments list
  exposes no paiement rowid) + the existing bank line.
- All three refuse to POST blind when the dedupe lookup fails with anything but
  the documented empty-list 404 (the voir_tous trap would otherwise mint dupes).
- promote-apply.sh: marks each op created / deduped=true inline and totals them
  in the summary — an all-deduped second run is visible proof of a no-op.
- promote-plan.sh: advertises each op's dedupe key (and flags tx=MISSING as
  'a replay WILL double-pay').

Proof:
- tests/run-tests.sh: 5 new offline cases (11 total) — dedupe hits POST
  nothing, conflicts/ambiguity abort pre-POST, long-form history dedupes,
  draft convergence validates; stub extended to serve the new lookups with the
  live-observed empty behaviors ([] for invoices/payments, 404 for tiers).
- tests/replay-idempotency.sh (new, live): double-applies a self-contained
  manifest on the sandbox — run 1 '3 created' (rows 1/1/1), run 2 '3 deduped'
  with row counts unchanged and the stored num in erp#37 short form.
- The historic manifest-B now replays on the sandbox as 5/5 deduped, zero new
  rows — the exact replay Learning #4 declared impossible.

SKILL.md updated in the same change (per-op dedupe keys, replay-safety section,
gotchas); the 2026-07-11 runbook's Learning #4 carries a dated resolution
addendum.

Closes erp#44.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
2026-07-18 23:46:05 +02:00

211 lines
10 KiB
Bash
Executable File

#!/usr/bin/env bash
# Record a payment (règlement) on a validated invoice in the SANDBOX —
# IDEMPOTENT (erp#44): replaying the same règlement is a no-op, not a double
# payment.
#
# Input: a JSON object on stdin (or a file path in $1):
# invoice_id (required) the invoice to pay
# kind "customer" | "supplier" (default "customer")
# mode "VIR" | "CB" | "CHQ" | "LIQ" (default "VIR")
# account_id (required) the bank account id receiving/paying
# date "YYYY-MM-DD" (default today)
# amount (REQUIRED for supplier; customer pays the full remaining)
# transaction_id (recommended) the originating bank transaction id (the Qonto/Wise
# tx id from the feed). Stored on the payment's bank line
# (llx_bank.num_chq) so the règlement reconciles to the feed by id.
# `num` is a back-compat alias for the same field.
# NORMALIZED before POST: Dolibarr stores num_payment in varchar(50)
# (llx_paiement.num_paiement / llx_paiementfourn.num_paiement) while
# Qonto ids run ~67 chars (<org>-<n>-<n>-transaction-<uuid>), so
# everything through "transaction-" is stripped and the UUID suffix
# (globally unique, ~37 chars) is the canonical stored form. Wise ids
# (short numerics) pass through unchanged. An id still >50 chars after
# normalization is REFUSED with an error — never truncated silently.
# comment (optional)
#
# Idempotency (erp#44, composing with the erp#37 normalization): BEFORE any POST,
# the invoice's payment list is fetched and deduped by
# (invoice, amount, normalized transaction_id):
# - a row whose num — normalized the same way (historical rows may still carry
# long-form Qonto ids) — equals the normalized transaction_id is a REPLAY:
# for supplier payments the amounts must also agree (±0.005; same tx id with
# a DIFFERENT amount ABORTS as a data conflict); customer payments settle the
# full remaining so the tx id alone is the key.
# - a dedupe hit emits {"id": null, "bank_transaction_id": <existing line>,
# "transaction_id": <normalized>, "deduped": true} and exits 0 WITHOUT
# posting (the payments list does not expose the paiement rowid — id is null
# by honesty, the bank line is the stable handle reconciliation keys on).
# - the SAME normalized tx appearing on 2+ rows of this invoice ABORTS (the
# target already holds duplicates; never guess).
# - WITHOUT a transaction_id there is NO dedupe key — the payment posts with a
# warning (as before) and a replay WILL duplicate it. Always pass the tx id.
# - a listing failure other than 404 ABORTS: paying blind would double-pay.
#
# The invoice must be VALIDATED first (invoice-create.sh ... "validate":true).
# Emits {id, bank_transaction_id, transaction_id, deduped} on stdout —
# `transaction_id` is the NORMALIZED num actually stored (what bank-match keys
# on), `bank_transaction_id` the Dolibarr bank line (llx_bank.fk_bank_line).
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
SRC="${1:-}"
if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi
TMPD="$(mktemp -d -t payrec.XXXXXX)"
trap 'rm -rf "${TMPD}"' EXIT
cat > "${TMPD}/map.py" <<'PY'
import json, sys, datetime, re
d = json.loads(sys.stdin.read())
if not d.get("invoice_id"):
sys.exit("payment-record.sh: 'invoice_id' is required")
if not d.get("account_id"):
sys.exit("payment-record.sh: 'account_id' is required")
# Stable Dolibarr c_paiement ids (sandbox seeded from prod / standard defaults).
MODE = {"VIR": 2, "CB": 6, "CHQ": 7, "LIQ": 4}
mode = MODE.get(str(d.get("mode", "VIR")).upper())
if mode is None:
sys.exit("payment-record.sh: unknown mode (use VIR|CB|CHQ|LIQ)")
supplier = d.get("kind", "customer").lower() in ("supplier", "fournisseur")
ds = d.get("date")
epoch = int((datetime.datetime.strptime(ds, "%Y-%m-%d") if ds
else datetime.datetime.now()).timestamp())
inv = d["invoice_id"]
# transaction_id is the first-class bank-feed tx id; num is the back-compat alias.
raw_tx = str(d.get("transaction_id") or d.get("num") or "")
# Normalize to the canonical short form (erp#37). Dolibarr stores num_payment in
# varchar(50) (llx_paiement.num_paiement / llx_paiementfourn.num_paiement) and
# a Qonto id (~67 chars, <org>-<n>-<n>-transaction-<uuid>) blows past it —
# HTTP 400 "value too long for type character varying(50)". Strip everything
# through "transaction-" and store the UUID suffix (globally unique). Wise ids
# (short numerics, no "transaction-") pass through unchanged. bank-match.sh
# normalizes feed ids the same way, so the short form still reconciles by id.
tx = re.sub(r'^.*transaction-', '', raw_tx)
if tx != raw_tx:
sys.stderr.write("payment-record.sh: transaction_id normalized to %r "
"(Qonto prefix stripped — num_payment is varchar(50))\n" % tx)
if len(tx) > 50:
sys.exit("payment-record.sh: transaction_id %r is %d chars even after "
"normalization — Dolibarr's num_payment is varchar(50) and silent "
"truncation would break bank reconciliation; pass a shorter id"
% (tx, len(tx)))
if not tx:
sys.stderr.write("payment-record.sh: WARNING — no transaction_id given; this "
"règlement won't auto-reconcile to the bank feed AND cannot "
"be deduped on a replay (erp#44) — a re-run will double-pay\n")
if supplier:
if d.get("amount") is None:
sys.exit("payment-record.sh: supplier payments require an 'amount'")
endpoint = "/supplierinvoices/%s/payments" % inv
body = {"datepaye": epoch, "payment_mode_id": mode, "closepaidinvoices": "yes",
"accountid": d["account_id"],
"amount": str(d["amount"]), "num_payment": tx,
"comment": d.get("comment", "")}
else:
endpoint = "/invoices/%s/payments" % inv
body = {"datepaye": epoch, "paymentid": mode, "closepaidinvoices": "yes",
"accountid": d["account_id"], "num_payment": tx,
"comment": d.get("comment", "")}
print(endpoint)
print(json.dumps(body))
print(tx)
print(json.dumps({"supplier": supplier,
"amount": (float(d["amount"]) if supplier else None)}))
PY
# Dedupe a replay against the invoice's existing payments (erp#44).
cat > "${TMPD}/dedupe.py" <<'PY'
import json, sys, os, re
ded = json.loads(os.environ["DEDUPE"])
tx = os.environ.get("TX", "")
try:
rows = json.load(sys.stdin)
except Exception:
rows = []
rows = rows if isinstance(rows, list) else []
norm = lambda s: re.sub(r'^.*transaction-', '', str(s or "")) # erp#37, both sides
hits = [r for r in rows if tx and norm(r.get("num")) == tx]
if not hits:
sys.exit(0) # no match -> proceed to POST
if len(hits) > 1:
sys.exit("payment-record.sh: ABORT — transaction_id %r already appears on %d "
"payments of this invoice; the target holds duplicates, refusing to "
"guess" % (tx, len(hits)))
r = hits[0]
if ded["supplier"] and ded["amount"] is not None:
try:
got = float(r.get("amount"))
except (TypeError, ValueError):
got = None
if got is None or abs(got - ded["amount"]) > 0.005:
sys.exit("payment-record.sh: ABORT — transaction_id %r is already recorded "
"on this invoice with amount %s, but %s was requested. Same tx, "
"different amount is a conflict to resolve, not a dedupe."
% (tx, r.get("amount"), ded["amount"]))
btx = r.get("fk_bank_line")
print(json.dumps({"id": None,
"bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx,
"transaction_id": tx, "deduped": True}))
PY
# Correlate the created payment back to its bank transaction line. The payments
# list carries fk_bank_line but not the paiement rowid, so match on the provided
# transaction_id (the external bank ref), else fall back to the most recent line.
cat > "${TMPD}/correlate.py" <<'PY'
import json, sys, os
rows = json.load(sys.stdin); rows = rows if isinstance(rows, list) else []
tx = os.environ.get("TX", ""); pid = int(os.environ["PAYID"])
pick = None
if tx:
cand = [r for r in rows if str(r.get("num", "")) == tx]
if cand:
pick = cand[-1]
if pick is None and rows:
pick = max(rows, key=lambda r: r.get("date", ""))
btx = (pick or {}).get("fk_bank_line")
# transaction_id: the num as stored (normalized) — fall back to the sent TX so
# the JSON always reports the canonical short form even on the recency fallback.
print(json.dumps({"id": pid,
"bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx,
"transaction_id": (pick or {}).get("num", "") or tx,
"deduped": False}))
PY
MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")"
ENDPOINT="$(sed -n 1p <<<"${MAPPED}")"
BODY="$(sed -n 2p <<<"${MAPPED}")"
TX="$(sed -n 3p <<<"${MAPPED}")"
DEDUPE="$(sed -n 4p <<<"${MAPPED}")"
# --- Pre-POST dedupe (erp#44) — only possible when a transaction_id was given ---
if [[ -n "${TX}" ]]; then
set +e
"${W}" GET "${ENDPOINT}" > "${TMPD}/payments.json" 2> "${TMPD}/list.err"
rc=$?
set -e
if [[ ${rc} -ne 0 ]]; then
if grep -q "HTTP 404" "${TMPD}/list.err"; then
printf '[]' > "${TMPD}/payments.json" # empty lists can answer 404 (dolibarr skill gotcha)
else
cat "${TMPD}/list.err" >&2
echo "payment-record.sh: could not list ${ENDPOINT} — refusing to pay blind (dedupe impossible)" >&2
exit 1
fi
fi
MATCH="$(DEDUPE="${DEDUPE}" TX="${TX}" python3 "${TMPD}/dedupe.py" < "${TMPD}/payments.json")"
if [[ -n "${MATCH}" ]]; then
echo "payment-record.sh: transaction_id ${TX} already recorded on this invoice — deduped, no POST" >&2
printf '%s\n' "${MATCH}"
exit 0
fi
fi
PAYID="$("${W}" POST "${ENDPOINT}" "${BODY}")"
if [[ ! "${PAYID}" =~ ^[0-9]+$ ]]; then
echo "payment-record.sh: payment POST did not return an id: ${PAYID}" >&2
exit 1
fi
# Same path serves the GET list; resolve fk_bank_line and emit the enriched record.
"${W}" GET "${ENDPOINT}" | PAYID="${PAYID}" TX="${TX}" python3 "${TMPD}/correlate.py"