feat(write-skill): idempotency keys — manifest replay is a no-op (erp#44)
Learning #4 of the 2026-07-11 rehearsal: manifest B failed mid-run and could not be re-applied — op 1 (the DARNIS invoice) had already run and a replay would have duplicated it. Every write op now dedupes BEFORE any POST: - thirdparty-create.sh: by exact name (promote '#thirdparty:name=' semantics); ambiguous (2+) aborts; an existing fiche missing the requested role aborts (refuse-never-repair). Emits {"id", "deduped"} instead of a bare id. - invoice-create.sh: supplier kind by (socid, ref_supplier) — same key with a different total aborts as a conflict; customer kind (or supplier without ref_supplier) by (socid, date, total_ttc ±0.02, line fingerprint) with descs HTML-unescaped. Credit notes are never candidates. A deduped DRAFT with validate:true is validated on replay, so an interrupted run converges. - payment-record.sh: by (invoice, amount, normalized transaction_id), composing with the erp#37 varchar(50) normalization on BOTH sides so historical long-form nums still match; same tx + different amount aborts; without a tx id there is no dedupe key (warned). Dedupe answers id:null (the payments list exposes no paiement rowid) + the existing bank line. - All three refuse to POST blind when the dedupe lookup fails with anything but the documented empty-list 404 (the voir_tous trap would otherwise mint dupes). - promote-apply.sh: marks each op created / deduped=true inline and totals them in the summary — an all-deduped second run is visible proof of a no-op. - promote-plan.sh: advertises each op's dedupe key (and flags tx=MISSING as 'a replay WILL double-pay'). Proof: - tests/run-tests.sh: 5 new offline cases (11 total) — dedupe hits POST nothing, conflicts/ambiguity abort pre-POST, long-form history dedupes, draft convergence validates; stub extended to serve the new lookups with the live-observed empty behaviors ([] for invoices/payments, 404 for tiers). - tests/replay-idempotency.sh (new, live): double-applies a self-contained manifest on the sandbox — run 1 '3 created' (rows 1/1/1), run 2 '3 deduped' with row counts unchanged and the stored num in erp#37 short form. - The historic manifest-B now replays on the sandbox as 5/5 deduped, zero new rows — the exact replay Learning #4 declared impossible. SKILL.md updated in the same change (per-op dedupe keys, replay-safety section, gotchas); the 2026-07-11 runbook's Learning #4 carries a dated resolution addendum. Closes erp#44. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
Claude Fable 5
@@ -1,5 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Record a payment (règlement) on a validated invoice in the SANDBOX.
|
||||
# Record a payment (règlement) on a validated invoice in the SANDBOX —
|
||||
# IDEMPOTENT (erp#44): replaying the same règlement is a no-op, not a double
|
||||
# payment.
|
||||
#
|
||||
# Input: a JSON object on stdin (or a file path in $1):
|
||||
# invoice_id (required) the invoice to pay
|
||||
@@ -21,12 +23,28 @@
|
||||
# normalization is REFUSED with an error — never truncated silently.
|
||||
# comment (optional)
|
||||
#
|
||||
# Idempotency (erp#44, composing with the erp#37 normalization): BEFORE any POST,
|
||||
# the invoice's payment list is fetched and deduped by
|
||||
# (invoice, amount, normalized transaction_id):
|
||||
# - a row whose num — normalized the same way (historical rows may still carry
|
||||
# long-form Qonto ids) — equals the normalized transaction_id is a REPLAY:
|
||||
# for supplier payments the amounts must also agree (±0.005; same tx id with
|
||||
# a DIFFERENT amount ABORTS as a data conflict); customer payments settle the
|
||||
# full remaining so the tx id alone is the key.
|
||||
# - a dedupe hit emits {"id": null, "bank_transaction_id": <existing line>,
|
||||
# "transaction_id": <normalized>, "deduped": true} and exits 0 WITHOUT
|
||||
# posting (the payments list does not expose the paiement rowid — id is null
|
||||
# by honesty, the bank line is the stable handle reconciliation keys on).
|
||||
# - the SAME normalized tx appearing on 2+ rows of this invoice ABORTS (the
|
||||
# target already holds duplicates; never guess).
|
||||
# - WITHOUT a transaction_id there is NO dedupe key — the payment posts with a
|
||||
# warning (as before) and a replay WILL duplicate it. Always pass the tx id.
|
||||
# - a listing failure other than 404 ABORTS: paying blind would double-pay.
|
||||
#
|
||||
# The invoice must be VALIDATED first (invoice-create.sh ... "validate":true).
|
||||
# Emits {id, bank_transaction_id, transaction_id} on stdout — `transaction_id` is
|
||||
# the NORMALIZED num actually stored (what bank-match keys on). `bank_transaction_id`
|
||||
# is the Dolibarr bank line (llx_bank.fk_bank_line) the payment created — the id
|
||||
# bank reconciliation (arcodange-bank-reco) keys on to link this règlement to a
|
||||
# statement line. Recording without a transaction_id warns (it won't auto-reconcile).
|
||||
# Emits {id, bank_transaction_id, transaction_id, deduped} on stdout —
|
||||
# `transaction_id` is the NORMALIZED num actually stored (what bank-match keys
|
||||
# on), `bank_transaction_id` the Dolibarr bank line (llx_bank.fk_bank_line).
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
|
||||
@@ -34,9 +52,9 @@ W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
|
||||
SRC="${1:-}"
|
||||
if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi
|
||||
|
||||
PYF="$(mktemp -t dolpy.XXXXXX)"; PYF2="$(mktemp -t dolpy2.XXXXXX)"
|
||||
trap 'rm -f "${PYF}" "${PYF2}"' EXIT
|
||||
cat > "${PYF}" <<'PY'
|
||||
TMPD="$(mktemp -d -t payrec.XXXXXX)"
|
||||
trap 'rm -rf "${TMPD}"' EXIT
|
||||
cat > "${TMPD}/map.py" <<'PY'
|
||||
import json, sys, datetime, re
|
||||
d = json.loads(sys.stdin.read())
|
||||
if not d.get("invoice_id"):
|
||||
@@ -55,7 +73,7 @@ epoch = int((datetime.datetime.strptime(ds, "%Y-%m-%d") if ds
|
||||
inv = d["invoice_id"]
|
||||
# transaction_id is the first-class bank-feed tx id; num is the back-compat alias.
|
||||
raw_tx = str(d.get("transaction_id") or d.get("num") or "")
|
||||
# Normalize to the canonical short form. Dolibarr stores num_payment in
|
||||
# Normalize to the canonical short form (erp#37). Dolibarr stores num_payment in
|
||||
# varchar(50) (llx_paiement.num_paiement / llx_paiementfourn.num_paiement) and
|
||||
# a Qonto id (~67 chars, <org>-<n>-<n>-transaction-<uuid>) blows past it —
|
||||
# HTTP 400 "value too long for type character varying(50)". Strip everything
|
||||
@@ -73,7 +91,8 @@ if len(tx) > 50:
|
||||
% (tx, len(tx)))
|
||||
if not tx:
|
||||
sys.stderr.write("payment-record.sh: WARNING — no transaction_id given; this "
|
||||
"règlement won't auto-reconcile to the bank feed\n")
|
||||
"règlement won't auto-reconcile to the bank feed AND cannot "
|
||||
"be deduped on a replay (erp#44) — a re-run will double-pay\n")
|
||||
if supplier:
|
||||
if d.get("amount") is None:
|
||||
sys.exit("payment-record.sh: supplier payments require an 'amount'")
|
||||
@@ -90,12 +109,49 @@ else:
|
||||
print(endpoint)
|
||||
print(json.dumps(body))
|
||||
print(tx)
|
||||
print(json.dumps({"supplier": supplier,
|
||||
"amount": (float(d["amount"]) if supplier else None)}))
|
||||
PY
|
||||
|
||||
# Dedupe a replay against the invoice's existing payments (erp#44).
|
||||
cat > "${TMPD}/dedupe.py" <<'PY'
|
||||
import json, sys, os, re
|
||||
ded = json.loads(os.environ["DEDUPE"])
|
||||
tx = os.environ.get("TX", "")
|
||||
try:
|
||||
rows = json.load(sys.stdin)
|
||||
except Exception:
|
||||
rows = []
|
||||
rows = rows if isinstance(rows, list) else []
|
||||
norm = lambda s: re.sub(r'^.*transaction-', '', str(s or "")) # erp#37, both sides
|
||||
hits = [r for r in rows if tx and norm(r.get("num")) == tx]
|
||||
if not hits:
|
||||
sys.exit(0) # no match -> proceed to POST
|
||||
if len(hits) > 1:
|
||||
sys.exit("payment-record.sh: ABORT — transaction_id %r already appears on %d "
|
||||
"payments of this invoice; the target holds duplicates, refusing to "
|
||||
"guess" % (tx, len(hits)))
|
||||
r = hits[0]
|
||||
if ded["supplier"] and ded["amount"] is not None:
|
||||
try:
|
||||
got = float(r.get("amount"))
|
||||
except (TypeError, ValueError):
|
||||
got = None
|
||||
if got is None or abs(got - ded["amount"]) > 0.005:
|
||||
sys.exit("payment-record.sh: ABORT — transaction_id %r is already recorded "
|
||||
"on this invoice with amount %s, but %s was requested. Same tx, "
|
||||
"different amount is a conflict to resolve, not a dedupe."
|
||||
% (tx, r.get("amount"), ded["amount"]))
|
||||
btx = r.get("fk_bank_line")
|
||||
print(json.dumps({"id": None,
|
||||
"bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx,
|
||||
"transaction_id": tx, "deduped": True}))
|
||||
PY
|
||||
|
||||
# Correlate the created payment back to its bank transaction line. The payments
|
||||
# list carries fk_bank_line but not the paiement rowid, so match on the provided
|
||||
# transaction_id (the external bank ref), else fall back to the most recent line.
|
||||
cat > "${PYF2}" <<'PY'
|
||||
cat > "${TMPD}/correlate.py" <<'PY'
|
||||
import json, sys, os
|
||||
rows = json.load(sys.stdin); rows = rows if isinstance(rows, list) else []
|
||||
tx = os.environ.get("TX", ""); pid = int(os.environ["PAYID"])
|
||||
@@ -111,13 +167,38 @@ btx = (pick or {}).get("fk_bank_line")
|
||||
# the JSON always reports the canonical short form even on the recency fallback.
|
||||
print(json.dumps({"id": pid,
|
||||
"bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx,
|
||||
"transaction_id": (pick or {}).get("num", "") or tx}))
|
||||
"transaction_id": (pick or {}).get("num", "") or tx,
|
||||
"deduped": False}))
|
||||
PY
|
||||
|
||||
MAPPED="$(printf '%s' "${INPUT}" | python3 "${PYF}")"
|
||||
MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")"
|
||||
ENDPOINT="$(sed -n 1p <<<"${MAPPED}")"
|
||||
BODY="$(sed -n 2p <<<"${MAPPED}")"
|
||||
TX="$(sed -n 3p <<<"${MAPPED}")"
|
||||
DEDUPE="$(sed -n 4p <<<"${MAPPED}")"
|
||||
|
||||
# --- Pre-POST dedupe (erp#44) — only possible when a transaction_id was given ---
|
||||
if [[ -n "${TX}" ]]; then
|
||||
set +e
|
||||
"${W}" GET "${ENDPOINT}" > "${TMPD}/payments.json" 2> "${TMPD}/list.err"
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ ${rc} -ne 0 ]]; then
|
||||
if grep -q "HTTP 404" "${TMPD}/list.err"; then
|
||||
printf '[]' > "${TMPD}/payments.json" # empty lists can answer 404 (dolibarr skill gotcha)
|
||||
else
|
||||
cat "${TMPD}/list.err" >&2
|
||||
echo "payment-record.sh: could not list ${ENDPOINT} — refusing to pay blind (dedupe impossible)" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
MATCH="$(DEDUPE="${DEDUPE}" TX="${TX}" python3 "${TMPD}/dedupe.py" < "${TMPD}/payments.json")"
|
||||
if [[ -n "${MATCH}" ]]; then
|
||||
echo "payment-record.sh: transaction_id ${TX} already recorded on this invoice — deduped, no POST" >&2
|
||||
printf '%s\n' "${MATCH}"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
PAYID="$("${W}" POST "${ENDPOINT}" "${BODY}")"
|
||||
if [[ ! "${PAYID}" =~ ^[0-9]+$ ]]; then
|
||||
@@ -126,4 +207,4 @@ if [[ ! "${PAYID}" =~ ^[0-9]+$ ]]; then
|
||||
fi
|
||||
|
||||
# Same path serves the GET list; resolve fk_bank_line and emit the enriched record.
|
||||
"${W}" GET "${ENDPOINT}" | PAYID="${PAYID}" TX="${TX}" python3 "${PYF2}"
|
||||
"${W}" GET "${ENDPOINT}" | PAYID="${PAYID}" TX="${TX}" python3 "${TMPD}/correlate.py"
|
||||
|
||||
Reference in New Issue
Block a user