feat(write-skill): idempotency keys — manifest replay is a no-op (erp#44)
Learning #4 of the 2026-07-11 rehearsal: manifest B failed mid-run and could not be re-applied — op 1 (the DARNIS invoice) had already run and a replay would have duplicated it. Every write op now dedupes BEFORE any POST: - thirdparty-create.sh: by exact name (promote '#thirdparty:name=' semantics); ambiguous (2+) aborts; an existing fiche missing the requested role aborts (refuse-never-repair). Emits {"id", "deduped"} instead of a bare id. - invoice-create.sh: supplier kind by (socid, ref_supplier) — same key with a different total aborts as a conflict; customer kind (or supplier without ref_supplier) by (socid, date, total_ttc ±0.02, line fingerprint) with descs HTML-unescaped. Credit notes are never candidates. A deduped DRAFT with validate:true is validated on replay, so an interrupted run converges. - payment-record.sh: by (invoice, amount, normalized transaction_id), composing with the erp#37 varchar(50) normalization on BOTH sides so historical long-form nums still match; same tx + different amount aborts; without a tx id there is no dedupe key (warned). Dedupe answers id:null (the payments list exposes no paiement rowid) + the existing bank line. - All three refuse to POST blind when the dedupe lookup fails with anything but the documented empty-list 404 (the voir_tous trap would otherwise mint dupes). - promote-apply.sh: marks each op created / deduped=true inline and totals them in the summary — an all-deduped second run is visible proof of a no-op. - promote-plan.sh: advertises each op's dedupe key (and flags tx=MISSING as 'a replay WILL double-pay'). Proof: - tests/run-tests.sh: 5 new offline cases (11 total) — dedupe hits POST nothing, conflicts/ambiguity abort pre-POST, long-form history dedupes, draft convergence validates; stub extended to serve the new lookups with the live-observed empty behaviors ([] for invoices/payments, 404 for tiers). - tests/replay-idempotency.sh (new, live): double-applies a self-contained manifest on the sandbox — run 1 '3 created' (rows 1/1/1), run 2 '3 deduped' with row counts unchanged and the stored num in erp#37 short form. - The historic manifest-B now replays on the sandbox as 5/5 deduped, zero new rows — the exact replay Learning #4 declared impossible. SKILL.md updated in the same change (per-op dedupe keys, replay-safety section, gotchas); the 2026-07-11 runbook's Learning #4 carries a dated resolution addendum. Closes erp#44. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
Claude Fable 5
@@ -1,6 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Create a customer or supplier invoice (facture) with product/service lines in
|
||||
# the SANDBOX, optionally validating it.
|
||||
# the SANDBOX, optionally validating it — IDEMPOTENT (erp#44): replaying the
|
||||
# same create returns the existing invoice instead of minting a duplicate.
|
||||
#
|
||||
# Input: a JSON object on stdin (or a file path in $1):
|
||||
# socid (required) thirdparty id
|
||||
@@ -10,7 +11,28 @@
|
||||
# validate true|false (default false = leave draft)
|
||||
# lines: [ { desc, qty, price_ht, tva, type: "product"|"service", product_id? } ]
|
||||
#
|
||||
# Emits {id, ref, ref_supplier, total_ht, total_ttc, statut} on stdout.
|
||||
# Idempotency (erp#44 — Learning #4 of the 2026-07-11 rehearsal, where a manifest
|
||||
# that failed mid-run could not be replayed because op 1 would have re-created
|
||||
# the DARNIS invoice). BEFORE any POST, list the thirdparty's invoices of the
|
||||
# same kind and dedupe:
|
||||
# - supplier kind WITH ref_supplier : by (socid, ref_supplier) — the supplier's
|
||||
# own ref is the natural key. A match whose total_ttc DIFFERS from the
|
||||
# requested lines ABORTS (same key, different content = a data conflict to
|
||||
# resolve by a human, not a dedupe).
|
||||
# - customer kind (or supplier without ref_supplier) : by (socid, date,
|
||||
# total_ttc ±0.02, line fingerprint) where the fingerprint is the multiset of
|
||||
# (desc, qty, subprice, tva_tx) per line — desc HTML-unescaped and
|
||||
# whitespace-collapsed, since Dolibarr returns stored descriptions
|
||||
# HTML-encoded.
|
||||
# - 2+ matches ABORT (ambiguous — the target already has duplicates; never
|
||||
# guess). Credit notes (type=2) are never dedupe candidates.
|
||||
# On a match: if `validate:true` was requested and the match is still a DRAFT
|
||||
# (statut=0 — e.g. run 1 died between create and validate), it is validated now,
|
||||
# so a replayed manifest CONVERGES instead of leaving a half-done op behind.
|
||||
# A listing failure other than 404 ABORTS — POSTing blind would mint duplicates.
|
||||
#
|
||||
# Emits {id, ref, ref_supplier, total_ht, total_ttc, statut, deduped} on stdout
|
||||
# — deduped:true means no invoice was created (the id is the pre-existing one).
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
|
||||
@@ -18,8 +40,8 @@ W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
|
||||
SRC="${1:-}"
|
||||
if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi
|
||||
|
||||
PYF="$(mktemp -t dolpy.XXXXXX)"; trap 'rm -f "${PYF}"' EXIT
|
||||
cat > "${PYF}" <<'PY'
|
||||
TMPD="$(mktemp -d -t invcre.XXXXXX)"; trap 'rm -rf "${TMPD}"' EXIT
|
||||
cat > "${TMPD}/map.py" <<'PY'
|
||||
import json, sys, datetime
|
||||
d = json.loads(sys.stdin.read())
|
||||
if not d.get("socid"):
|
||||
@@ -27,8 +49,8 @@ if not d.get("socid"):
|
||||
supplier = d.get("kind", "customer").lower() in ("supplier", "fournisseur")
|
||||
endpoint = "/supplierinvoices" if supplier else "/invoices"
|
||||
ds = d.get("date")
|
||||
epoch = int((datetime.datetime.strptime(ds, "%Y-%m-%d") if ds
|
||||
else datetime.datetime.now()).timestamp())
|
||||
dt = datetime.datetime.strptime(ds, "%Y-%m-%d") if ds else datetime.datetime.now()
|
||||
epoch = int(dt.timestamp())
|
||||
lines = []
|
||||
for ln in d.get("lines", []):
|
||||
is_product = ln.get("type", "service").lower() in ("product", "produit")
|
||||
@@ -45,16 +67,127 @@ for ln in d.get("lines", []):
|
||||
body = {"socid": d["socid"], "date": epoch, "type": 0, "lines": lines}
|
||||
if supplier and d.get("ref_supplier"):
|
||||
body["ref_supplier"] = d["ref_supplier"]
|
||||
# Expected total_ttc, Dolibarr-style (per-line rounding), for the dedupe key.
|
||||
total_ttc = round(sum(round(float(L["qty"]) * float(L["subprice"]), 2)
|
||||
* (1 + float(L["tva_tx"]) / 100.0) for L in lines), 2)
|
||||
dedupe = {"supplier": supplier, "socid": str(d["socid"]),
|
||||
"ref_supplier": str(d.get("ref_supplier") or "") if supplier else "",
|
||||
"date": dt.strftime("%Y-%m-%d"), "total_ttc": total_ttc,
|
||||
"lines": [[L["desc"], L["qty"], L["subprice"], L["tva_tx"]] for L in lines]}
|
||||
print(endpoint)
|
||||
print(json.dumps(body))
|
||||
print("1" if d.get("validate") else "0")
|
||||
print(json.dumps(dedupe))
|
||||
PY
|
||||
|
||||
MAPPED="$(printf '%s' "${INPUT}" | python3 "${PYF}")"
|
||||
MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")"
|
||||
ENDPOINT="$(sed -n 1p <<<"${MAPPED}")"
|
||||
BODY="$(sed -n 2p <<<"${MAPPED}")"
|
||||
VALIDATE="$(sed -n 3p <<<"${MAPPED}")"
|
||||
DEDUPE="$(sed -n 4p <<<"${MAPPED}")"
|
||||
printf '%s' "${DEDUPE}" > "${TMPD}/dedupe.json"
|
||||
|
||||
# Final read-back: shared by the create and dedupe paths.
|
||||
emit_summary() { # $1 = invoice id, $2 = deduped true|false
|
||||
"${W}" GET "${ENDPOINT}/$1" | DEDUPED="$2" python3 -c "import json,sys,os
|
||||
d=json.load(sys.stdin)
|
||||
o={k:d.get(k) for k in ('id','ref','ref_supplier','total_ht','total_ttc','statut')}
|
||||
o['deduped']=os.environ['DEDUPED']=='true'
|
||||
print(json.dumps(o))"
|
||||
}
|
||||
|
||||
# --- Dedupe lookup (erp#44): list this thirdparty's invoices of the same kind ---
|
||||
SOCID="$(python3 -c "import json,sys; print(json.load(sys.stdin)['socid'])" < "${TMPD}/dedupe.json")"
|
||||
set +e
|
||||
"${W}" GET "${ENDPOINT}?thirdparty_ids=${SOCID}&limit=500" \
|
||||
> "${TMPD}/list.json" 2> "${TMPD}/list.err"
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ ${rc} -ne 0 ]]; then
|
||||
if grep -q "HTTP 404" "${TMPD}/list.err"; then
|
||||
printf '[]' > "${TMPD}/list.json" # empty lists can answer 404 (dolibarr skill gotcha)
|
||||
else
|
||||
cat "${TMPD}/list.err" >&2
|
||||
echo "invoice-create.sh: could not list ${ENDPOINT} for socid ${SOCID} — refusing to POST blind (dedupe impossible)" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
cat > "${TMPD}/match.py" <<'PY'
|
||||
import json, sys, html, datetime
|
||||
ded = json.load(open(sys.argv[1]))
|
||||
try:
|
||||
rows = json.load(open(sys.argv[2]))
|
||||
except Exception:
|
||||
rows = []
|
||||
rows = [r for r in (rows if isinstance(rows, list) else [])
|
||||
if str(r.get("type", "0")) != "2"] # credit notes never dedupe a create
|
||||
|
||||
def norm_desc(s):
|
||||
return " ".join(html.unescape(str(s or "")).split())
|
||||
def fnum(v, nd):
|
||||
try: return round(float(v), nd)
|
||||
except Exception: return None
|
||||
def fingerprint(lines):
|
||||
return sorted((norm_desc(l[0]), fnum(l[1], 3), fnum(l[2], 2), fnum(l[3], 3))
|
||||
for l in lines)
|
||||
|
||||
TOL = 0.02
|
||||
want_fp = fingerprint(ded["lines"])
|
||||
hits = []
|
||||
if ded["supplier"] and ded["ref_supplier"]:
|
||||
key = ded["ref_supplier"].strip()
|
||||
hits = [r for r in rows if str(r.get("ref_supplier") or "").strip() == key]
|
||||
if len(hits) == 1 and ded["lines"]:
|
||||
got = fnum(hits[0].get("total_ttc"), 2)
|
||||
if got is not None and abs(got - ded["total_ttc"]) > TOL:
|
||||
sys.exit("invoice-create.sh: ABORT — supplier invoice with ref_supplier "
|
||||
"%r already exists on socid %s (id %s) but its total_ttc %.2f "
|
||||
"differs from the requested %.2f. Same key, different content "
|
||||
"is a conflict to resolve, not a dedupe." %
|
||||
(key, ded["socid"], hits[0].get("id"), got, ded["total_ttc"]))
|
||||
else:
|
||||
for r in rows:
|
||||
try:
|
||||
rdate = datetime.datetime.fromtimestamp(int(r.get("date"))).strftime("%Y-%m-%d")
|
||||
except Exception:
|
||||
continue
|
||||
if rdate != ded["date"]:
|
||||
continue
|
||||
got = fnum(r.get("total_ttc"), 2)
|
||||
if got is None or abs(got - ded["total_ttc"]) > TOL:
|
||||
continue
|
||||
rl = r.get("lines")
|
||||
if not isinstance(rl, list):
|
||||
sys.exit("invoice-create.sh: ABORT — candidate invoice id %s matches "
|
||||
"(date, total) but carries no inline lines to fingerprint; "
|
||||
"refusing to guess" % r.get("id"))
|
||||
if fingerprint([[l.get("desc"), l.get("qty"), l.get("subprice"), l.get("tva_tx")]
|
||||
for l in rl]) == want_fp:
|
||||
hits.append(r)
|
||||
if len(hits) > 1:
|
||||
sys.exit("invoice-create.sh: ABORT — dedupe key matches %d invoices on socid %s "
|
||||
"(ids %s): the target already holds duplicates; refusing to guess."
|
||||
% (len(hits), ded["socid"], ", ".join(str(h.get("id")) for h in hits)))
|
||||
if hits:
|
||||
print(json.dumps({"id": int(hits[0]["id"]), "statut": str(hits[0].get("statut"))}))
|
||||
PY
|
||||
MATCH="$(python3 "${TMPD}/match.py" "${TMPD}/dedupe.json" "${TMPD}/list.json")"
|
||||
|
||||
if [[ -n "${MATCH}" ]]; then
|
||||
ID="$(python3 -c "import json,sys; print(json.load(sys.stdin)['id'])" <<<"${MATCH}")"
|
||||
STATUT="$(python3 -c "import json,sys; print(json.load(sys.stdin)['statut'])" <<<"${MATCH}")"
|
||||
echo "invoice-create.sh: invoice already exists on socid ${SOCID} (id ${ID}) — deduped, no POST" >&2
|
||||
# Converge an interrupted run: validate the matched draft if validation was asked.
|
||||
if [[ "${VALIDATE}" == "1" && "${STATUT}" == "0" ]]; then
|
||||
echo "invoice-create.sh: matched invoice ${ID} is still a draft — validating it now (completes the interrupted op)" >&2
|
||||
"${W}" POST "${ENDPOINT}/${ID}/validate" '{}' >/dev/null
|
||||
fi
|
||||
emit_summary "${ID}" true
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# --- Create (no match) ----------------------------------------------------------
|
||||
ID="$("${W}" POST "${ENDPOINT}" "${BODY}")"
|
||||
if [[ ! "${ID}" =~ ^[0-9]+$ ]]; then
|
||||
echo "invoice-create.sh: create did not return an id: ${ID}" >&2
|
||||
@@ -63,6 +196,4 @@ fi
|
||||
if [[ "${VALIDATE}" == "1" ]]; then
|
||||
"${W}" POST "${ENDPOINT}/${ID}/validate" '{}' >/dev/null
|
||||
fi
|
||||
"${W}" GET "${ENDPOINT}/${ID}" | python3 -c "import json,sys
|
||||
d=json.load(sys.stdin)
|
||||
print(json.dumps({k:d.get(k) for k in ('id','ref','ref_supplier','total_ht','total_ttc','statut')}))"
|
||||
emit_summary "${ID}" false
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Record a payment (règlement) on a validated invoice in the SANDBOX.
|
||||
# Record a payment (règlement) on a validated invoice in the SANDBOX —
|
||||
# IDEMPOTENT (erp#44): replaying the same règlement is a no-op, not a double
|
||||
# payment.
|
||||
#
|
||||
# Input: a JSON object on stdin (or a file path in $1):
|
||||
# invoice_id (required) the invoice to pay
|
||||
@@ -21,12 +23,28 @@
|
||||
# normalization is REFUSED with an error — never truncated silently.
|
||||
# comment (optional)
|
||||
#
|
||||
# Idempotency (erp#44, composing with the erp#37 normalization): BEFORE any POST,
|
||||
# the invoice's payment list is fetched and deduped by
|
||||
# (invoice, amount, normalized transaction_id):
|
||||
# - a row whose num — normalized the same way (historical rows may still carry
|
||||
# long-form Qonto ids) — equals the normalized transaction_id is a REPLAY:
|
||||
# for supplier payments the amounts must also agree (±0.005; same tx id with
|
||||
# a DIFFERENT amount ABORTS as a data conflict); customer payments settle the
|
||||
# full remaining so the tx id alone is the key.
|
||||
# - a dedupe hit emits {"id": null, "bank_transaction_id": <existing line>,
|
||||
# "transaction_id": <normalized>, "deduped": true} and exits 0 WITHOUT
|
||||
# posting (the payments list does not expose the paiement rowid — id is null
|
||||
# by honesty, the bank line is the stable handle reconciliation keys on).
|
||||
# - the SAME normalized tx appearing on 2+ rows of this invoice ABORTS (the
|
||||
# target already holds duplicates; never guess).
|
||||
# - WITHOUT a transaction_id there is NO dedupe key — the payment posts with a
|
||||
# warning (as before) and a replay WILL duplicate it. Always pass the tx id.
|
||||
# - a listing failure other than 404 ABORTS: paying blind would double-pay.
|
||||
#
|
||||
# The invoice must be VALIDATED first (invoice-create.sh ... "validate":true).
|
||||
# Emits {id, bank_transaction_id, transaction_id} on stdout — `transaction_id` is
|
||||
# the NORMALIZED num actually stored (what bank-match keys on). `bank_transaction_id`
|
||||
# is the Dolibarr bank line (llx_bank.fk_bank_line) the payment created — the id
|
||||
# bank reconciliation (arcodange-bank-reco) keys on to link this règlement to a
|
||||
# statement line. Recording without a transaction_id warns (it won't auto-reconcile).
|
||||
# Emits {id, bank_transaction_id, transaction_id, deduped} on stdout —
|
||||
# `transaction_id` is the NORMALIZED num actually stored (what bank-match keys
|
||||
# on), `bank_transaction_id` the Dolibarr bank line (llx_bank.fk_bank_line).
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
|
||||
@@ -34,9 +52,9 @@ W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
|
||||
SRC="${1:-}"
|
||||
if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi
|
||||
|
||||
PYF="$(mktemp -t dolpy.XXXXXX)"; PYF2="$(mktemp -t dolpy2.XXXXXX)"
|
||||
trap 'rm -f "${PYF}" "${PYF2}"' EXIT
|
||||
cat > "${PYF}" <<'PY'
|
||||
TMPD="$(mktemp -d -t payrec.XXXXXX)"
|
||||
trap 'rm -rf "${TMPD}"' EXIT
|
||||
cat > "${TMPD}/map.py" <<'PY'
|
||||
import json, sys, datetime, re
|
||||
d = json.loads(sys.stdin.read())
|
||||
if not d.get("invoice_id"):
|
||||
@@ -55,7 +73,7 @@ epoch = int((datetime.datetime.strptime(ds, "%Y-%m-%d") if ds
|
||||
inv = d["invoice_id"]
|
||||
# transaction_id is the first-class bank-feed tx id; num is the back-compat alias.
|
||||
raw_tx = str(d.get("transaction_id") or d.get("num") or "")
|
||||
# Normalize to the canonical short form. Dolibarr stores num_payment in
|
||||
# Normalize to the canonical short form (erp#37). Dolibarr stores num_payment in
|
||||
# varchar(50) (llx_paiement.num_paiement / llx_paiementfourn.num_paiement) and
|
||||
# a Qonto id (~67 chars, <org>-<n>-<n>-transaction-<uuid>) blows past it —
|
||||
# HTTP 400 "value too long for type character varying(50)". Strip everything
|
||||
@@ -73,7 +91,8 @@ if len(tx) > 50:
|
||||
% (tx, len(tx)))
|
||||
if not tx:
|
||||
sys.stderr.write("payment-record.sh: WARNING — no transaction_id given; this "
|
||||
"règlement won't auto-reconcile to the bank feed\n")
|
||||
"règlement won't auto-reconcile to the bank feed AND cannot "
|
||||
"be deduped on a replay (erp#44) — a re-run will double-pay\n")
|
||||
if supplier:
|
||||
if d.get("amount") is None:
|
||||
sys.exit("payment-record.sh: supplier payments require an 'amount'")
|
||||
@@ -90,12 +109,49 @@ else:
|
||||
print(endpoint)
|
||||
print(json.dumps(body))
|
||||
print(tx)
|
||||
print(json.dumps({"supplier": supplier,
|
||||
"amount": (float(d["amount"]) if supplier else None)}))
|
||||
PY
|
||||
|
||||
# Dedupe a replay against the invoice's existing payments (erp#44).
|
||||
cat > "${TMPD}/dedupe.py" <<'PY'
|
||||
import json, sys, os, re
|
||||
ded = json.loads(os.environ["DEDUPE"])
|
||||
tx = os.environ.get("TX", "")
|
||||
try:
|
||||
rows = json.load(sys.stdin)
|
||||
except Exception:
|
||||
rows = []
|
||||
rows = rows if isinstance(rows, list) else []
|
||||
norm = lambda s: re.sub(r'^.*transaction-', '', str(s or "")) # erp#37, both sides
|
||||
hits = [r for r in rows if tx and norm(r.get("num")) == tx]
|
||||
if not hits:
|
||||
sys.exit(0) # no match -> proceed to POST
|
||||
if len(hits) > 1:
|
||||
sys.exit("payment-record.sh: ABORT — transaction_id %r already appears on %d "
|
||||
"payments of this invoice; the target holds duplicates, refusing to "
|
||||
"guess" % (tx, len(hits)))
|
||||
r = hits[0]
|
||||
if ded["supplier"] and ded["amount"] is not None:
|
||||
try:
|
||||
got = float(r.get("amount"))
|
||||
except (TypeError, ValueError):
|
||||
got = None
|
||||
if got is None or abs(got - ded["amount"]) > 0.005:
|
||||
sys.exit("payment-record.sh: ABORT — transaction_id %r is already recorded "
|
||||
"on this invoice with amount %s, but %s was requested. Same tx, "
|
||||
"different amount is a conflict to resolve, not a dedupe."
|
||||
% (tx, r.get("amount"), ded["amount"]))
|
||||
btx = r.get("fk_bank_line")
|
||||
print(json.dumps({"id": None,
|
||||
"bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx,
|
||||
"transaction_id": tx, "deduped": True}))
|
||||
PY
|
||||
|
||||
# Correlate the created payment back to its bank transaction line. The payments
|
||||
# list carries fk_bank_line but not the paiement rowid, so match on the provided
|
||||
# transaction_id (the external bank ref), else fall back to the most recent line.
|
||||
cat > "${PYF2}" <<'PY'
|
||||
cat > "${TMPD}/correlate.py" <<'PY'
|
||||
import json, sys, os
|
||||
rows = json.load(sys.stdin); rows = rows if isinstance(rows, list) else []
|
||||
tx = os.environ.get("TX", ""); pid = int(os.environ["PAYID"])
|
||||
@@ -111,13 +167,38 @@ btx = (pick or {}).get("fk_bank_line")
|
||||
# the JSON always reports the canonical short form even on the recency fallback.
|
||||
print(json.dumps({"id": pid,
|
||||
"bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx,
|
||||
"transaction_id": (pick or {}).get("num", "") or tx}))
|
||||
"transaction_id": (pick or {}).get("num", "") or tx,
|
||||
"deduped": False}))
|
||||
PY
|
||||
|
||||
MAPPED="$(printf '%s' "${INPUT}" | python3 "${PYF}")"
|
||||
MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")"
|
||||
ENDPOINT="$(sed -n 1p <<<"${MAPPED}")"
|
||||
BODY="$(sed -n 2p <<<"${MAPPED}")"
|
||||
TX="$(sed -n 3p <<<"${MAPPED}")"
|
||||
DEDUPE="$(sed -n 4p <<<"${MAPPED}")"
|
||||
|
||||
# --- Pre-POST dedupe (erp#44) — only possible when a transaction_id was given ---
|
||||
if [[ -n "${TX}" ]]; then
|
||||
set +e
|
||||
"${W}" GET "${ENDPOINT}" > "${TMPD}/payments.json" 2> "${TMPD}/list.err"
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ ${rc} -ne 0 ]]; then
|
||||
if grep -q "HTTP 404" "${TMPD}/list.err"; then
|
||||
printf '[]' > "${TMPD}/payments.json" # empty lists can answer 404 (dolibarr skill gotcha)
|
||||
else
|
||||
cat "${TMPD}/list.err" >&2
|
||||
echo "payment-record.sh: could not list ${ENDPOINT} — refusing to pay blind (dedupe impossible)" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
MATCH="$(DEDUPE="${DEDUPE}" TX="${TX}" python3 "${TMPD}/dedupe.py" < "${TMPD}/payments.json")"
|
||||
if [[ -n "${MATCH}" ]]; then
|
||||
echo "payment-record.sh: transaction_id ${TX} already recorded on this invoice — deduped, no POST" >&2
|
||||
printf '%s\n' "${MATCH}"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
PAYID="$("${W}" POST "${ENDPOINT}" "${BODY}")"
|
||||
if [[ ! "${PAYID}" =~ ^[0-9]+$ ]]; then
|
||||
@@ -126,4 +207,4 @@ if [[ ! "${PAYID}" =~ ^[0-9]+$ ]]; then
|
||||
fi
|
||||
|
||||
# Same path serves the GET list; resolve fk_bank_line and emit the enriched record.
|
||||
"${W}" GET "${ENDPOINT}" | PAYID="${PAYID}" TX="${TX}" python3 "${PYF2}"
|
||||
"${W}" GET "${ENDPOINT}" | PAYID="${PAYID}" TX="${TX}" python3 "${TMPD}/correlate.py"
|
||||
|
||||
@@ -40,6 +40,7 @@ OP_SCRIPT = {"thirdparty": "thirdparty-create.sh", "invoice": "invoice-create.sh
|
||||
"creditnote": "creditnote-create.sh", "payment": "payment-record.sh",
|
||||
"thirdparty_update": "thirdparty-update.sh", "contact": "contact-create.sh"}
|
||||
refmap = {}
|
||||
n_created = n_deduped = 0
|
||||
|
||||
import urllib.parse
|
||||
DOL_WRITE = os.environ.get("DOL_WRITE") # GET wrapper for the chosen target
|
||||
@@ -117,16 +118,28 @@ for i, op in enumerate(ops, 1):
|
||||
ref = op.get("ref")
|
||||
if ref and rid is not None:
|
||||
refmap[ref] = int(rid) if str(rid).isdigit() else rid
|
||||
# Surface the idempotency evidence inline: contact dedupes and
|
||||
# thirdparty-update read-back diffs are the proof a re-apply is a no-op.
|
||||
# Surface the idempotency evidence inline: every op script reports
|
||||
# deduped=true/false (erp#44), and thirdparty-update reports its read-back
|
||||
# diff — together the proof a re-apply is a no-op.
|
||||
extra = ""
|
||||
if isinstance(parsed, dict):
|
||||
if parsed.get("deduped"):
|
||||
n_deduped += 1
|
||||
extra += " deduped=true (already on target — no write)"
|
||||
elif "deduped" in parsed:
|
||||
n_created += 1
|
||||
extra += " created"
|
||||
ch = parsed.get("changed")
|
||||
if isinstance(ch, dict):
|
||||
extra += " changed=%d%s" % (len(ch),
|
||||
(" [%s]" % ", ".join(sorted(ch))) if ch else " (no-op)")
|
||||
print(" [%d/%d] %-17s %-8s -> id=%s%s" % (i, len(ops), t, ("@" + ref) if ref else "", rid, extra))
|
||||
print("OK — promote complete. ref -> id: %s" % json.dumps(refmap))
|
||||
print(" [%d/%d] %-17s %-8s -> id=%s%s" % (i, len(ops), t, ("@" + ref) if ref else "",
|
||||
rid if rid is not None else "-", extra))
|
||||
counts = []
|
||||
if n_created:
|
||||
counts.append("%d created" % n_created)
|
||||
if n_deduped:
|
||||
counts.append("%d deduped" % n_deduped)
|
||||
print("OK — promote complete%s. ref -> id: %s"
|
||||
% ((" (%s)" % ", ".join(counts)) if counts else "", json.dumps(refmap)))
|
||||
PY
|
||||
|
||||
@@ -17,11 +17,17 @@ for i, op in enumerate(ops, 1):
|
||||
t = op["op"]; inp = op.get("input", {}); ref = op.get("ref")
|
||||
print(" %d. %s%s" % (i, t, (" => @%s" % ref) if ref else ""))
|
||||
if t == "thirdparty":
|
||||
print(" name=%r role=%s%s" % (inp.get("name"), inp.get("role", "client"),
|
||||
print(" name=%r role=%s%s (idempotent: dedupe by exact name; ambiguous aborts)"
|
||||
% (inp.get("name"), inp.get("role", "client"),
|
||||
(" tva=%s" % inp["tva_intra"]) if inp.get("tva_intra") else ""))
|
||||
elif t == "invoice":
|
||||
print(" socid=%s kind=%s%s validate=%s" % (inp.get("socid"), inp.get("kind", "customer"),
|
||||
(" ref_supplier=%s" % inp["ref_supplier"]) if inp.get("ref_supplier") else "", bool(inp.get("validate"))))
|
||||
supplier = str(inp.get("kind", "customer")).lower() in ("supplier", "fournisseur")
|
||||
dk = ("socid+ref_supplier" if supplier and inp.get("ref_supplier")
|
||||
else "socid+date+total+lines")
|
||||
print(" socid=%s kind=%s%s validate=%s (idempotent: dedupe by %s)"
|
||||
% (inp.get("socid"), inp.get("kind", "customer"),
|
||||
(" ref_supplier=%s" % inp["ref_supplier"]) if inp.get("ref_supplier") else "",
|
||||
bool(inp.get("validate")), dk))
|
||||
for ln in inp.get("lines", []):
|
||||
print(" - %r qty=%s pu_ht=%s tva=%s%% [%s]" % (ln.get("desc", ""), ln.get("qty", 1),
|
||||
ln.get("price_ht", ln.get("subprice")), ln.get("tva", ln.get("tva_tx", 20)), ln.get("type", "service")))
|
||||
@@ -34,7 +40,8 @@ for i, op in enumerate(ops, 1):
|
||||
txid = inp.get("transaction_id") or inp.get("num")
|
||||
print(" invoice=%s mode=%s account=%s %s%s" % (inp.get("invoice_id"), inp.get("mode", "VIR"),
|
||||
inp.get("account_id"), ("amount=%s" % inp["amount"]) if inp.get("amount") else "(full)",
|
||||
(" tx=%s" % txid) if txid else " tx=MISSING"))
|
||||
(" tx=%s (idempotent: dedupe by invoice+amount+tx)" % txid) if txid
|
||||
else " tx=MISSING (no dedupe key — a replay WILL double-pay)"))
|
||||
elif t == "thirdparty_update":
|
||||
flds = inp.get("fields") or {}
|
||||
print(" socid=%s update %d dossier field(s): %s" % (inp.get("socid"), len(flds),
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# Create a client and/or supplier thirdparty (fiche tiers) in the SANDBOX.
|
||||
# Create a client and/or supplier thirdparty (fiche tiers) in the SANDBOX —
|
||||
# IDEMPOTENT (erp#44): replaying the same create is a no-op, not a duplicate.
|
||||
#
|
||||
# Input: a JSON object on stdin (or a file path in $1). Fields:
|
||||
# name (required)
|
||||
@@ -8,8 +9,21 @@
|
||||
# client_code / supplier_code default "-1" = auto-generate via the code mask
|
||||
# siret, tva_intra, address, zip, town, email, phone, idprof1 (optional)
|
||||
#
|
||||
# Emits the new thirdparty id on stdout. All writes go through dol-write.sh,
|
||||
# which refuses any host that is not the sandbox.
|
||||
# Idempotency (erp#44): BEFORE any POST, look the name up on the target with the
|
||||
# same semantics as promote-apply's `#thirdparty:name=...` lookup
|
||||
# (GET /thirdparties?sqlfilters=(t.nom:=:'name'), limit 2):
|
||||
# - 0 matches (the API answers HTTP 404, not []) -> create
|
||||
# - 1 match whose roles cover the requested role -> {"id": <existing>, "deduped": true}
|
||||
# - 1 match MISSING the requested role -> ABORT (refuse-never-repair:
|
||||
# silently reusing a client fiche as a supplier would skip the code mask and
|
||||
# hide a data problem — fix the fiche deliberately, not as a create side effect)
|
||||
# - 2+ matches -> ABORT (ambiguous, never guess)
|
||||
# Any other listing failure ABORTS: assuming "no match" on e.g. a 403 would mint
|
||||
# duplicates — the exact failure mode this dedupe exists to prevent.
|
||||
#
|
||||
# Emits {"id": N, "deduped": false} after a create, {"id": N, "deduped": true}
|
||||
# after a dedupe hit. All writes go through dol-write.sh, which refuses any host
|
||||
# that is not the sandbox.
|
||||
#
|
||||
# Examples:
|
||||
# echo '{"name":"KissMetrics","role":"client","tva_intra":"US.."}' | thirdparty-create.sh
|
||||
@@ -21,9 +35,11 @@ W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
|
||||
SRC="${1:-}"
|
||||
if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPUT="$(cat)"; fi
|
||||
|
||||
PYF="$(mktemp -t dolpy.XXXXXX)"; trap 'rm -f "${PYF}"' EXIT
|
||||
cat > "${PYF}" <<'PY'
|
||||
import json, sys
|
||||
TMPD="$(mktemp -d -t tpcre.XXXXXX)"; trap 'rm -rf "${TMPD}"' EXIT
|
||||
|
||||
# --- 1. Validate + map the POST body (before ANY request) ----------------------
|
||||
cat > "${TMPD}/map.py" <<'PY'
|
||||
import json, sys, urllib.parse
|
||||
d = json.loads(sys.stdin.read())
|
||||
if not d.get("name"):
|
||||
sys.exit("thirdparty-create.sh: 'name' is required")
|
||||
@@ -43,8 +59,74 @@ body = {
|
||||
for k in ("siret", "tva_intra", "address", "zip", "town", "email", "phone", "idprof1"):
|
||||
if d.get(k):
|
||||
body[k] = d[k]
|
||||
# Line 1: POST body. Line 2: the dedupe lookup path (exact name, promote-apply
|
||||
# `#thirdparty:name=` semantics — SQL-escape ' by doubling, then URL-encode).
|
||||
print(json.dumps(body))
|
||||
flt = "(t.nom:=:'%s')" % str(d["name"]).replace("'", "''")
|
||||
print("/thirdparties?limit=2&sqlfilters=" + urllib.parse.quote(flt))
|
||||
PY
|
||||
MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/map.py")"
|
||||
BODY="$(sed -n 1p <<<"${MAPPED}")"
|
||||
LOOKUP="$(sed -n 2p <<<"${MAPPED}")"
|
||||
printf '%s' "${BODY}" > "${TMPD}/body.json"
|
||||
|
||||
BODY="$(printf '%s' "${INPUT}" | python3 "${PYF}")"
|
||||
"${W}" POST /thirdparties "${BODY}"
|
||||
# --- 2. Dedupe by exact name against the target (erp#44) -----------------------
|
||||
set +e
|
||||
"${W}" GET "${LOOKUP}" > "${TMPD}/list.json" 2> "${TMPD}/list.err"
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ ${rc} -ne 0 ]]; then
|
||||
if grep -q "HTTP 404" "${TMPD}/list.err"; then
|
||||
printf '[]' > "${TMPD}/list.json" # no-match answers 404, not [] (dolibarr skill gotcha)
|
||||
else
|
||||
cat "${TMPD}/list.err" >&2
|
||||
echo "thirdparty-create.sh: could not look up name on the target — refusing to POST blind (dedupe impossible)" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
cat > "${TMPD}/match.py" <<'PY'
|
||||
import json, sys
|
||||
body = json.load(open(sys.argv[1]))
|
||||
try:
|
||||
rows = json.load(open(sys.argv[2]))
|
||||
except Exception:
|
||||
rows = []
|
||||
rows = rows if isinstance(rows, list) else []
|
||||
if len(rows) > 1:
|
||||
sys.exit("thirdparty-create.sh: ABORT — name %r matches %d thirdparties on the "
|
||||
"target (ids %s); ambiguous, refusing to guess (same rule as the "
|
||||
"promote '#thirdparty:name=' lookup)"
|
||||
% (body["name"], len(rows), ", ".join(str(r.get("id")) for r in rows)))
|
||||
if rows:
|
||||
r = rows[0]
|
||||
want_client = body["client"] == "1"
|
||||
want_supp = body["fournisseur"] == "1"
|
||||
# Dolibarr: client '1'=customer '2'=prospect '3'=both; fournisseur '1'=yes.
|
||||
has_client = str(r.get("client") or "0") in ("1", "2", "3")
|
||||
has_supp = str(r.get("fournisseur") or "0") == "1"
|
||||
missing = []
|
||||
if want_client and not has_client: missing.append("client")
|
||||
if want_supp and not has_supp: missing.append("supplier")
|
||||
if missing:
|
||||
sys.exit("thirdparty-create.sh: ABORT — %r already exists (id %s) but "
|
||||
"without the requested role(s): %s. Refusing to dedupe onto a "
|
||||
"fiche that can't carry the ops that follow, and refusing to "
|
||||
"mutate its roles as a create side effect — fix the fiche "
|
||||
"deliberately first." % (body["name"], r.get("id"), ", ".join(missing)))
|
||||
print(json.dumps({"id": int(r["id"]), "deduped": True}))
|
||||
PY
|
||||
MATCH="$(python3 "${TMPD}/match.py" "${TMPD}/body.json" "${TMPD}/list.json")"
|
||||
if [[ -n "${MATCH}" ]]; then
|
||||
echo "thirdparty-create.sh: thirdparty already exists — deduped, no POST" >&2
|
||||
printf '%s\n' "${MATCH}"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# --- 3. POST the new thirdparty -------------------------------------------------
|
||||
NEWID="$("${W}" POST /thirdparties "${BODY}")"
|
||||
if [[ ! "${NEWID}" =~ ^[0-9]+$ ]]; then
|
||||
echo "thirdparty-create.sh: create did not return an id: ${NEWID}" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf '{"id": %s, "deduped": false}\n' "${NEWID}"
|
||||
|
||||
Reference in New Issue
Block a user