feat(write-skill): idempotency keys — manifest replay is a no-op (erp#44)
Learning #4 of the 2026-07-11 rehearsal: manifest B failed mid-run and could not be re-applied — op 1 (the DARNIS invoice) had already run and a replay would have duplicated it. Every write op now dedupes BEFORE any POST: - thirdparty-create.sh: by exact name (promote '#thirdparty:name=' semantics); ambiguous (2+) aborts; an existing fiche missing the requested role aborts (refuse-never-repair). Emits {"id", "deduped"} instead of a bare id. - invoice-create.sh: supplier kind by (socid, ref_supplier) — same key with a different total aborts as a conflict; customer kind (or supplier without ref_supplier) by (socid, date, total_ttc ±0.02, line fingerprint) with descs HTML-unescaped. Credit notes are never candidates. A deduped DRAFT with validate:true is validated on replay, so an interrupted run converges. - payment-record.sh: by (invoice, amount, normalized transaction_id), composing with the erp#37 varchar(50) normalization on BOTH sides so historical long-form nums still match; same tx + different amount aborts; without a tx id there is no dedupe key (warned). Dedupe answers id:null (the payments list exposes no paiement rowid) + the existing bank line. - All three refuse to POST blind when the dedupe lookup fails with anything but the documented empty-list 404 (the voir_tous trap would otherwise mint dupes). - promote-apply.sh: marks each op created / deduped=true inline and totals them in the summary — an all-deduped second run is visible proof of a no-op. - promote-plan.sh: advertises each op's dedupe key (and flags tx=MISSING as 'a replay WILL double-pay'). Proof: - tests/run-tests.sh: 5 new offline cases (11 total) — dedupe hits POST nothing, conflicts/ambiguity abort pre-POST, long-form history dedupes, draft convergence validates; stub extended to serve the new lookups with the live-observed empty behaviors ([] for invoices/payments, 404 for tiers). - tests/replay-idempotency.sh (new, live): double-applies a self-contained manifest on the sandbox — run 1 '3 created' (rows 1/1/1), run 2 '3 deduped' with row counts unchanged and the stored num in erp#37 short form. - The historic manifest-B now replays on the sandbox as 5/5 deduped, zero new rows — the exact replay Learning #4 declared impossible. SKILL.md updated in the same change (per-op dedupe keys, replay-safety section, gotchas); the 2026-07-11 runbook's Learning #4 carries a dated resolution addendum. Closes erp#44. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
Claude Fable 5
@@ -16,11 +16,25 @@
|
||||
# 6. happy path: thirdparty_update + contact through promote-apply
|
||||
# --target sandbox (stubbed); a second apply is a proven no-op
|
||||
# (changed=0 for the fiche, deduped=true for the contact).
|
||||
# idempotency keys (erp#44):
|
||||
# 7. invoice-create supplier dedupe by (socid, ref_supplier) → no POST,
|
||||
# deduped:true; same key + different total ABORTS as a conflict.
|
||||
# 8. invoice-create customer dedupe by (socid, date, total, line fingerprint)
|
||||
# → no POST; a different desc misses and creates.
|
||||
# 9. payment-record dedupe by (invoice, amount, normalized tx) → no POST;
|
||||
# same tx + different amount ABORTS; a historical LONG-form stored num
|
||||
# still dedupes (composes with the erp#37 normalization).
|
||||
# 10. thirdparty-create dedupe by exact name → no POST; ambiguous ABORTS;
|
||||
# an existing fiche missing the requested role ABORTS; a miss creates.
|
||||
# 11. a deduped DRAFT with validate:true is validated on replay (converges an
|
||||
# op that died between create and validate).
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PR="${SCRIPT_DIR}/../scripts/payment-record.sh"
|
||||
TU="${SCRIPT_DIR}/../scripts/thirdparty-update.sh"
|
||||
CC="${SCRIPT_DIR}/../scripts/contact-create.sh"
|
||||
IC="${SCRIPT_DIR}/../scripts/invoice-create.sh"
|
||||
TC="${SCRIPT_DIR}/../scripts/thirdparty-create.sh"
|
||||
PA="${SCRIPT_DIR}/../scripts/promote-apply.sh"
|
||||
PP="${SCRIPT_DIR}/../scripts/promote-plan.sh"
|
||||
STUB="${SCRIPT_DIR}/stub-dol-write.sh"
|
||||
@@ -30,6 +44,8 @@ fail() { echo "FAIL: $*" >&2; exit 1; }
|
||||
bash -n "${PR}" || fail "bash -n payment-record.sh"
|
||||
bash -n "${TU}" || fail "bash -n thirdparty-update.sh"
|
||||
bash -n "${CC}" || fail "bash -n contact-create.sh"
|
||||
bash -n "${IC}" || fail "bash -n invoice-create.sh"
|
||||
bash -n "${TC}" || fail "bash -n thirdparty-create.sh"
|
||||
bash -n "${PA}" || fail "bash -n promote-apply.sh"
|
||||
bash -n "${PP}" || fail "bash -n promote-plan.sh"
|
||||
bash -n "${STUB}" || fail "bash -n stub-dol-write.sh"
|
||||
@@ -139,4 +155,175 @@ grep -q -- '-> id=88' <<<"${OUT2}" || fail "run 2: dedupe must return the
|
||||
[[ ! -f "${S6}/contact_post_body.json" ]] || fail "run 2: must NOT POST a duplicate contact"
|
||||
echo "OK: promote-apply happy path — run 1 applies (changed=2, contact id 88), run 2 is a no-op (changed=0, deduped)"
|
||||
|
||||
# ============================ erp#44 idempotency ==============================
|
||||
EPOCH_0630="$(python3 -c "import datetime; print(int(datetime.datetime(2026,6,30).timestamp()))")"
|
||||
EPOCH_0531="$(python3 -c "import datetime; print(int(datetime.datetime(2026,5,31).timestamp()))")"
|
||||
|
||||
# --- Case 7: supplier invoice dedupe by (socid, ref_supplier) ---
|
||||
S7="$(mktemp -d -t ictest7.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}"' EXIT
|
||||
cat > "${S7}/supplierinvoices.json" <<JSON
|
||||
[{"id":"5","ref":"FAF2026005","ref_supplier":"F1045","socid":"7","type":"0",
|
||||
"date":${EPOCH_0630},"total_ht":"214.70000000","total_ttc":"257.64000000","statut":"1",
|
||||
"lines":[{"desc":"Apport d'affaire – juin 2026","qty":"1","subprice":"214.70000000","tva_tx":"20.0000"}]}]
|
||||
JSON
|
||||
printf '%s' '{"id":"5","ref":"FAF2026005","ref_supplier":"F1045","total_ht":"214.70000000","total_ttc":"257.64000000","statut":"1"}' \
|
||||
> "${S7}/invoice_detail_5.json"
|
||||
IN7='{"socid":7,"kind":"supplier","date":"2026-06-30","ref_supplier":"F1045","validate":true,
|
||||
"lines":[{"desc":"Apport d'"'"'affaire – juin 2026","qty":1,"price_ht":214.70,"tva":20,"type":"service"}]}'
|
||||
OUT="$(printf '%s' "${IN7}" | DOL_WRITE="${STUB}" STUB_STATE="${S7}" bash "${IC}" 2>/dev/null)" \
|
||||
|| fail "si-dedupe: expected success, got $?"
|
||||
python3 -c "
|
||||
import json
|
||||
o = json.loads('''${OUT}''')
|
||||
assert o['id'] == '5' and o['deduped'] is True, o
|
||||
" || fail "si-dedupe: must return existing id 5 with deduped:true, got: ${OUT}"
|
||||
[[ ! -f "${S7}/post_body.json" ]] || fail "si-dedupe: must NOT POST when ref_supplier matches"
|
||||
[[ ! -f "${S7}/validated_endpoint" ]] || fail "si-dedupe: an already-validated match must NOT be re-validated"
|
||||
# 7b — same (socid, ref_supplier) but different total = conflict, never a dedupe
|
||||
rc=0
|
||||
printf '%s' '{"socid":7,"kind":"supplier","date":"2026-06-30","ref_supplier":"F1045",
|
||||
"lines":[{"desc":"X","qty":1,"price_ht":999,"tva":20,"type":"service"}]}' \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S7}" bash "${IC}" >/dev/null 2>"${S7}/stderr7b" || rc=$?
|
||||
[[ "${rc}" -ne 0 ]] || fail "si-conflict: same ref_supplier + different total must abort"
|
||||
grep -q 'conflict' "${S7}/stderr7b" || fail "si-conflict: error must say it is a conflict"
|
||||
[[ ! -f "${S7}/post_body.json" ]] || fail "si-conflict: must NOT POST on a conflict"
|
||||
echo "OK: supplier invoice dedupe — (socid, ref_supplier) hit returns id, no POST; total mismatch aborts"
|
||||
|
||||
# --- Case 8: customer invoice dedupe by (socid, date, total, line fingerprint) ---
|
||||
S8="$(mktemp -d -t ictest8.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}"' EXIT
|
||||
cat > "${S8}/invoices.json" <<JSON
|
||||
[{"id":"21","ref":"FAC003","socid":"1","type":"0","date":${EPOCH_0531},
|
||||
"total_ht":"1020.00000000","total_ttc":"1020.00000000","statut":"1",
|
||||
"lines":[{"desc":"Prestation mai","qty":"1","subprice":"1020.00000000","tva_tx":"0.0000"}]}]
|
||||
JSON
|
||||
printf '%s' '{"id":"21","ref":"FAC003","ref_supplier":null,"total_ht":"1020.00000000","total_ttc":"1020.00000000","statut":"1"}' \
|
||||
> "${S8}/invoice_detail_21.json"
|
||||
OUT="$(printf '%s' '{"socid":1,"kind":"customer","date":"2026-05-31",
|
||||
"lines":[{"desc":"Prestation mai","qty":1,"price_ht":1020,"tva":0,"type":"service"}]}' \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S8}" bash "${IC}" 2>/dev/null)" \
|
||||
|| fail "ci-dedupe: expected success, got $?"
|
||||
python3 -c "
|
||||
import json
|
||||
o = json.loads('''${OUT}''')
|
||||
assert o['id'] == '21' and o['deduped'] is True, o
|
||||
" || fail "ci-dedupe: must return existing id 21 with deduped:true, got: ${OUT}"
|
||||
[[ ! -f "${S8}/post_body.json" ]] || fail "ci-dedupe: must NOT POST when the fingerprint matches"
|
||||
# 8b — a different desc breaks the fingerprint: the invoice is CREATED
|
||||
printf '%s' '{"id":"77","ref":"FAC004","ref_supplier":null,"total_ht":"1020.00000000","total_ttc":"1020.00000000","statut":"0"}' \
|
||||
> "${S8}/invoice_detail_77.json"
|
||||
OUT="$(printf '%s' '{"socid":1,"kind":"customer","date":"2026-05-31",
|
||||
"lines":[{"desc":"Prestation juin","qty":1,"price_ht":1020,"tva":0,"type":"service"}]}' \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S8}" bash "${IC}" 2>/dev/null)" \
|
||||
|| fail "ci-miss: expected success, got $?"
|
||||
python3 -c "
|
||||
import json
|
||||
o = json.loads('''${OUT}''')
|
||||
assert o['id'] == '77' and o['deduped'] is False, o
|
||||
" || fail "ci-miss: a fingerprint miss must create (deduped:false), got: ${OUT}"
|
||||
[[ -f "${S8}/post_body.json" ]] || fail "ci-miss: the create path must POST"
|
||||
echo "OK: customer invoice dedupe — fingerprint hit returns id, no POST; desc change misses and creates"
|
||||
|
||||
# --- Case 9: payment dedupe by (invoice, amount, normalized tx) ---
|
||||
S9="$(mktemp -d -t prtest9.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}"' EXIT
|
||||
cat > "${S9}/payments.json" <<JSON
|
||||
[{"amount":"96.00000000","type":"VIR","date":"2026-06-29 12:00:00",
|
||||
"num":"${SHORT}","ref":"REF2026009","fk_bank_line":"556"}]
|
||||
JSON
|
||||
OUT="$(printf '{"invoice_id":13,"kind":"supplier","account_id":1,"amount":96,"transaction_id":"%s"}' "${LONG}" \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S9}" bash "${PR}" 2>/dev/null)" \
|
||||
|| fail "pay-dedupe: expected success, got $?"
|
||||
python3 -c "
|
||||
import json
|
||||
o = json.loads('''${OUT}''')
|
||||
assert o == {'id': None, 'bank_transaction_id': 556, 'transaction_id': '${SHORT}', 'deduped': True}, o
|
||||
" || fail "pay-dedupe: must dedupe on the normalized tx, got: ${OUT}"
|
||||
[[ ! -f "${S9}/post_body.json" ]] || fail "pay-dedupe: must NOT POST a duplicate payment"
|
||||
# 9b — same tx, different amount = conflict
|
||||
rc=0
|
||||
printf '{"invoice_id":13,"kind":"supplier","account_id":1,"amount":97,"transaction_id":"%s"}' "${LONG}" \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S9}" bash "${PR}" >/dev/null 2>"${S9}/stderr9b" || rc=$?
|
||||
[[ "${rc}" -ne 0 ]] || fail "pay-conflict: same tx + different amount must abort"
|
||||
grep -q 'conflict' "${S9}/stderr9b" || fail "pay-conflict: error must say it is a conflict"
|
||||
[[ ! -f "${S9}/post_body.json" ]] || fail "pay-conflict: must NOT POST on a conflict"
|
||||
# 9c — a HISTORICAL long-form stored num still dedupes (erp#37 composition)
|
||||
cat > "${S9}/payments.json" <<JSON
|
||||
[{"amount":"96.00000000","type":"VIR","date":"2026-06-29 12:00:00",
|
||||
"num":"${LONG}","ref":"REF2026009","fk_bank_line":"556"}]
|
||||
JSON
|
||||
OUT="$(printf '{"invoice_id":13,"kind":"supplier","account_id":1,"amount":96,"transaction_id":"%s"}' "${LONG}" \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S9}" bash "${PR}" 2>/dev/null)" \
|
||||
|| fail "pay-dedupe-longnum: expected success, got $?"
|
||||
python3 -c "
|
||||
import json
|
||||
o = json.loads('''${OUT}''')
|
||||
assert o['deduped'] is True and o['transaction_id'] == '${SHORT}', o
|
||||
" || fail "pay-dedupe-longnum: historical long num must normalize and dedupe, got: ${OUT}"
|
||||
[[ ! -f "${S9}/post_body.json" ]] || fail "pay-dedupe-longnum: must NOT POST"
|
||||
echo "OK: payment dedupe — normalized tx hit is a no-op, amount mismatch aborts, long-form history still dedupes"
|
||||
|
||||
# --- Case 10: thirdparty dedupe by exact name ---
|
||||
S10="$(mktemp -d -t tctest.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}" "${S10}"' EXIT
|
||||
printf '%s' '[{"id":"7","name":"Darnis Operations","client":"0","fournisseur":"1"}]' > "${S10}/thirdparties.json"
|
||||
OUT="$(printf '%s' '{"name":"Darnis Operations","role":"supplier"}' \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S10}" bash "${TC}" 2>/dev/null)" \
|
||||
|| fail "tp-dedupe: expected success, got $?"
|
||||
python3 -c "
|
||||
import json
|
||||
o = json.loads('''${OUT}''')
|
||||
assert o == {'id': 7, 'deduped': True}, o
|
||||
" || fail "tp-dedupe: must return existing id 7 with deduped:true, got: ${OUT}"
|
||||
[[ ! -f "${S10}/tp_post_body.json" ]] || fail "tp-dedupe: must NOT POST when the name matches"
|
||||
# 10b — two matches = ambiguous, abort
|
||||
printf '%s' '[{"id":"7","name":"Darnis Operations","client":"0","fournisseur":"1"},
|
||||
{"id":"8","name":"Darnis Operations","client":"0","fournisseur":"1"}]' > "${S10}/thirdparties.json"
|
||||
rc=0
|
||||
printf '%s' '{"name":"Darnis Operations","role":"supplier"}' \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S10}" bash "${TC}" >/dev/null 2>"${S10}/stderr10b" || rc=$?
|
||||
[[ "${rc}" -ne 0 ]] || fail "tp-ambiguous: 2 name matches must abort"
|
||||
grep -qi 'ambiguous' "${S10}/stderr10b" || fail "tp-ambiguous: error must say ambiguous"
|
||||
[[ ! -f "${S10}/tp_post_body.json" ]] || fail "tp-ambiguous: must NOT POST"
|
||||
# 10c — existing fiche missing the requested role = abort (refuse-never-repair)
|
||||
printf '%s' '[{"id":"7","name":"Darnis Operations","client":"0","fournisseur":"1"}]' > "${S10}/thirdparties.json"
|
||||
rc=0
|
||||
printf '%s' '{"name":"Darnis Operations","role":"client"}' \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S10}" bash "${TC}" >/dev/null 2>"${S10}/stderr10c" || rc=$?
|
||||
[[ "${rc}" -ne 0 ]] || fail "tp-role: role mismatch must abort"
|
||||
grep -q 'client' "${S10}/stderr10c" || fail "tp-role: error must name the missing role"
|
||||
[[ ! -f "${S10}/tp_post_body.json" ]] || fail "tp-role: must NOT POST"
|
||||
# 10d — no match (stub answers the Dolibarr 404) → the create path POSTs
|
||||
rm -f "${S10}/thirdparties.json"
|
||||
OUT="$(printf '%s' '{"name":"Fresh Supplier","role":"supplier"}' \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S10}" bash "${TC}" 2>/dev/null)" \
|
||||
|| fail "tp-miss: expected success, got $?"
|
||||
python3 -c "
|
||||
import json
|
||||
o = json.loads('''${OUT}''')
|
||||
assert o == {'id': 90, 'deduped': False}, o
|
||||
" || fail "tp-miss: a miss must create (deduped:false), got: ${OUT}"
|
||||
[[ -f "${S10}/tp_post_body.json" ]] || fail "tp-miss: the create path must POST"
|
||||
echo "OK: thirdparty dedupe — exact-name hit, ambiguous abort, role-mismatch abort, miss creates"
|
||||
|
||||
# --- Case 11: a deduped DRAFT with validate:true is validated (run converges) ---
|
||||
S11="$(mktemp -d -t ictest11.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}" "${S10}" "${S11}"' EXIT
|
||||
cat > "${S11}/supplierinvoices.json" <<JSON
|
||||
[{"id":"5","ref":"(PROV5)","ref_supplier":"F1045","socid":"7","type":"0",
|
||||
"date":${EPOCH_0630},"total_ht":"214.70000000","total_ttc":"257.64000000","statut":"0",
|
||||
"lines":[{"desc":"Apport","qty":"1","subprice":"214.70000000","tva_tx":"20.0000"}]}]
|
||||
JSON
|
||||
printf '%s' '{"id":"5","ref":"FAF2026005","ref_supplier":"F1045","total_ht":"214.70000000","total_ttc":"257.64000000","statut":"1"}' \
|
||||
> "${S11}/invoice_detail_5.json"
|
||||
OUT="$(printf '%s' '{"socid":7,"kind":"supplier","date":"2026-06-30","ref_supplier":"F1045","validate":true,
|
||||
"lines":[{"desc":"Apport","qty":1,"price_ht":214.70,"tva":20,"type":"service"}]}' \
|
||||
| DOL_WRITE="${STUB}" STUB_STATE="${S11}" bash "${IC}" 2>/dev/null)" \
|
||||
|| fail "draft-converge: expected success, got $?"
|
||||
python3 -c "
|
||||
import json
|
||||
o = json.loads('''${OUT}''')
|
||||
assert o['id'] == '5' and o['deduped'] is True and o['statut'] == '1', o
|
||||
" || fail "draft-converge: must dedupe AND report the validated statut, got: ${OUT}"
|
||||
[[ ! -f "${S11}/post_body.json" ]] || fail "draft-converge: must NOT create a second invoice"
|
||||
grep -q '/supplierinvoices/5/validate' "${S11}/validated_endpoint" \
|
||||
|| fail "draft-converge: the matched draft must be validated"
|
||||
echo "OK: draft convergence — replay validates the half-done invoice instead of duplicating it"
|
||||
|
||||
echo "OK: all offline tests passed"
|
||||
|
||||
Reference in New Issue
Block a user