arcodange-org/erp
1
0
Fork 0
Code Issues 18 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Merge pull request 'fix(txid): normalize bank tx ids to fit Dolibarr's num_payment varchar(50)' (#37) from arcodange/txid-varchar50-normalize into main

Browse Source 
Reviewed-on: #37
This commit was merged in pull request #37.
This commit is contained in:
arcodange
2026-07-11 18:05:55 +02:00
parent 5f5b6f872a f9d83037b6
commit 0d66d6a6dc
21 changed files with 349 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/dolibarr-sandbox-write/SKILL.md
+22 -4
View File
@@ -96,9 +96,10 @@ to leave a draft. Emits `{id, ref, ref_supplier, total_ht, total_ttc, statut}`.
### 3 · Payment (règlement) — `scripts/payment-record.sh`
```sh
echo '{"invoice_id":19,"mode":"VIR","account_id":1,"transaction_id":"QONTO-TX-1234"}' | scripts/payment-record.sh
echo '{"invoice_id":13,"kind":"supplier","mode":"VIR","account_id":1,"amount":96,"transaction_id":"WISE-TX-5678"}' \
| scripts/payment-record.sh
echo '{"invoice_id":19,"mode":"VIR","account_id":1,"transaction_id":"2159468139"}' | scripts/payment-record.sh
echo '{"invoice_id":13,"kind":"supplier","mode":"VIR","account_id":1,"amount":96,
"transaction_id":"arcodange-1246-1-transaction-019f14c5-e254-7ac9-9e9f-307ed9-d55f44"}' \
| scripts/payment-record.sh # stores num 019f14c5-e254-7ac9-9e9f-307ed9-d55f44
```
The invoice must be **validated** first. `mode`: `VIR|CB|CHQ|LIQ`. Customer
payments settle the full remaining amount and mark the invoice paid; **supplier**
@@ -112,9 +113,22 @@ reconciliation matches **by id** rather than by fuzzy amount/date. (`num` is a
back-compat alias for the same field.) Recording without it prints a warning — the
payment still posts, but it won't auto-reconcile.
**Canonical short form — the varchar(50) constraint.** Dolibarr stores
`num_payment` in **varchar(50)** columns (`llx_paiement.num_paiement`,
`llx_paiementfourn.num_paiement`) while Qonto transaction ids run ~67 chars
(`<org>-<n>-<n>-transaction-<uuid>`) — POSTing one raw fails with HTTP 400
"value too long for type character varying(50)". The script therefore normalizes
before POST: everything through `transaction-` is stripped and the **UUID suffix**
(globally unique, ~37 chars) is what gets stored. Wise ids (short numerics) pass
through unchanged. An id still >50 chars after normalization is **refused** with
an explicit error — never truncated silently. `arcodange-bank-reco`'s bank-match
normalizes feed ids the same way, so short-form nums keep reconciling by id
(historical long-form nums too). Pass the raw feed id; the script does the rest.
Emits **`{id, bank_transaction_id, transaction_id}`**. `bank_transaction_id` is the
Dolibarr bank line (`llx_bank.fk_bank_line`) the payment created — the id the
reconciliation (`arcodange-bank-reco`) keys on. Both ends are captured at write time.
reconciliation (`arcodange-bank-reco`) keys on. `transaction_id` echoes the
**normalized** num actually stored. Both ends are captured at write time.
### 4 · Credit note (avoir) — `scripts/creditnote-create.sh`
@@ -180,6 +194,10 @@ sandbox KissMetrics on `--target sandbox` and the prod one on `--target prod`.
REST create needs `code_client`/`code_fournisseur = "-1"` to trigger it — the
script does this; without it the API errors `ErrorCustomerCodeRequired`.
- **Dates** are sent as Unix epochs; pass `date:"YYYY-MM-DD"` or omit for today.
- **Qonto tx ids don't fit varchar(50).** `payment-record.sh` strips the
`<org>-<n>-<n>-transaction-` prefix and stores the UUID suffix (see workflow 3);
ids still >50 chars after that are refused. Offline proof: `tests/run-tests.sh`
(stubbed `dol-write.sh` via the `DOL_WRITE` hook — writes nothing anywhere).
- **`banque lire`** (rights id 111) is granted → `scripts/bank-accounts.sh` lists
accounts (id/label/bank) so a payment can pick its `account_id`. It's in the
provisioner's `WRITE_IDS`, so a fresh `provisionSandbox.ts` run includes it.
.claude/skills/dolibarr-sandbox-write/scripts/payment-record.sh
+30 -4
View File
@@ -12,10 +12,18 @@
# tx id from the feed). Stored on the payment's bank line
# (llx_bank.num_chq) so the règlement reconciles to the feed by id.
# `num` is a back-compat alias for the same field.
# NORMALIZED before POST: Dolibarr stores num_payment in varchar(50)
# (llx_paiement.num_paiement / llx_paiementfourn.num_paiement) while
# Qonto ids run ~67 chars (<org>-<n>-<n>-transaction-<uuid>), so
# everything through "transaction-" is stripped and the UUID suffix
# (globally unique, ~37 chars) is the canonical stored form. Wise ids
# (short numerics) pass through unchanged. An id still >50 chars after
# normalization is REFUSED with an error — never truncated silently.
# comment (optional)
#
# The invoice must be VALIDATED first (invoice-create.sh ... "validate":true).
# Emits {id, bank_transaction_id, transaction_id} on stdout. `bank_transaction_id`
# Emits {id, bank_transaction_id, transaction_id} on stdout — `transaction_id` is
# the NORMALIZED num actually stored (what bank-match keys on). `bank_transaction_id`
# is the Dolibarr bank line (llx_bank.fk_bank_line) the payment created — the id
# bank reconciliation (arcodange-bank-reco) keys on to link this règlement to a
# statement line. Recording without a transaction_id warns (it won't auto-reconcile).
@@ -29,7 +37,7 @@ if [[ -n "${SRC}" && "${SRC}" != "-" ]]; then INPUT="$(cat "${SRC}")"; else INPU
PYF="$(mktemp -t dolpy.XXXXXX)"; PYF2="$(mktemp -t dolpy2.XXXXXX)"
trap 'rm -f "${PYF}" "${PYF2}"' EXIT
cat > "${PYF}" <<'PY'
import json, sys, datetime
import json, sys, datetime, re
d = json.loads(sys.stdin.read())
if not d.get("invoice_id"):
sys.exit("payment-record.sh: 'invoice_id' is required")
@@ -46,7 +54,23 @@ epoch = int((datetime.datetime.strptime(ds, "%Y-%m-%d") if ds
else datetime.datetime.now()).timestamp())
inv = d["invoice_id"]
# transaction_id is the first-class bank-feed tx id; num is the back-compat alias.
tx = str(d.get("transaction_id") or d.get("num") or "")
raw_tx = str(d.get("transaction_id") or d.get("num") or "")
# Normalize to the canonical short form. Dolibarr stores num_payment in
# varchar(50) (llx_paiement.num_paiement / llx_paiementfourn.num_paiement) and
# a Qonto id (~67 chars, <org>-<n>-<n>-transaction-<uuid>) blows past it —
# HTTP 400 "value too long for type character varying(50)". Strip everything
# through "transaction-" and store the UUID suffix (globally unique). Wise ids
# (short numerics, no "transaction-") pass through unchanged. bank-match.sh
# normalizes feed ids the same way, so the short form still reconciles by id.
tx = re.sub(r'^.*transaction-', '', raw_tx)
if tx != raw_tx:
sys.stderr.write("payment-record.sh: transaction_id normalized to %r "
"(Qonto prefix stripped — num_payment is varchar(50))\n" % tx)
if len(tx) > 50:
sys.exit("payment-record.sh: transaction_id %r is %d chars even after "
"normalization — Dolibarr's num_payment is varchar(50) and silent "
"truncation would break bank reconciliation; pass a shorter id"
% (tx, len(tx)))
if not tx:
sys.stderr.write("payment-record.sh: WARNING — no transaction_id given; this "
"règlement won't auto-reconcile to the bank feed\n")
@@ -83,9 +107,11 @@ if tx:
if pick is None and rows:
pick = max(rows, key=lambda r: r.get("date", ""))
btx = (pick or {}).get("fk_bank_line")
# transaction_id: the num as stored (normalized) — fall back to the sent TX so
# the JSON always reports the canonical short form even on the recency fallback.
print(json.dumps({"id": pid,
"bank_transaction_id": int(btx) if btx and str(btx).isdigit() else btx,
"transaction_id": (pick or {}).get("num", "")}))
"transaction_id": (pick or {}).get("num", "") or tx}))
PY
MAPPED="$(printf '%s' "${INPUT}" | python3 "${PYF}")"
.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh
Executable
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env bash
# Offline tests for payment-record.sh's transaction_id normalization — the writer
# side of the varchar(50) fix (Dolibarr's num_payment columns truncate at 50 chars,
# Qonto ids run ~67). Uses tests/stub-dol-write.sh via the DOL_WRITE env hook, so
# NOTHING is written to the sandbox or prod.
# 1. Long Qonto id → POST carries the UUID suffix; JSON reports it; stderr says so.
# 2. Wise numeric id → passes through untouched, no normalization notice.
# 3. Id still >50 chars after normalization → refused BEFORE any POST, error
# cites varchar(50) (never a silent truncation).
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PR="${SCRIPT_DIR}/../scripts/payment-record.sh"
STUB="${SCRIPT_DIR}/stub-dol-write.sh"
fail() { echo "FAIL: $*" >&2; exit 1; }
bash -n "${PR}" || fail "bash -n payment-record.sh"
bash -n "${STUB}" || fail "bash -n stub-dol-write.sh"
STATE="$(mktemp -d -t prtest.XXXXXX)"
trap 'rm -rf "${STATE}"' EXIT
LONG="arcodange-1246-1-transaction-019f14c5-e254-7ac9-9e9f-307ed9-d55f44"
SHORT="019f14c5-e254-7ac9-9e9f-307ed9-d55f44"
# --- Case 1: long Qonto id is normalized to the UUID suffix ---
OUT="$(printf '{"invoice_id":13,"kind":"supplier","account_id":1,"amount":96,"transaction_id":"%s"}' "${LONG}" \
| DOL_WRITE="${STUB}" STUB_STATE="${STATE}" bash "${PR}" 2>"${STATE}/stderr1")" \
|| fail "long-qonto-id: expected success, got $?"
grep -q "\"num_payment\": \"${SHORT}\"" "${STATE}/post_body.json" \
|| fail "long-qonto-id: POST body must carry the SHORT num, got: $(cat "${STATE}/post_body.json")"
python3 -c "
import json, sys
o = json.loads('''${OUT}''')
assert o['transaction_id'] == '${SHORT}', o
assert o['id'] == 77 and o['bank_transaction_id'] == 556, o
" || fail "long-qonto-id: output JSON must report the normalized num, got: ${OUT}"
grep -q 'normalized' "${STATE}/stderr1" || fail "long-qonto-id: normalization must be announced on stderr"
# --- Case 2: Wise numeric id passes through unchanged ---
OUT="$(printf '{"invoice_id":19,"account_id":2,"transaction_id":"2159468139"}' \
| DOL_WRITE="${STUB}" STUB_STATE="${STATE}" bash "${PR}" 2>"${STATE}/stderr2")" \
|| fail "wise-id: expected success, got $?"
grep -q '"num_payment": "2159468139"' "${STATE}/post_body.json" \
|| fail "wise-id: POST body must carry the id untouched"
grep -q 'normalized' "${STATE}/stderr2" && fail "wise-id: must NOT announce a normalization"
# --- Case 3: >50 chars after normalization is refused before any POST ---
rm -f "${STATE}/post_body.json" "${STATE}/post_endpoint"
BAD="qonto-migration-batch-7-payment-reference-0123456789-0123456789" # 63 chars, no "transaction-"
rc=0
printf '{"invoice_id":13,"kind":"supplier","account_id":1,"amount":96,"transaction_id":"%s"}' "${BAD}" \
| DOL_WRITE="${STUB}" STUB_STATE="${STATE}" bash "${PR}" >/dev/null 2>"${STATE}/stderr3" || rc=$?
[[ "${rc}" -ne 0 ]] || fail "overlong-id: must exit non-zero"
[[ ! -f "${STATE}/post_body.json" ]] || fail "overlong-id: must refuse BEFORE any POST"
grep -q 'varchar(50)' "${STATE}/stderr3" || fail "overlong-id: error must cite the varchar(50) constraint"
echo "OK: payment-record normalization tests passed (long→short, wise untouched, >50 refused pre-POST)"
.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh
Executable
+27
View File
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
# Offline stand-in for dol-write.sh, used ONLY by tests/run-tests.sh (injected via
# the DOL_WRITE env hook). Records the POST body under $STUB_STATE, then serves it
# back as the payments list on GET so payment-record.sh can correlate. Never talks
# to any host — sandbox and prod are both out of reach by construction.
set -euo pipefail
STATE="${STUB_STATE:?stub-dol-write.sh: STUB_STATE not set}"
METHOD="$1"; ENDPOINT="$2"; BODY="${3:-}"
case "${METHOD}" in
POST)
printf '%s' "${BODY}" > "${STATE}/post_body.json"
printf '%s\n' "${ENDPOINT}" > "${STATE}/post_endpoint"
echo "77"
;;
GET)
python3 - "${STATE}/post_body.json" <<'PY'
import json, sys
body = json.load(open(sys.argv[1]))
print(json.dumps([{"num": body.get("num_payment", ""),
"date": "2026-06-20 12:00:00",
"fk_bank_line": "556"}]))
PY
;;
*)
echo "stub-dol-write.sh: unexpected method ${METHOD}" >&2; exit 2
;;
esac