feat(bank-match): exact tx-id match pass (consumes payment transaction_id)
The consumer side of erp#26/#27: now that a règlement stores its originating bank transaction id (transaction_id -> llx_bank.num_chq), bank-match uses it. New PASS 0 (exact), highest priority, before wire-ref and amt+date: - carry each feed movement's own id (Qonto transaction id; Wise activity + transfer resource id) as feed_ids, and each Dolibarr payment's num. - match when a payment's num equals a feed id. Tagged [tx-id]. - DATE-WINDOW-INDEPENDENT — the id is proof, so it pairs movements whose bank settlement and Dolibarr saisie are weeks apart (which amt+date would miss). Pass 0 runs before the ref index is built, so its matches are excluded from the later passes (no double-match). Fixture-proven: a payment dated 15d off the bank movement (outside the ±7d window) matches via [tx-id] when num carries the Qonto id, and correctly does NOT match when num is empty. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -175,7 +175,7 @@ V7 adds three improvements that reshape the output buckets:
|
|||||||
|
|
||||||
| Bucket | Meaning | Counts toward exit-1? |
|
| Bucket | Meaning | Counts toward exit-1? |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| **MATCHED** | Bank ↔ Dolibarr paired. Annotated with match kind: `[wire-ref]` (strong, via `--enrich`) or `[amt+date]` (loose). | No |
|
| **MATCHED** | Bank ↔ Dolibarr paired. Match kind: `[tx-id]` (exact — the payment's `transaction_id` equals the feed tx id; date-independent), `[wire-ref]` (strong, via `--enrich`), or `[amt+date]` (loose). | No |
|
||||||
| **INTERNAL** | Wise↔Qonto consolidations (5000€ moved between Arcodange's own accounts). | No |
|
| **INTERNAL** | Wise↔Qonto consolidations (5000€ moved between Arcodange's own accounts). | No |
|
||||||
| **AVOIR-NETTED** | Dolibarr AVC + FAC cancellation cycles paired and excluded (the bank only saw the net). | No |
|
| **AVOIR-NETTED** | Dolibarr AVC + FAC cancellation cycles paired and excluded (the bank only saw the net). | No |
|
||||||
| **BANK-ONLY — known patterns** | Bank movement with a `known-patterns.json` annotation. Intentional gap. | No |
|
| **BANK-ONLY — known patterns** | Bank movement with a `known-patterns.json` annotation. Intentional gap. | No |
|
||||||
@@ -185,6 +185,18 @@ V7 adds three improvements that reshape the output buckets:
|
|||||||
|
|
||||||
Exit code 0 iff the two "real gap" buckets are empty.
|
Exit code 0 iff the two "real gap" buckets are empty.
|
||||||
|
|
||||||
|
### Matching priority — exact tx-id first
|
||||||
|
|
||||||
|
Matching runs in three passes, highest confidence first:
|
||||||
|
|
||||||
|
1. **`[tx-id]` (exact)** — a Dolibarr payment whose stored `num` (`llx_bank.num_chq`,
|
||||||
|
set from the règlement's `transaction_id`, see `dolibarr-sandbox-write`) equals the
|
||||||
|
feed transaction's own id. **Date-window-independent** — the id is proof, so it
|
||||||
|
matches even when bank settlement and Dolibarr saisie are weeks apart. Record
|
||||||
|
payments with their `transaction_id` and reconciliation becomes deterministic.
|
||||||
|
2. **`[wire-ref]` (strong)** — via `--enrich`, below.
|
||||||
|
3. **`[amt+date]` (loose)** — the fallback heuristic.
|
||||||
|
|
||||||
### `--enrich` — wire-reference strong matching
|
### `--enrich` — wire-reference strong matching
|
||||||
|
|
||||||
`bank-match.sh --enrich` fetches `/v1/transfers/{id}` for each Wise TRANSFER and reads the `reference` field (the wire memo from the sender, e.g. `FROM KISSMETRICS HOLDINGS INC FOR INVOICE FAC002CL0001002/ VENDOR:DEV`). When the reference contains a `FAC\d+(CL\d+)?` pattern matching a Dolibarr customer invoice, that pairing takes precedence over the loose date+amount match. Only the strong-matched ones get `[wire-ref]`; the rest fall through to `[amt+date]`. Cost: 1 extra HTTP call per Wise transfer.
|
`bank-match.sh --enrich` fetches `/v1/transfers/{id}` for each Wise TRANSFER and reads the `reference` field (the wire memo from the sender, e.g. `FROM KISSMETRICS HOLDINGS INC FOR INVOICE FAC002CL0001002/ VENDOR:DEV`). When the reference contains a `FAC\d+(CL\d+)?` pattern matching a Dolibarr customer invoice, that pairing takes precedence over the loose date+amount match. Only the strong-matched ones get `[wire-ref]`; the rest fall through to `[amt+date]`. Cost: 1 extra HTTP call per Wise transfer.
|
||||||
|
|||||||
@@ -112,7 +112,10 @@ for t in (json.load(open(os.path.join(work,"qonto.json"))).get("transactions") o
|
|||||||
amt = float(t.get("amount") or 0)
|
amt = float(t.get("amount") or 0)
|
||||||
sign = "+" if t.get("side") == "credit" else "-"
|
sign = "+" if t.get("side") == "credit" else "-"
|
||||||
label = t.get("label") or t.get("operation_type") or "-"
|
label = t.get("label") or t.get("operation_type") or "-"
|
||||||
qonto_movs.append({"bank":"Qonto", "date":dt, "sign":sign, "amount":amt, "label":label[:40], "op":t.get("operation_type",""), "matched_dol":None, "matched_internal":False})
|
# feed_ids: the Qonto transaction's own id — exact-match key against a Dolibarr
|
||||||
|
# payment whose stored num (num_chq) is that id (set via payment transaction_id).
|
||||||
|
feed_ids = [str(t["id"])] if t.get("id") else []
|
||||||
|
qonto_movs.append({"bank":"Qonto", "date":dt, "sign":sign, "amount":amt, "label":label[:40], "op":t.get("operation_type",""), "feed_ids":feed_ids, "matched_dol":None, "matched_internal":False})
|
||||||
|
|
||||||
# 4b. Normalize Wise
|
# 4b. Normalize Wise
|
||||||
wise_movs = []
|
wise_movs = []
|
||||||
@@ -129,7 +132,12 @@ for a in (json.load(open(os.path.join(work,"wise.json"))).get("activities") or [
|
|||||||
title = strip(a.get("title") or "")[:40]
|
title = strip(a.get("title") or "")[:40]
|
||||||
res = a.get("resource") or {}
|
res = a.get("resource") or {}
|
||||||
resource_id = str(res.get("id")) if res.get("type") == "TRANSFER" else None
|
resource_id = str(res.get("id")) if res.get("type") == "TRANSFER" else None
|
||||||
wise_movs.append({"bank":"Wise", "date":dt, "sign":sign, "amount":amt, "label":title, "op":typ, "matched_dol":None, "matched_internal":False, "wise_resource_id":resource_id, "wire_ref":""})
|
# feed_ids: the Wise activity id and (for transfers) the resource id — either
|
||||||
|
# may have been stored as the payment's transaction_id; match against both.
|
||||||
|
feed_ids = []
|
||||||
|
if a.get("id"): feed_ids.append(str(a["id"]))
|
||||||
|
if resource_id: feed_ids.append(resource_id)
|
||||||
|
wise_movs.append({"bank":"Wise", "date":dt, "sign":sign, "amount":amt, "label":title, "op":typ, "feed_ids":feed_ids, "matched_dol":None, "matched_internal":False, "wise_resource_id":resource_id, "wire_ref":""})
|
||||||
|
|
||||||
# 4b'. If --enrich, load per-transfer wire references and attach to Wise movs
|
# 4b'. If --enrich, load per-transfer wire references and attach to Wise movs
|
||||||
if enrich:
|
if enrich:
|
||||||
@@ -165,6 +173,7 @@ for fn in os.listdir(os.path.join(work,"dol_pay")):
|
|||||||
amt = float(p.get("amount") or 0)
|
amt = float(p.get("amount") or 0)
|
||||||
dol_pays.append({"side":"customer", "ref":inv["ref"], "date":d, "amount":amt,
|
dol_pays.append({"side":"customer", "ref":inv["ref"], "date":d, "amount":amt,
|
||||||
"fk_account":inv.get("fk_account"), "socid":inv.get("socid"),
|
"fk_account":inv.get("fk_account"), "socid":inv.get("socid"),
|
||||||
|
"num": (p.get("num") or ""),
|
||||||
"matched_bank":None, "netted_against":None})
|
"matched_bank":None, "netted_against":None})
|
||||||
|
|
||||||
sup_by_id = {str(r["id"]): r for r in json.load(open(os.path.join(work,"dol_sup.json")))}
|
sup_by_id = {str(r["id"]): r for r in json.load(open(os.path.join(work,"dol_sup.json")))}
|
||||||
@@ -177,6 +186,7 @@ for fn in os.listdir(os.path.join(work,"dol_supay")):
|
|||||||
amt = float(p.get("amount") or 0)
|
amt = float(p.get("amount") or 0)
|
||||||
dol_pays.append({"side":"supplier", "ref":sup["ref"], "date":d, "amount":amt,
|
dol_pays.append({"side":"supplier", "ref":sup["ref"], "date":d, "amount":amt,
|
||||||
"fk_account":sup.get("fk_account"), "socid":sup.get("socid"),
|
"fk_account":sup.get("fk_account"), "socid":sup.get("socid"),
|
||||||
|
"num": (p.get("num") or ""),
|
||||||
"matched_bank":None, "netted_against":None})
|
"matched_bank":None, "netted_against":None})
|
||||||
|
|
||||||
# 4d.1. AVOIR cycle netting: an AVC (credit note) for -X on socid S cancels out
|
# 4d.1. AVOIR cycle netting: an AVC (credit note) for -X on socid S cancels out
|
||||||
@@ -204,12 +214,25 @@ for avc in avcs:
|
|||||||
avc["netted_against"] = partner["ref"]
|
avc["netted_against"] = partner["ref"]
|
||||||
partner["netted_against"] = avc["ref"]
|
partner["netted_against"] = avc["ref"]
|
||||||
|
|
||||||
# 4e. Match — two-pass:
|
# 4e. Match — three-pass, highest confidence first:
|
||||||
|
# PASS 0 (exact) : a Dolibarr payment whose stored num (num_chq = the
|
||||||
|
# transaction_id recorded with the règlement) equals the feed
|
||||||
|
# transaction's own id. Date-window-independent — the id is proof.
|
||||||
# PASS 1 (strong) : Wise transfers with an --enrich'd wire reference containing
|
# PASS 1 (strong) : Wise transfers with an --enrich'd wire reference containing
|
||||||
# a "FAC***" pattern try to match the Dolibarr invoice with
|
# a "FAC***" pattern match the Dolibarr invoice with that ref.
|
||||||
# that exact ref. This is the highest-confidence match.
|
|
||||||
# PASS 2 (loose) : remaining bank movements use the date+amount heuristic.
|
# PASS 2 (loose) : remaining bank movements use the date+amount heuristic.
|
||||||
# Netted Dolibarr entries (avoir cycle) are excluded from both passes.
|
# Netted Dolibarr entries (avoir cycle) are excluded from all passes.
|
||||||
|
|
||||||
|
# Pass 0: exact match on the feed transaction id stored as the payment's num
|
||||||
|
# (num_chq = the transaction_id recorded with the règlement). Date-independent.
|
||||||
|
for m in [x for x in bank_movs if not x["matched_internal"] and not x["matched_dol"] and x.get("feed_ids")]:
|
||||||
|
fids = set(m["feed_ids"])
|
||||||
|
for p in dol_pays:
|
||||||
|
if (p["matched_bank"] is None and p["netted_against"] is None
|
||||||
|
and p.get("num") and str(p["num"]) in fids):
|
||||||
|
m["matched_dol"] = p; m["match_kind"] = "tx-id"
|
||||||
|
p["matched_bank"] = m
|
||||||
|
break
|
||||||
|
|
||||||
# Build customer ref -> dol payment index (only un-netted, un-matched entries)
|
# Build customer ref -> dol payment index (only un-netted, un-matched entries)
|
||||||
ref_index = collections.defaultdict(list)
|
ref_index = collections.defaultdict(list)
|
||||||
|
|||||||
Reference in New Issue
Block a user