From fb13bdcc4f4576c76a463538211de2e4b1fec15e Mon Sep 17 00:00:00 2001 From: Gabriel Radureau Date: Sun, 19 Jul 2026 00:11:59 +0200 Subject: [PATCH] =?UTF-8?q?feat(write-skill):=20GED=20attach=20op=20?= =?UTF-8?q?=E2=80=94=20upload=20the=20source=20document=20onto=20its=20inv?= =?UTF-8?q?oice=20(erp#43)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit document-attach.sh uploads a source piece (the supplier's own PDF) onto an invoice's GED via POST /documents/upload — idempotent by (object, filename, sha256): before any POST the object's GED is listed and a same-named entry is downloaded back and sha256-compared. Identical → deduped no-op; different content → ABORT (refuse-never-repair, overwriteifexists always 0, never Dolibarr's overwrite flag). Read-back after upload: re-list + download + sha256-verify. Module-relative download paths are derived from the listing's fullname (supplier invoices carry an id-derived get_exdir prefix like 9/2/FAF2026013/…, so reconstruction would be wrong). Promote integration: new `attach` op in promote-plan/promote-apply (OP_SCRIPT), object_id resolvable via @ref and #supplierinvoice lookups; a relative `file` resolves against the manifest's directory (replay packs carry pdfs/ beside the manifest, gitignored — README documents the books@ re-fetch message ids). promote-plan prints each file's sha256 (or a loud MISSING) at review time. CLI: `arcodange sandbox attach`. Proof: offline case 12 in tests/run-tests.sh (upload body, dedupe, conflict abort, field refusal, manifest-relative resolution via stubbed /documents); live: manifest-C-ged-attach.json applied twice on the sandbox — run 1 four created, run 2 four deduped, one GED file per FAF2026010-013, stored sha256s equal to the re-fetched sources; tests/replay-idempotency.sh extended with an attach op (4 created → 4 deduped, ged_files count unchanged) and a live same-name/different-bytes abort verified. Closes erp#43 Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh --- .../skills/dolibarr-sandbox-write/SKILL.md | 55 +++- .../2026-07-11-books-regularization/README.md | 14 +- .../manifest-C-ged-attach.json | 34 +++ .../scripts/document-attach.sh | 289 ++++++++++++++++++ .../scripts/promote-apply.sh | 8 +- .../scripts/promote-plan.sh | 19 +- .../tests/replay-idempotency.sh | 54 ++-- .../dolibarr-sandbox-write/tests/run-tests.sh | 85 ++++++ .../tests/stub-dol-write.sh | 52 ++++ .gitignore | 4 + bin/arcodange | 4 + 11 files changed, 587 insertions(+), 31 deletions(-) create mode 100644 .claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/manifest-C-ged-attach.json create mode 100755 .claude/skills/dolibarr-sandbox-write/scripts/document-attach.sh diff --git a/.claude/skills/dolibarr-sandbox-write/SKILL.md b/.claude/skills/dolibarr-sandbox-write/SKILL.md index 57fae34..2f2aeb7 100644 --- a/.claude/skills/dolibarr-sandbox-write/SKILL.md +++ b/.claude/skills/dolibarr-sandbox-write/SKILL.md @@ -12,13 +12,17 @@ description: >- (thirdparty by exact name, supplier invoice by socid+ref_supplier, customer invoice by socid+date+total+line fingerprint, payment by invoice+amount+ normalized transaction id; credit notes excepted, a follow-up), so replaying - a manifest — even one that failed mid-run — is a no-op, never a duplicate. Every write goes through dol-write.sh, + a manifest — even one that failed mid-run — is a no-op, never a duplicate. + Also attaches source documents (the supplier's PDF) onto an invoice's GED + (erp#43): upload via the Documents API, idempotent by (object, filename, + sha256) — re-attach is a no-op, same name + different content aborts. Every write goes through dol-write.sh, which REFUSES any host that is not the sandbox — the structural guarantee (ADR-0003) that this skill can never mutate production. Use when the user asks to "create a thirdparty / supplier / client fiche", "compléter / mettre à jour la fiche client", "add a contact to a thirdparty", "saisir une facture", "record an invoice with lines", "enregistrer un règlement / - paiement", or to rehearse a write before promoting it to prod. SKIP for production writes + paiement", "attacher la pièce / le justificatif / le PDF à la facture", + or to rehearse a write before promoting it to prod. SKIP for production writes (prod stays read-only via the `dolibarr` skill's `ai_agent` key; promotion is a separate, human-gated replay). Depends on the write-scoped `ai_agent_sandbox` Dolibarr user + its API key. @@ -232,6 +236,31 @@ exits 0 without writing. Otherwise POST `/contacts` → `{"id": , `phone_perso`, `address`, `zip`, `town`, `country_id`, `note_public`, `note_private`. Unknown fields are refused, never dropped. +### 7 · Attach a source document (GED) — `scripts/document-attach.sh` + +```sh +echo '{"modulepart":"facture_fournisseur","ref":"FAF2026013", + "file":"pdfs/F1045_ARCODANGE_2026-06-30.pdf"}' | scripts/document-attach.sh +echo '{"modulepart":"facture","object_id":19,"file":"/abs/path/piece.pdf", + "filename":"stored-name.pdf"}' | scripts/document-attach.sh +``` +Uploads the **source piece** (the supplier's own PDF, a contract…) onto an +invoice's GED directory via `POST /documents/upload` (base64 content, +`overwriteifexists` always 0). `modulepart`: `facture`/`invoice` (customer) or +`facture_fournisseur`/`invoice_supplier`/`supplier_invoice` (supplier). Address +the invoice by `object_id` or by Dolibarr `ref` (both = cross-checked; a +ref-only lookup matching 0 or 2+ aborts). `filename` defaults to the file's +basename. Emits `{object_id, ref, modulepart, filename, sha256, size, deduped}`. + +**Idempotent by (object, filename, sha256) — our own check, never Dolibarr's +overwrite flag.** Before any POST the object's GED is listed +(`GET /documents?modulepart=…&id=…`, where 404 on a proven-existing object means +"no documents yet") and a same-named entry is **downloaded back and +sha256-compared**: identical → `{"deduped": true}`, no upload; **different +content ABORTS** — silently replacing a stored piece would rewrite evidence +(refuse-never-repair). After an upload the file is re-listed, downloaded back +and sha256-verified (read-back proof the GED holds exactly the bytes sent). + ## Promote to prod (rehearse → review → replay) The ADR-0003 capstone: take a change rehearsed in the sandbox and apply the **same @@ -240,7 +269,11 @@ array of write ops using **symbolic refs** (`@name`) instead of ids, so it is portable from sandbox to prod (an invoice references `@tp1`, the thirdparty created earlier in the run). See `examples/promote-manifest.json`. Op kinds: `thirdparty`, `thirdparty_update` (input: `socid` + `fields`), `contact`, `invoice`, `creditnote`, -`payment` — each mapping to its workflow script above. +`payment`, `attach` — each mapping to its workflow script above. An `attach` op's +relative `file` path resolves against the **manifest's directory** (the manifest +is the portable unit: a replay pack carries its `pdfs/` beside it), and +`promote-plan` prints the file's sha256 — or a loud MISSING warning — at review +time so the content is pinned before any apply. ```sh scripts/promote-plan.sh change.json # 1. human-readable review @@ -266,7 +299,8 @@ ops answer `deduped=true`, the rest execute. `promote-apply` marks each op (`OK — promote complete (1 created, 2 deduped)`), so an all-`deduped` second run is visible proof of a no-op. Live acceptance: `tests/replay-idempotency.sh` double-applies a self-contained manifest (thirdparty + supplier invoice + -payment) on the sandbox and asserts run 2 dedupes all ops with zero new rows. +payment + GED attach) on the sandbox and asserts run 2 dedupes all ops with +zero new rows and zero new GED files. A manifest value can reference another entity two ways, both resolved against the **target** so the same file is portable sandbox↔prod: @@ -313,13 +347,22 @@ sandbox KissMetrics on `--target sandbox` and the prod one on `--target prod`. (ambiguous or role-mismatch aborts), supplier invoice by (socid, ref_supplier) (total mismatch aborts), customer invoice by (socid, date, total, line fingerprint), payment by (invoice, amount, normalized tx id), contact by - (socid, email) then (socid, lastname+firstname) — each answering + (socid, email) then (socid, lastname+firstname), GED attach by (object, + filename, sha256) — each answering `{"deduped": true}` instead of minting a duplicate, so replaying a manifest is always safe. Two holes: a **payment without a `transaction_id`** has no dedupe key and WILL double-pay on a replay, and **`creditnote-create.sh` does not dedupe yet** (supplier-avoir parity follow-up) — do not replay a manifest containing a creditnote op past a mid-run failure. Offline proof: `tests/run-tests.sh`; live double-apply proof: `tests/replay-idempotency.sh`. +- **GED attach paths carry a get_exdir prefix for supplier invoices.** A + supplier-invoice document lives at `fournisseur/facture////` + (the `/` split derives from the object id), so `/documents/download` + needs `0/3/FAF2026014/file.pdf`, not `FAF2026014/file.pdf`. `document-attach.sh` + derives the module-relative path from the listing's `fullname` (substring + after the last `/facture/`) — never reconstructs it. Customer invoices have + no prefix (`facture//`). Upload answers the bare filename as a + JSON string; the script verifies it and then read-back-verifies the sha256. - **A dedupe lookup that fails (non-404) aborts the op** — the scripts refuse to POST blind, because assuming "no match" on a 403/timeout is precisely how duplicates get minted (cf. the `voir_tous` ACL trap in the `dolibarr` skill: @@ -328,5 +371,5 @@ sandbox KissMetrics on `--target sandbox` and the prod one on `--target prod`. `soc2` (seen in WIP operator payloads) is not a Dolibarr field and the API would drop it silently — `contact-create.sh` refuses it with a pointer to `poste`. -- **CLI:** all of these are also `arcodange sandbox {thirdparty|invoice|payment|creditnote|write}` +- **CLI:** all of these are also `arcodange sandbox {thirdparty|invoice|payment|creditnote|attach|write}` (JSON on stdin) — `arcodange sandbox help` for the list. diff --git a/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/README.md b/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/README.md index 428aab8..c302dfc 100644 --- a/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/README.md +++ b/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/README.md @@ -16,6 +16,7 @@ Remaining: bucket C non-invoice UI entries (erp#57). | `manifest-A-km-payments.json` | 2 KissMetrics customer payments (Wise wires 2 147,00 € + 2 195,97 €, tx ids from the activity feed) | | `manifest-B-suppliers.json` | DARNIS F1045 supplier invoice (214,70 HT / 257,64 TTC) + its payment + Anthropic/Mistral payment ops | | `manifest-B2-sandbox-payments.json` | Sandbox-side payment re-run after the varchar(50) fix (erp#37) | +| `manifest-C-ged-attach.json` | erp#43 — attach the four source PDFs onto the FAF supplier invoices (GED). `#supplierinvoice:ref_supplier=` lookups + `pdfs/`-relative paths, so the same file replays sandbox↔prod once `pdfs/` is populated (see below). Rehearsed green on the sandbox 2026-07-19: run 1 = 4 created, run 2 = 4 deduped (sha256 no-op). | | `prod-replay-prelude.sh` | Pre-replay guards: fresh checkpoint assumptions, target checks, env pinning | | `rehearsal-runbook.md` | Step-by-step of the rehearsal: what ran, in what order, with which gates | | `verify-provenance.py` | **The anti-hallucination PoC** — 36 field-level checks: every critical value re-verified against source-PDF text (pdftotext) + FRESH Qonto/Wise pulls; locale-normalized (`219,50`≡`219.50`, Wise `2,147` thousands format). 36/36 green at rehearsal time. | @@ -25,15 +26,22 @@ Remaining: bucket C non-invoice UI entries (erp#57). `verify-provenance.py` expects `./pdfs/` containing: `F1045_ARCODANGE_2026-06-30.pdf`, `F1046_ARCODANGE_2026-06-29.pdf`, `Invoice-9BF0758D-695749.pdf` (Anthropic), `invoice-MSTRL-API-814045-001.pdf` (Mistral). -Re-fetch via the `arcodange-email-ingest` skill (they live in `books@` — workflow 2 downloads -attachments by message id) or from the GED once erp#43 attaches them. +Re-fetch via the `arcodange-email-ingest` skill (workflow 2 downloads attachments by +message id) or from the GED once erp#43's attach replays on prod. Message ids (Zoho, +verified 2026-07-19 — sha256 matched the golden-set sidecars): F1045 → +`1780782481239014300` (`/Notification`), F1046 → `1782725019202004300` (`/Notification`), +Anthropic Invoice-9BF0758D-695749 → `1776017238960014300` (`/Inbox/books`), Mistral → +`1775141901205014300` (`/Inbox/books`). `pdfs/` is gitignored — populate it, then +`promote plan` prints each file's sha256 before any apply. ## Who consumes this pack - **erp#41** (provenance checker as a promote-plan stage) — industrializes `verify-provenance.py`. - **erp#42** (compliance linter) — these manifests are the *legitimate* fixtures that must PASS. - **erp#44** (idempotency keys) — replay these manifests twice on a checkpoint; run 2 must be all-deduped. -- **erp#43** (GED attach) — attaches the four PDFs to FAF2026010–013 on the sandbox. +- **erp#43** (GED attach) — `manifest-C-ged-attach.json` attaches the four PDFs to + FAF2026010–013. Sandbox rehearsal done (attach + re-attach no-op proven); the prod + replay of manifest C is the remaining human-gated step. ## Rule this pack proves diff --git a/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/manifest-C-ged-attach.json b/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/manifest-C-ged-attach.json new file mode 100644 index 0000000..a72b954 --- /dev/null +++ b/.claude/skills/dolibarr-sandbox-write/replay-packs/2026-07-11-books-regularization/manifest-C-ged-attach.json @@ -0,0 +1,34 @@ +[ + { + "op": "attach", + "input": { + "modulepart": "facture_fournisseur", + "object_id": "#supplierinvoice:ref_supplier=F1045", + "file": "pdfs/F1045_ARCODANGE_2026-06-30.pdf" + } + }, + { + "op": "attach", + "input": { + "modulepart": "facture_fournisseur", + "object_id": "#supplierinvoice:ref_supplier=F1046", + "file": "pdfs/F1046_ARCODANGE_2026-06-29.pdf" + } + }, + { + "op": "attach", + "input": { + "modulepart": "facture_fournisseur", + "object_id": "#supplierinvoice:ref_supplier=9BF0758D-695749", + "file": "pdfs/Invoice-9BF0758D-695749.pdf" + } + }, + { + "op": "attach", + "input": { + "modulepart": "facture_fournisseur", + "object_id": "#supplierinvoice:ref_supplier=MSTRL-API-814045-001", + "file": "pdfs/invoice-MSTRL-API-814045-001.pdf" + } + } +] diff --git a/.claude/skills/dolibarr-sandbox-write/scripts/document-attach.sh b/.claude/skills/dolibarr-sandbox-write/scripts/document-attach.sh new file mode 100755 index 0000000..92eb737 --- /dev/null +++ b/.claude/skills/dolibarr-sandbox-write/scripts/document-attach.sh @@ -0,0 +1,289 @@ +#!/usr/bin/env bash +# Attach a source document onto a Dolibarr invoice in the SANDBOX (GED upload) +# — IDEMPOTENT by (object, filename, sha256): re-attaching the same file is a +# no-op, and the same filename with DIFFERENT content ABORTS (a conflict for a +# human, never an overwrite). erp#43. +# +# Input: a JSON object on stdin (or a file path / inline JSON in $1): +# modulepart (required) "facture"|"invoice" → customer invoice GED +# "facture_fournisseur"|"invoice_supplier"| +# "supplier_invoice" → supplier invoice GED +# object_id invoice id on the target (one of object_id / ref +# ref Dolibarr's own ref, e.g. FAF2026013 required; both given = +# cross-checked) +# file (required) path to the file to attach. Resolved against the CWD; +# promote-apply resolves manifest-relative paths BEFORE invoking. +# filename stored name in the GED (default: basename of file) +# +# Idempotency (erp#43, the erp#44 pattern): BEFORE any POST the object's GED +# directory is listed (GET /documents) and a same-named entry — if any — is +# DOWNLOADED BACK and sha256-compared against the local file: +# - same sha256 → {"deduped": true}, no upload (re-attach = no-op); +# - different sha → ABORT. The idempotency check is OUR OWN sha256 compare, +# never Dolibarr's overwriteifexists flag (whose semantics vary by version): +# silently replacing a stored piece would rewrite evidence — refuse, never +# repair. overwriteifexists is always sent as 0. +# - a listing/download failure other than "no documents yet" (HTTP 404 on an +# object we just proved exists) ABORTS — uploading blind could duplicate. +# After an upload the GED is re-listed and the file downloaded back and +# sha256-verified: read-back proof the GED holds exactly the bytes we sent. +# +# Path gotcha: supplier-invoice documents live under an id-derived get_exdir +# prefix (e.g. fournisseur/facture/0/3/FAF2026014/…), so /documents/download +# needs "0/3/REF/file.pdf", not "REF/file.pdf". The module-relative path is +# derived from the listing's fullname (substring after the last "/facture/"). +# +# Emits {"object_id", "ref", "modulepart", "filename", "sha256", "size", +# "deduped"} on stdout. All requests go through dol-write.sh (or $DOL_WRITE), +# which refuses any host that is not the sandbox. +set -euo pipefail +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}" + +SRC="${1:-}" +if [[ -z "${SRC}" || "${SRC}" == "-" ]]; then INPUT="$(cat)" +elif [[ -f "${SRC}" ]]; then INPUT="$(cat "${SRC}")" +else INPUT="${SRC}"; fi # inline JSON + +TMPD="$(mktemp -d -t docatt.XXXXXX)"; trap 'rm -rf "${TMPD}"' EXIT + +# --- 1. Validate + normalize the input (before ANY request) -------------------- +cat > "${TMPD}/validate.py" <<'PY' +import json, os, sys +ALLOWED = ["modulepart", "object_id", "ref", "file", "filename"] +MP = {"facture": "facture", "invoice": "facture", + "facture_fournisseur": "facture_fournisseur", + "invoice_supplier": "facture_fournisseur", + "supplier_invoice": "facture_fournisseur"} +try: + d = json.loads(sys.stdin.read() or "null") +except Exception as e: + sys.exit("document-attach.sh: input is not valid JSON: %s" % e) +if not isinstance(d, dict): + sys.exit("document-attach.sh: input must be a JSON object") +bad = sorted(k for k in d if k not in ALLOWED) +if bad: + sys.exit("document-attach.sh: REFUSED — unknown field(s): %s\n allowed: %s" + % (", ".join(bad), ", ".join(ALLOWED))) +mp_raw = str(d.get("modulepart", "")).strip().lower() +if mp_raw not in MP: + sys.exit("document-attach.sh: 'modulepart' must be one of %s (got %r) — this " + "op attaches onto customer or supplier invoices only" + % ("/".join(sorted(set(MP))), mp_raw)) +mp = MP[mp_raw] +oid = d.get("object_id") +ref = str(d.get("ref") or "").strip() +if oid in (None, "") and not ref: + sys.exit("document-attach.sh: one of 'object_id' / 'ref' is required") +if oid not in (None, "") and not str(oid).isdigit(): + sys.exit("document-attach.sh: 'object_id' must be numeric, got %r" % oid) +path = str(d.get("file") or "") +if not path: + sys.exit("document-attach.sh: 'file' is required") +if not os.path.isfile(path): + sys.exit("document-attach.sh: file not found: %s\n (paths resolve against " + "the CWD; promote-apply resolves manifest-relative paths first)" % path) +filename = str(d.get("filename") or "").strip() or os.path.basename(path) +if "/" in filename or filename in (".", ".."): + sys.exit("document-attach.sh: 'filename' must be a bare file name, got %r" % filename) +print(mp) +print("/invoices" if mp == "facture" else "/supplierinvoices") +print(str(oid) if oid not in (None, "") else "") +print(ref) +print(path) +print(filename) +PY +MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/validate.py")" +MP="$(sed -n 1p <<<"${MAPPED}")" +ENDPOINT="$(sed -n 2p <<<"${MAPPED}")" +OID="$(sed -n 3p <<<"${MAPPED}")" +REF_IN="$(sed -n 4p <<<"${MAPPED}")" +FILE="$(sed -n 5p <<<"${MAPPED}")" +FILENAME="$(sed -n 6p <<<"${MAPPED}")" + +# Local sha256 + base64 (binary-safe, portable — no macOS/Linux base64 flag +# drift; the base64 goes to a file, never through argv, so size doesn't matter). +LOCAL_SHA="$(python3 - "${FILE}" "${TMPD}/b64.txt" <<'PY' +import base64, hashlib, sys +data = open(sys.argv[1], "rb").read() +open(sys.argv[2], "w").write(base64.b64encode(data).decode()) +print(hashlib.sha256(data).hexdigest()) +PY +)" + +urlenc() { python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=''))" "$1"; } + +# --- 2. Resolve the object on the target (id ⇄ ref) ---------------------------- +if [[ -n "${OID}" ]]; then + if ! "${W}" GET "${ENDPOINT}/${OID}" > "${TMPD}/obj.json" 2> "${TMPD}/obj.err"; then + cat "${TMPD}/obj.err" >&2 + echo "document-attach.sh: object ${ENDPOINT}/${OID} not readable on the target — refusing to upload blind" >&2 + exit 1 + fi + REF="$(python3 -c "import json,sys; print(json.load(sys.stdin).get('ref') or '')" < "${TMPD}/obj.json")" + if [[ -z "${REF}" ]]; then + echo "document-attach.sh: object ${ENDPOINT}/${OID} has no ref — cannot address its GED directory" >&2 + exit 1 + fi + if [[ -n "${REF_IN}" && "${REF_IN}" != "${REF}" ]]; then + echo "document-attach.sh: ABORT — object_id ${OID} is '${REF}' on the target, not the requested ref '${REF_IN}'" >&2 + exit 1 + fi +else + # ref-only: look the object up by its Dolibarr ref — 0 or 2+ matches abort. + FLT="$(urlenc "(t.ref:=:'${REF_IN}')")" + set +e + "${W}" GET "${ENDPOINT}?limit=2&sqlfilters=${FLT}" > "${TMPD}/lookup.json" 2> "${TMPD}/lookup.err" + rc=$? + set -e + if [[ ${rc} -ne 0 ]]; then + if grep -q "HTTP 404" "${TMPD}/lookup.err"; then printf '[]' > "${TMPD}/lookup.json" + else + cat "${TMPD}/lookup.err" >&2 + echo "document-attach.sh: could not look up ref '${REF_IN}' on ${ENDPOINT} — refusing to upload blind" >&2 + exit 1 + fi + fi + OID="$(python3 - "${TMPD}/lookup.json" "${REF_IN}" <<'PY' +import json, sys +try: + rows = json.load(open(sys.argv[1])) +except Exception: + rows = [] +rows = rows if isinstance(rows, list) else [] +if len(rows) == 0: + sys.exit("document-attach.sh: ref %r matched nothing on the target" % sys.argv[2]) +if len(rows) > 1: + sys.exit("document-attach.sh: ref %r is ambiguous (%d matches) — pass object_id" % (sys.argv[2], len(rows))) +print(int(rows[0]["id"])) +PY +)" + REF="${REF_IN}" +fi + +# --- 3. List the object's GED directory (the dedupe source of truth) ----------- +list_documents() { # $1 = output file; 404 on a proven-existing object = "no docs yet" + set +e + "${W}" GET "/documents?modulepart=${MP}&id=${OID}" > "$1" 2> "${TMPD}/list.err" + local rc=$? + set -e + if [[ ${rc} -ne 0 ]]; then + if grep -q "HTTP 404" "${TMPD}/list.err"; then + printf '[]' > "$1" + else + cat "${TMPD}/list.err" >&2 + echo "document-attach.sh: could not list documents for ${MP} id ${OID} — refusing to upload blind (dedupe impossible)" >&2 + exit 1 + fi + fi +} +list_documents "${TMPD}/list.json" + +# Find a same-named entry and derive its module-relative download path from +# fullname (everything after the last "/facture/" — covers both moduleparts, +# including the supplier-invoice get_exdir prefix like 0/3/REF/…). +cat > "${TMPD}/match.py" <<'PY' +import json, os, sys +try: + rows = json.load(open(sys.argv[1])) +except Exception: + rows = [] +rows = rows if isinstance(rows, list) else [] +fn = sys.argv[2] +hits = [] +for r in rows: + if str(r.get("type") or "") == "dir": + continue + name = str(r.get("relativename") or "").strip() \ + or os.path.basename(str(r.get("fullname") or "")) + if name == fn: + hits.append(r) +if len(hits) > 1: + sys.exit("document-attach.sh: ABORT — %d GED entries named %r on this object; " + "refusing to guess" % (len(hits), fn)) +if hits: + full = str(hits[0].get("fullname") or "") + i = full.rfind("/facture/") + if i < 0: + sys.exit("document-attach.sh: ABORT — cannot derive the module-relative " + "path from %r" % full) + print(json.dumps({"relpath": full[i + len("/facture/"):], + "size": hits[0].get("size")})) +PY + +remote_sha() { # $1 = module-relative path → prints sha256 of the stored bytes + local enc dl + enc="$(urlenc "$1")" + if ! dl="$("${W}" GET "/documents/download?modulepart=${MP}&original_file=${enc}" 2> "${TMPD}/dl.err")"; then + cat "${TMPD}/dl.err" >&2 + echo "document-attach.sh: could not download back '$1' to verify its sha256 — refusing to act blind" >&2 + exit 1 + fi + python3 -c " +import base64, hashlib, json, sys +d = json.load(sys.stdin) +print(hashlib.sha256(base64.b64decode(d['content'])).hexdigest())" <<<"${dl}" +} + +MATCH="$(python3 "${TMPD}/match.py" "${TMPD}/list.json" "${FILENAME}")" +if [[ -n "${MATCH}" ]]; then + RELPATH="$(python3 -c "import json,sys; print(json.load(sys.stdin)['relpath'])" <<<"${MATCH}")" + SIZE="$(python3 -c "import json,sys; print(json.load(sys.stdin)['size'])" <<<"${MATCH}")" + STORED_SHA="$(remote_sha "${RELPATH}")" + if [[ "${STORED_SHA}" == "${LOCAL_SHA}" ]]; then + echo "document-attach.sh: '${FILENAME}' already attached to ${REF} with identical sha256 — deduped, no upload" >&2 + python3 -c " +import json, sys +print(json.dumps({'object_id': int(sys.argv[1]), 'ref': sys.argv[2], + 'modulepart': sys.argv[3], 'filename': sys.argv[4], + 'sha256': sys.argv[5], 'size': int(sys.argv[6]), + 'deduped': True}))" "${OID}" "${REF}" "${MP}" "${FILENAME}" "${LOCAL_SHA}" "${SIZE}" + exit 0 + fi + echo "document-attach.sh: ABORT — '${FILENAME}' already exists on ${REF} with DIFFERENT content" >&2 + echo " stored sha256: ${STORED_SHA}" >&2 + echo " local sha256: ${LOCAL_SHA}" >&2 + echo " Same name + different bytes is a conflict for a human to resolve — never an overwrite." >&2 + exit 1 +fi + +# --- 4. Upload (no same-named entry) ------------------------------------------- +python3 - "${FILENAME}" "${MP}" "${REF}" "${TMPD}/b64.txt" > "${TMPD}/upload.json" <<'PY' +import json, sys +print(json.dumps({"filename": sys.argv[1], "modulepart": sys.argv[2], + "ref": sys.argv[3], "filecontent": open(sys.argv[4]).read().strip(), + "fileencoding": "base64", "overwriteifexists": "0"})) +PY + +RESP="$("${W}" POST /documents/upload @"${TMPD}/upload.json")" +python3 - "${RESP}" "${FILENAME}" <<'PY' +import json, sys +try: + got = json.loads(sys.argv[1]) +except Exception: + got = None +if got != sys.argv[2]: + sys.exit("document-attach.sh: upload did not confirm the filename " + "(expected %r, got %s)" % (sys.argv[2], sys.argv[1])) +PY + +# --- 5. Read-back: re-list + download + sha256 compare ------------------------- +list_documents "${TMPD}/list2.json" +MATCH2="$(python3 "${TMPD}/match.py" "${TMPD}/list2.json" "${FILENAME}")" +if [[ -z "${MATCH2}" ]]; then + echo "document-attach.sh: read-back FAILED — '${FILENAME}' not listed on ${REF} after upload" >&2 + exit 1 +fi +RELPATH2="$(python3 -c "import json,sys; print(json.load(sys.stdin)['relpath'])" <<<"${MATCH2}")" +SIZE2="$(python3 -c "import json,sys; print(json.load(sys.stdin)['size'])" <<<"${MATCH2}")" +STORED_SHA2="$(remote_sha "${RELPATH2}")" +if [[ "${STORED_SHA2}" != "${LOCAL_SHA}" ]]; then + echo "document-attach.sh: read-back FAILED — stored sha256 ${STORED_SHA2} != local ${LOCAL_SHA}" >&2 + exit 1 +fi +python3 -c " +import json, sys +print(json.dumps({'object_id': int(sys.argv[1]), 'ref': sys.argv[2], + 'modulepart': sys.argv[3], 'filename': sys.argv[4], + 'sha256': sys.argv[5], 'size': int(sys.argv[6]), + 'deduped': False}))" "${OID}" "${REF}" "${MP}" "${FILENAME}" "${LOCAL_SHA}" "${SIZE2}" diff --git a/.claude/skills/dolibarr-sandbox-write/scripts/promote-apply.sh b/.claude/skills/dolibarr-sandbox-write/scripts/promote-apply.sh index 84c8440..a014948 100755 --- a/.claude/skills/dolibarr-sandbox-write/scripts/promote-apply.sh +++ b/.claude/skills/dolibarr-sandbox-write/scripts/promote-apply.sh @@ -38,7 +38,8 @@ manifest_path, script_dir = sys.argv[1], sys.argv[2] ops = json.load(open(manifest_path)) OP_SCRIPT = {"thirdparty": "thirdparty-create.sh", "invoice": "invoice-create.sh", "creditnote": "creditnote-create.sh", "payment": "payment-record.sh", - "thirdparty_update": "thirdparty-update.sh", "contact": "contact-create.sh"} + "thirdparty_update": "thirdparty-update.sh", "contact": "contact-create.sh", + "attach": "document-attach.sh"} refmap = {} n_created = n_deduped = 0 @@ -100,6 +101,11 @@ for i, op in enumerate(ops, 1): if not script: sys.exit("promote-apply: unknown op '%s'" % t) inp = resolve(op.get("input", {})) + # attach: a relative "file" is relative to the MANIFEST, not the CWD — the + # manifest is the portable unit (replay packs carry their pdfs/ beside it). + if t == "attach" and isinstance(inp.get("file"), str) and not os.path.isabs(inp["file"]): + inp["file"] = os.path.normpath(os.path.join( + os.path.dirname(os.path.abspath(manifest_path)), inp["file"])) r = subprocess.run([os.path.join(script_dir, script)], input=json.dumps(inp), capture_output=True, text=True, env=os.environ) if r.returncode != 0: diff --git a/.claude/skills/dolibarr-sandbox-write/scripts/promote-plan.sh b/.claude/skills/dolibarr-sandbox-write/scripts/promote-plan.sh index 22299e3..27edcef 100755 --- a/.claude/skills/dolibarr-sandbox-write/scripts/promote-plan.sh +++ b/.claude/skills/dolibarr-sandbox-write/scripts/promote-plan.sh @@ -10,7 +10,7 @@ set -euo pipefail MANIFEST="${1:?usage: promote-plan.sh }" python3 - "$MANIFEST" <<'PY' -import json, sys +import hashlib, json, os, sys ops = json.load(open(sys.argv[1])) print("Promote plan — %d operation(s) (symbolic refs resolve at apply time):\n" % len(ops)) for i, op in enumerate(ops, 1): @@ -46,6 +46,23 @@ for i, op in enumerate(ops, 1): flds = inp.get("fields") or {} print(" socid=%s update %d dossier field(s): %s" % (inp.get("socid"), len(flds), ", ".join(sorted(flds)) if flds else "NONE (will be refused)")) + elif t == "attach": + obj = inp.get("object_id") or inp.get("ref") + f = inp.get("file") + # A relative file resolves against the MANIFEST (same rule as apply). + resolved = None + if isinstance(f, str): + resolved = f if os.path.isabs(f) else os.path.normpath( + os.path.join(os.path.dirname(os.path.abspath(sys.argv[1])), f)) + print(" object=%s modulepart=%s file=%s" % (obj, inp.get("modulepart"), f)) + if resolved and os.path.isfile(resolved): + data = open(resolved, "rb").read() + print(" sha256=%s (%d bytes) (idempotent: dedupe by object+filename+sha256; " + "same name + different content aborts)" + % (hashlib.sha256(data).hexdigest(), len(data))) + else: + print(" !! FILE MISSING at plan time: %s — apply WILL fail; " + "re-fetch the source first" % resolved) elif t == "contact": name = " ".join(x for x in (inp.get("firstname"), inp.get("lastname")) if x) or "?" print(" socid=%s contact %s%s%s (idempotent: dedupe by email, then lastname+firstname)" diff --git a/.claude/skills/dolibarr-sandbox-write/tests/replay-idempotency.sh b/.claude/skills/dolibarr-sandbox-write/tests/replay-idempotency.sh index dd1ecf8..0d1ba2b 100755 --- a/.claude/skills/dolibarr-sandbox-write/tests/replay-idempotency.sh +++ b/.claude/skills/dolibarr-sandbox-write/tests/replay-idempotency.sh @@ -8,10 +8,12 @@ # What it does (writes go ONLY through the host-guarded dol-write.sh): # 1. builds a small self-contained manifest with a unique-per-run fixture: # one supplier thirdparty + one validated supplier invoice (Qonto-style -# transaction id, so the erp#37 normalization is exercised too) + one payment -# 2. applies it → expects 3 created, no dedupe -# 3. applies it AGAIN → expects 3 deduped, zero new rows (verified by -# row-counting thirdparties / invoices / payments via the API) +# transaction id, so the erp#37 normalization is exercised too) + one +# payment + one GED attach (erp#43: object_id via @ref, file path relative +# to the manifest, idempotent by sha256) +# 2. applies it → expects 4 created, no dedupe +# 3. applies it AGAIN → expects 4 deduped, zero new rows (verified by +# row-counting thirdparties / invoices / payments / GED files via the API) # # Sandbox etiquette: the fixture rows stay behind (the sandbox is disposable; # a checkpoint refresh reclaims them). Run from anywhere: @@ -45,9 +47,13 @@ cat > "${EV}/manifest.json" < "${EV}/attach-fixture.pdf" count_rows() { # $1 = path, counts a JSON array (Dolibarr 404-on-empty => 0) local out @@ -61,37 +67,45 @@ FLT="$(python3 -c "import urllib.parse,sys; print(urllib.parse.quote(\"(t.nom:=: tp_count() { count_rows "/thirdparties?limit=100&sqlfilters=${FLT}"; } inv_count() { count_rows "/supplierinvoices?thirdparty_ids=$1&limit=500"; } pay_count() { count_rows "/supplierinvoices/$1/payments"; } +doc_count() { # GED files on a supplier invoice (404 = none yet => 0) + local out + if out="$("${W}" GET "/documents?modulepart=facture_fournisseur&id=$1" 2>/dev/null)"; then + python3 -c "import json,sys; r=json.load(sys.stdin); print(len([x for x in r if x.get('type')=='file']) if isinstance(r,list) else 0)" <<<"${out}" + else + echo 0 + fi +} echo "== erp#44 replay-idempotency — fixture ${STAMP} (evidence: ${EV}) ==" [[ "$(tp_count)" == "0" ]] || fail "fixture name already exists on the sandbox (clock collision?)" -echo; echo "-- RUN 1: expect 3 created ------------------------------------------------" +echo; echo "-- RUN 1: expect 4 created ------------------------------------------------" "${SCRIPTS}/promote-apply.sh" "${EV}/manifest.json" --target sandbox \ | tee "${EV}/run1.out" grep -q 'deduped' "${EV}/run1.out" && fail "run 1: nothing may dedupe on a fresh fixture" -[[ "$(grep -c ' created' "${EV}/run1.out")" == "3" ]] || fail "run 1: expected 3 created ops" -grep -q '(3 created)' "${EV}/run1.out" || fail "run 1: summary must say (3 created)" +[[ "$(grep -c ' created' "${EV}/run1.out")" == "4" ]] || fail "run 1: expected 4 created ops" +grep -q '(4 created)' "${EV}/run1.out" || fail "run 1: summary must say (4 created)" TPID="$(python3 -c "import json,sys,re m=re.search(r'ref -> id: (\{.*\})', open(sys.argv[1]).read()); print(json.loads(m.group(1))['tp'])" "${EV}/run1.out")" INVID="$(python3 -c "import json,sys,re m=re.search(r'ref -> id: (\{.*\})', open(sys.argv[1]).read()); print(json.loads(m.group(1))['inv'])" "${EV}/run1.out")" -TP1="$(tp_count)"; INV1="$(inv_count "${TPID}")"; PAY1="$(pay_count "${INVID}")" -echo "row counts after run 1: thirdparties=${TP1} invoices=${INV1} payments=${PAY1}" | tee "${EV}/counts-run1.txt" -[[ "${TP1}" == "1" && "${INV1}" == "1" && "${PAY1}" == "1" ]] || fail "run 1 must have created exactly 1 of each" +TP1="$(tp_count)"; INV1="$(inv_count "${TPID}")"; PAY1="$(pay_count "${INVID}")"; DOC1="$(doc_count "${INVID}")" +echo "row counts after run 1: thirdparties=${TP1} invoices=${INV1} payments=${PAY1} ged_files=${DOC1}" | tee "${EV}/counts-run1.txt" +[[ "${TP1}" == "1" && "${INV1}" == "1" && "${PAY1}" == "1" && "${DOC1}" == "1" ]] || fail "run 1 must have created exactly 1 of each" -echo; echo "-- RUN 2 (same manifest): expect 3 deduped, zero new rows -----------------" +echo; echo "-- RUN 2 (same manifest): expect 4 deduped, zero new rows -----------------" "${SCRIPTS}/promote-apply.sh" "${EV}/manifest.json" --target sandbox \ | tee "${EV}/run2.out" -[[ "$(grep -c 'deduped=true' "${EV}/run2.out")" == "3" ]] || fail "run 2: all 3 ops must dedupe" +[[ "$(grep -c 'deduped=true' "${EV}/run2.out")" == "4" ]] || fail "run 2: all 4 ops must dedupe" grep -q ' created' "${EV}/run2.out" && fail "run 2: nothing may be created on a replay" -grep -q '(3 deduped)' "${EV}/run2.out" || fail "run 2: summary must say (3 deduped)" +grep -q '(4 deduped)' "${EV}/run2.out" || fail "run 2: summary must say (4 deduped)" -TP2="$(tp_count)"; INV2="$(inv_count "${TPID}")"; PAY2="$(pay_count "${INVID}")" -echo "row counts after run 2: thirdparties=${TP2} invoices=${INV2} payments=${PAY2}" | tee "${EV}/counts-run2.txt" -[[ "${TP2}" == "${TP1}" && "${INV2}" == "${INV1}" && "${PAY2}" == "${PAY1}" ]] \ - || fail "run 2 must add ZERO rows (run1: ${TP1}/${INV1}/${PAY1}, run2: ${TP2}/${INV2}/${PAY2})" +TP2="$(tp_count)"; INV2="$(inv_count "${TPID}")"; PAY2="$(pay_count "${INVID}")"; DOC2="$(doc_count "${INVID}")" +echo "row counts after run 2: thirdparties=${TP2} invoices=${INV2} payments=${PAY2} ged_files=${DOC2}" | tee "${EV}/counts-run2.txt" +[[ "${TP2}" == "${TP1}" && "${INV2}" == "${INV1}" && "${PAY2}" == "${PAY1}" && "${DOC2}" == "${DOC1}" ]] \ + || fail "run 2 must add ZERO rows (run1: ${TP1}/${INV1}/${PAY1}/${DOC1}, run2: ${TP2}/${INV2}/${PAY2}/${DOC2})" # The payment's stored num must be the erp#37 canonical short form. "${W}" GET "/supplierinvoices/${INVID}/payments" > "${EV}/payments.json" @@ -99,5 +113,5 @@ grep -q "\"${TX_SHORT}\"" "${EV}/payments.json" \ || fail "stored num must be the normalized short form ${TX_SHORT}" echo -echo "PASS: replay is a no-op — run 1 created 3 rows (tp=${TPID}, inv=${INVID}), run 2 deduped all 3, row counts unchanged (${TP2}/${INV2}/${PAY2})." -echo "Evidence in ${EV}: manifest.json run1.out run2.out counts-run*.txt payments.json" +echo "PASS: replay is a no-op — run 1 created 4 rows (tp=${TPID}, inv=${INVID}), run 2 deduped all 4, row counts unchanged (${TP2}/${INV2}/${PAY2}/${DOC2})." +echo "Evidence in ${EV}: manifest.json attach-fixture.pdf run1.out run2.out counts-run*.txt payments.json" diff --git a/.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh b/.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh index 519673b..ba436e0 100755 --- a/.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh +++ b/.claude/skills/dolibarr-sandbox-write/tests/run-tests.sh @@ -28,6 +28,12 @@ # an existing fiche missing the requested role ABORTS; a miss creates. # 11. a deduped DRAFT with validate:true is validated on replay (converges an # op that died between create and validate). +# GED attach (erp#43): +# 12. document-attach.sh uploads with overwriteifexists=0 + read-back sha256; +# an identical stored file dedupes (no POST); the same filename with +# DIFFERENT content ABORTS (never an overwrite); unknown fields refused; +# promote-apply resolves a manifest-relative "file" and promote-plan +# prints its sha256. set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" PR="${SCRIPT_DIR}/../scripts/payment-record.sh" @@ -326,4 +332,83 @@ grep -q '/supplierinvoices/5/validate' "${S11}/validated_endpoint" \ || fail "draft-converge: the matched draft must be validated" echo "OK: draft convergence — replay validates the half-done invoice instead of duplicating it" +# --- Case 12: document-attach (erp#43) — upload, sha256 dedupe, conflict abort --- +DA="${SCRIPT_DIR}/../scripts/document-attach.sh" +bash -n "${DA}" || fail "bash -n document-attach.sh" +S12="$(mktemp -d -t datest.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}" "${S10}" "${S11}" "${S12}"' EXIT +printf 'ged43 offline fixture' > "${S12}/src.pdf" +SRC_SHA="$(python3 -c "import hashlib,sys; print(hashlib.sha256(open(sys.argv[1],'rb').read()).hexdigest())" "${S12}/src.pdf")" +SRC_B64="$(python3 -c "import base64,sys; print(base64.b64encode(open(sys.argv[1],'rb').read()).decode())" "${S12}/src.pdf")" +printf '%s' '{"id":"29","ref":"FAF2026013","ref_supplier":"F1045","statut":"1"}' \ + > "${S12}/invoice_detail_29.json" +# 12a — fresh attach: 404 listing → upload (overwriteifexists=0) → read-back sha +OUT="$(printf '{"modulepart":"facture_fournisseur","object_id":29,"file":"%s"}' "${S12}/src.pdf" \ + | DOL_WRITE="${STUB}" STUB_STATE="${S12}" bash "${DA}" 2>/dev/null)" \ + || fail "attach-fresh: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o['deduped'] is False and o['sha256'] == '${SRC_SHA}' and o['ref'] == 'FAF2026013' \ + and o['filename'] == 'src.pdf' and o['object_id'] == 29, o +" || fail "attach-fresh: bad output: ${OUT}" +python3 -c " +import json +b = json.load(open('${S12}/upload_body.json')) +assert b['overwriteifexists'] == '0' and b['fileencoding'] == 'base64' \ + and b['ref'] == 'FAF2026013' and b['filecontent'] == '${SRC_B64}', b +" || fail "attach-fresh: upload body must carry base64 content + overwriteifexists=0" +# 12b — re-attach identical content (staged listing + download, NO upload state) → dedupe +S12B="$(mktemp -d -t datest12b.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}" "${S10}" "${S11}" "${S12}" "${S12B}"' EXIT +cp "${S12}/src.pdf" "${S12B}/src.pdf"; cp "${S12}/invoice_detail_29.json" "${S12B}/" +printf '%s' '[{"name":null,"relativename":"src.pdf","type":"file","level1name":"FAF2026013", + "fullname":"/var/www/documents/fournisseur/facture/0/3/FAF2026013/src.pdf","size":21}]' \ + > "${S12B}/documents.json" +printf '{"filename":"src.pdf","content-type":"application/pdf","filesize":21,"content":"%s"}' "${SRC_B64}" \ + > "${S12B}/document_download.json" +OUT="$(printf '{"modulepart":"facture_fournisseur","object_id":29,"file":"%s"}' "${S12B}/src.pdf" \ + | DOL_WRITE="${STUB}" STUB_STATE="${S12B}" bash "${DA}" 2>/dev/null)" \ + || fail "attach-dedupe: expected success, got $?" +python3 -c " +import json +o = json.loads('''${OUT}''') +assert o['deduped'] is True and o['sha256'] == '${SRC_SHA}', o +" || fail "attach-dedupe: identical content must dedupe, got: ${OUT}" +[[ ! -f "${S12B}/upload_body.json" ]] || fail "attach-dedupe: must NOT upload on a sha256 match" +# 12c — same filename, DIFFERENT content → abort, no upload +OTHER_B64="$(printf 'ged43 DIFFERENT bytes' | python3 -c "import base64,sys; print(base64.b64encode(sys.stdin.buffer.read()).decode())")" +printf '{"filename":"src.pdf","content-type":"application/pdf","filesize":21,"content":"%s"}' "${OTHER_B64}" \ + > "${S12B}/document_download.json" +rc=0 +printf '{"modulepart":"facture_fournisseur","object_id":29,"file":"%s"}' "${S12B}/src.pdf" \ + | DOL_WRITE="${STUB}" STUB_STATE="${S12B}" bash "${DA}" >/dev/null 2>"${S12B}/stderr12c" || rc=$? +[[ "${rc}" -ne 0 ]] || fail "attach-conflict: same name + different content must abort" +grep -q 'DIFFERENT content' "${S12B}/stderr12c" || fail "attach-conflict: error must say DIFFERENT content" +[[ ! -f "${S12B}/upload_body.json" ]] || fail "attach-conflict: must NOT upload on a conflict" +# 12d — unknown field refused before any request +rc=0 +printf '{"modulepart":"facture_fournisseur","object_id":29,"file":"%s","overwrite":true}' "${S12}/src.pdf" \ + | DOL_WRITE="${STUB}" STUB_STATE="${S12}" bash "${DA}" >/dev/null 2>"${S12}/stderr12d" || rc=$? +[[ "${rc}" -ne 0 ]] || fail "attach-unknown-field: must refuse unknown fields" +grep -q 'overwrite' "${S12}/stderr12d" || fail "attach-unknown-field: error must name the offender" +# 12e — promote-apply resolves a manifest-relative file; promote-plan prints its sha +S12E="$(mktemp -d -t datest12e.XXXXXX)"; trap 'rm -rf "${STATE}" "${S4}" "${S5}" "${S6}" "${S7}" "${S8}" "${S9}" "${S10}" "${S11}" "${S12}" "${S12B}" "${S12E}"' EXIT +mkdir -p "${S12E}/pack/pdfs" +cp "${S12}/src.pdf" "${S12E}/pack/pdfs/src.pdf" +printf '%s' '{"id":"29","ref":"FAF2026013","ref_supplier":"F1045","statut":"1"}' \ + > "${S12E}/invoice_detail_29.json" +cat > "${S12E}/pack/manifest.json" <<'JSON' +[ { "op": "attach", + "input": { "modulepart": "facture_fournisseur", "object_id": 29, + "file": "pdfs/src.pdf" } } ] +JSON +( cd "${S12E}" \ + && bash "${PP}" pack/manifest.json > plan.out 2>&1 \ + && DOL_WRITE="${STUB}" STUB_STATE="${S12E}" bash "${PA}" pack/manifest.json --target sandbox > apply.out 2>&1 ) \ + || fail "attach-promote: plan/apply failed: $(cat "${S12E}/plan.out" "${S12E}/apply.out" 2>/dev/null)" +grep -q "sha256=${SRC_SHA}" "${S12E}/plan.out" || fail "attach-promote: plan must print the file sha256" +grep -q 'attach' "${S12E}/apply.out" || fail "attach-promote: apply must run the attach op" +grep -q '(1 created)' "${S12E}/apply.out" || fail "attach-promote: summary must say (1 created)" +[[ -f "${S12E}/upload_body.json" ]] || fail "attach-promote: the manifest-relative file must reach the upload" +echo "OK: document-attach — upload + read-back, sha256 dedupe, conflict abort, field refusal, manifest-relative file" + echo "OK: all offline tests passed" diff --git a/.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh b/.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh index eaa3b48..7a3308d 100755 --- a/.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh +++ b/.claude/skills/dolibarr-sandbox-write/tests/stub-dol-write.sh @@ -28,6 +28,9 @@ set -euo pipefail STATE="${STUB_STATE:?stub-dol-write.sh: STUB_STATE not set}" METHOD="$1"; ENDPOINT="$2"; BODY="${3:-}" +# dol-write.sh accepts @file bodies (document-attach.sh uses one for the base64 +# payload) — dereference it here so the recorded body is the JSON, not "@/path". +[[ "${BODY}" == @* ]] && BODY="$(cat "${BODY:1}")" case "${METHOD} ${ENDPOINT}" in GET\ *"/payments"*) if [[ -f "${STATE}/payments.json" ]]; then @@ -120,6 +123,55 @@ json.dump(rows, open(p, "w"), ensure_ascii=False) PY echo "88" ;; + "GET /documents/download?"*) + # Serve document_download.json when staged; else (post-upload read-back) + # synthesize from upload_body.json; else the live 404. + if [[ -f "${STATE}/document_download.json" ]]; then + cat "${STATE}/document_download.json" + elif [[ -f "${STATE}/upload_body.json" ]]; then + python3 - "${STATE}/upload_body.json" <<'PY' +import base64, json, sys +b = json.load(open(sys.argv[1])) +print(json.dumps({"filename": b.get("filename"), "content-type": "application/pdf", + "filesize": len(base64.b64decode(b.get("filecontent", ""))), + "content": b.get("filecontent", "")})) +PY + else + printf '%s' '{"error":{"code":404,"message":"Not Found"}}' + echo "stub-dol-write.sh: HTTP 404 on GET ${ENDPOINT}" >&2 + exit 1 + fi + ;; + "GET /documents?"*) + # Serve documents.json when staged; else (post-upload read-back) a listing + # built from upload_body.json — with the supplier-invoice get_exdir prefix + # (0/3/REF/…) so the relative-path derivation is exercised; else the live + # Dolibarr behavior: an object with no documents answers HTTP 404, not []. + if [[ -f "${STATE}/documents.json" ]]; then + cat "${STATE}/documents.json" + elif [[ -f "${STATE}/upload_body.json" ]]; then + python3 - "${STATE}/upload_body.json" <<'PY' +import base64, json, sys +b = json.load(open(sys.argv[1])) +ref, fn = b.get("ref", "REF"), b.get("filename", "file.pdf") +print(json.dumps([{"name": None, "relativename": fn, "type": "file", + "level1name": ref, + "fullname": "/var/www/documents/fournisseur/facture/0/3/%s/%s" % (ref, fn), + "size": len(base64.b64decode(b.get("filecontent", "")))}])) +PY + else + printf '%s' '{"error":{"code":404,"message":"Not Found: no document"}}' + echo "stub-dol-write.sh: HTTP 404 on GET ${ENDPOINT}" >&2 + exit 1 + fi + ;; + "POST /documents/upload") + printf '%s' "${BODY}" > "${STATE}/upload_body.json" + python3 - "${STATE}/upload_body.json" <<'PY' +import json, sys +print(json.dumps(json.load(open(sys.argv[1])).get("filename"))) +PY + ;; POST\ *"/validate") printf '%s\n' "${ENDPOINT}" > "${STATE}/validated_endpoint" echo '{"success":1}' diff --git a/.gitignore b/.gitignore index 01f1a48..d1d5f9e 100644 --- a/.gitignore +++ b/.gitignore @@ -8,4 +8,8 @@ secrets/ *.key .claude/skills/**/.env + +# Replay-pack source PDFs — evidence, re-fetchable from books@ (email-ingest), +# never committed (see each pack's README for the re-fetch recipe) +.claude/skills/dolibarr-sandbox-write/replay-packs/**/pdfs/ .claude/skills/**/examples/document_*.bin \ No newline at end of file diff --git a/bin/arcodange b/bin/arcodange index 4cb4e8a..ee8a1bd 100755 --- a/bin/arcodange +++ b/bin/arcodange @@ -87,6 +87,7 @@ COMMANDS invoice Customer/supplier invoice + product/service lines payment Record a règlement (transaction_id=bank tx) → bank_transaction_id creditnote Create an avoir — customer or supplier (kind) + attach Upload a source document onto an invoice (GED; sha256-idempotent) accounts List bank accounts (id/label) to pick account_id write [body] Raw host-guarded write checkpoint status|refresh|provision|relink-env Manage the iso-prod checkpoint @@ -287,6 +288,7 @@ EOF invoice) exec "${SKILLS}/dolibarr-sandbox-write/scripts/invoice-create.sh" "$@" ;; payment) exec "${SKILLS}/dolibarr-sandbox-write/scripts/payment-record.sh" "$@" ;; creditnote) exec "${SKILLS}/dolibarr-sandbox-write/scripts/creditnote-create.sh" "$@" ;; + attach) exec "${SKILLS}/dolibarr-sandbox-write/scripts/document-attach.sh" "$@" ;; accounts) exec "${SKILLS}/dolibarr-sandbox-write/scripts/bank-accounts.sh" "$@" ;; write) exec "${SKILLS}/dolibarr-sandbox-write/scripts/dol-write.sh" "$@" ;; checkpoint) @@ -327,6 +329,8 @@ Each subcommand reads a JSON object on stdin (or a file path arg). echo '{"invoice_id":19,"mode":"VIR","account_id":1}' | arcodange sandbox payment creditnote avoir (credit note) referencing a source invoice echo '{"socid":42,"source_invoice":19,"validate":true,"lines":[...]}' | arcodange sandbox creditnote + attach upload a source document onto an invoice's GED (idempotent by sha256) + echo '{"modulepart":"facture_fournisseur","ref":"FAF2026013","file":"F1045.pdf"}' | arcodange sandbox attach write raw host-guarded write arcodange sandbox write POST /thirdparties '{"name":".."}' checkpoint manage the iso-prod checkpoint (status|refresh|provision|relink-env) arcodange sandbox checkpoint status -- 2.54.0