feat(write-skill): GED attach op — upload the source document onto its invoice (erp#43)
document-attach.sh uploads a source piece (the supplier's own PDF) onto an invoice's GED via POST /documents/upload — idempotent by (object, filename, sha256): before any POST the object's GED is listed and a same-named entry is downloaded back and sha256-compared. Identical → deduped no-op; different content → ABORT (refuse-never-repair, overwriteifexists always 0, never Dolibarr's overwrite flag). Read-back after upload: re-list + download + sha256-verify. Module-relative download paths are derived from the listing's fullname (supplier invoices carry an id-derived get_exdir prefix like 9/2/FAF2026013/…, so reconstruction would be wrong). Promote integration: new `attach` op in promote-plan/promote-apply (OP_SCRIPT), object_id resolvable via @ref and #supplierinvoice lookups; a relative `file` resolves against the manifest's directory (replay packs carry pdfs/ beside the manifest, gitignored — README documents the books@ re-fetch message ids). promote-plan prints each file's sha256 (or a loud MISSING) at review time. CLI: `arcodange sandbox attach`. Proof: offline case 12 in tests/run-tests.sh (upload body, dedupe, conflict abort, field refusal, manifest-relative resolution via stubbed /documents); live: manifest-C-ged-attach.json applied twice on the sandbox — run 1 four created, run 2 four deduped, one GED file per FAF2026010-013, stored sha256s equal to the re-fetched sources; tests/replay-idempotency.sh extended with an attach op (4 created → 4 deduped, ged_files count unchanged) and a live same-name/different-bytes abort verified. Closes erp#43 Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VRShc4QhLLU73FLHx9vskh
This commit is contained in:
Claude Fable 5
@@ -0,0 +1,289 @@
|
||||
#!/usr/bin/env bash
|
||||
# Attach a source document onto a Dolibarr invoice in the SANDBOX (GED upload)
|
||||
# — IDEMPOTENT by (object, filename, sha256): re-attaching the same file is a
|
||||
# no-op, and the same filename with DIFFERENT content ABORTS (a conflict for a
|
||||
# human, never an overwrite). erp#43.
|
||||
#
|
||||
# Input: a JSON object on stdin (or a file path / inline JSON in $1):
|
||||
# modulepart (required) "facture"|"invoice" → customer invoice GED
|
||||
# "facture_fournisseur"|"invoice_supplier"|
|
||||
# "supplier_invoice" → supplier invoice GED
|
||||
# object_id invoice id on the target (one of object_id / ref
|
||||
# ref Dolibarr's own ref, e.g. FAF2026013 required; both given =
|
||||
# cross-checked)
|
||||
# file (required) path to the file to attach. Resolved against the CWD;
|
||||
# promote-apply resolves manifest-relative paths BEFORE invoking.
|
||||
# filename stored name in the GED (default: basename of file)
|
||||
#
|
||||
# Idempotency (erp#43, the erp#44 pattern): BEFORE any POST the object's GED
|
||||
# directory is listed (GET /documents) and a same-named entry — if any — is
|
||||
# DOWNLOADED BACK and sha256-compared against the local file:
|
||||
# - same sha256 → {"deduped": true}, no upload (re-attach = no-op);
|
||||
# - different sha → ABORT. The idempotency check is OUR OWN sha256 compare,
|
||||
# never Dolibarr's overwriteifexists flag (whose semantics vary by version):
|
||||
# silently replacing a stored piece would rewrite evidence — refuse, never
|
||||
# repair. overwriteifexists is always sent as 0.
|
||||
# - a listing/download failure other than "no documents yet" (HTTP 404 on an
|
||||
# object we just proved exists) ABORTS — uploading blind could duplicate.
|
||||
# After an upload the GED is re-listed and the file downloaded back and
|
||||
# sha256-verified: read-back proof the GED holds exactly the bytes we sent.
|
||||
#
|
||||
# Path gotcha: supplier-invoice documents live under an id-derived get_exdir
|
||||
# prefix (e.g. fournisseur/facture/0/3/FAF2026014/…), so /documents/download
|
||||
# needs "0/3/REF/file.pdf", not "REF/file.pdf". The module-relative path is
|
||||
# derived from the listing's fullname (substring after the last "/facture/").
|
||||
#
|
||||
# Emits {"object_id", "ref", "modulepart", "filename", "sha256", "size",
|
||||
# "deduped"} on stdout. All requests go through dol-write.sh (or $DOL_WRITE),
|
||||
# which refuses any host that is not the sandbox.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
W="${DOL_WRITE:-${SCRIPT_DIR}/dol-write.sh}"
|
||||
|
||||
SRC="${1:-}"
|
||||
if [[ -z "${SRC}" || "${SRC}" == "-" ]]; then INPUT="$(cat)"
|
||||
elif [[ -f "${SRC}" ]]; then INPUT="$(cat "${SRC}")"
|
||||
else INPUT="${SRC}"; fi # inline JSON
|
||||
|
||||
TMPD="$(mktemp -d -t docatt.XXXXXX)"; trap 'rm -rf "${TMPD}"' EXIT
|
||||
|
||||
# --- 1. Validate + normalize the input (before ANY request) --------------------
|
||||
cat > "${TMPD}/validate.py" <<'PY'
|
||||
import json, os, sys
|
||||
ALLOWED = ["modulepart", "object_id", "ref", "file", "filename"]
|
||||
MP = {"facture": "facture", "invoice": "facture",
|
||||
"facture_fournisseur": "facture_fournisseur",
|
||||
"invoice_supplier": "facture_fournisseur",
|
||||
"supplier_invoice": "facture_fournisseur"}
|
||||
try:
|
||||
d = json.loads(sys.stdin.read() or "null")
|
||||
except Exception as e:
|
||||
sys.exit("document-attach.sh: input is not valid JSON: %s" % e)
|
||||
if not isinstance(d, dict):
|
||||
sys.exit("document-attach.sh: input must be a JSON object")
|
||||
bad = sorted(k for k in d if k not in ALLOWED)
|
||||
if bad:
|
||||
sys.exit("document-attach.sh: REFUSED — unknown field(s): %s\n allowed: %s"
|
||||
% (", ".join(bad), ", ".join(ALLOWED)))
|
||||
mp_raw = str(d.get("modulepart", "")).strip().lower()
|
||||
if mp_raw not in MP:
|
||||
sys.exit("document-attach.sh: 'modulepart' must be one of %s (got %r) — this "
|
||||
"op attaches onto customer or supplier invoices only"
|
||||
% ("/".join(sorted(set(MP))), mp_raw))
|
||||
mp = MP[mp_raw]
|
||||
oid = d.get("object_id")
|
||||
ref = str(d.get("ref") or "").strip()
|
||||
if oid in (None, "") and not ref:
|
||||
sys.exit("document-attach.sh: one of 'object_id' / 'ref' is required")
|
||||
if oid not in (None, "") and not str(oid).isdigit():
|
||||
sys.exit("document-attach.sh: 'object_id' must be numeric, got %r" % oid)
|
||||
path = str(d.get("file") or "")
|
||||
if not path:
|
||||
sys.exit("document-attach.sh: 'file' is required")
|
||||
if not os.path.isfile(path):
|
||||
sys.exit("document-attach.sh: file not found: %s\n (paths resolve against "
|
||||
"the CWD; promote-apply resolves manifest-relative paths first)" % path)
|
||||
filename = str(d.get("filename") or "").strip() or os.path.basename(path)
|
||||
if "/" in filename or filename in (".", ".."):
|
||||
sys.exit("document-attach.sh: 'filename' must be a bare file name, got %r" % filename)
|
||||
print(mp)
|
||||
print("/invoices" if mp == "facture" else "/supplierinvoices")
|
||||
print(str(oid) if oid not in (None, "") else "")
|
||||
print(ref)
|
||||
print(path)
|
||||
print(filename)
|
||||
PY
|
||||
MAPPED="$(printf '%s' "${INPUT}" | python3 "${TMPD}/validate.py")"
|
||||
MP="$(sed -n 1p <<<"${MAPPED}")"
|
||||
ENDPOINT="$(sed -n 2p <<<"${MAPPED}")"
|
||||
OID="$(sed -n 3p <<<"${MAPPED}")"
|
||||
REF_IN="$(sed -n 4p <<<"${MAPPED}")"
|
||||
FILE="$(sed -n 5p <<<"${MAPPED}")"
|
||||
FILENAME="$(sed -n 6p <<<"${MAPPED}")"
|
||||
|
||||
# Local sha256 + base64 (binary-safe, portable — no macOS/Linux base64 flag
|
||||
# drift; the base64 goes to a file, never through argv, so size doesn't matter).
|
||||
LOCAL_SHA="$(python3 - "${FILE}" "${TMPD}/b64.txt" <<'PY'
|
||||
import base64, hashlib, sys
|
||||
data = open(sys.argv[1], "rb").read()
|
||||
open(sys.argv[2], "w").write(base64.b64encode(data).decode())
|
||||
print(hashlib.sha256(data).hexdigest())
|
||||
PY
|
||||
)"
|
||||
|
||||
urlenc() { python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=''))" "$1"; }
|
||||
|
||||
# --- 2. Resolve the object on the target (id ⇄ ref) ----------------------------
|
||||
if [[ -n "${OID}" ]]; then
|
||||
if ! "${W}" GET "${ENDPOINT}/${OID}" > "${TMPD}/obj.json" 2> "${TMPD}/obj.err"; then
|
||||
cat "${TMPD}/obj.err" >&2
|
||||
echo "document-attach.sh: object ${ENDPOINT}/${OID} not readable on the target — refusing to upload blind" >&2
|
||||
exit 1
|
||||
fi
|
||||
REF="$(python3 -c "import json,sys; print(json.load(sys.stdin).get('ref') or '')" < "${TMPD}/obj.json")"
|
||||
if [[ -z "${REF}" ]]; then
|
||||
echo "document-attach.sh: object ${ENDPOINT}/${OID} has no ref — cannot address its GED directory" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ -n "${REF_IN}" && "${REF_IN}" != "${REF}" ]]; then
|
||||
echo "document-attach.sh: ABORT — object_id ${OID} is '${REF}' on the target, not the requested ref '${REF_IN}'" >&2
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
# ref-only: look the object up by its Dolibarr ref — 0 or 2+ matches abort.
|
||||
FLT="$(urlenc "(t.ref:=:'${REF_IN}')")"
|
||||
set +e
|
||||
"${W}" GET "${ENDPOINT}?limit=2&sqlfilters=${FLT}" > "${TMPD}/lookup.json" 2> "${TMPD}/lookup.err"
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ ${rc} -ne 0 ]]; then
|
||||
if grep -q "HTTP 404" "${TMPD}/lookup.err"; then printf '[]' > "${TMPD}/lookup.json"
|
||||
else
|
||||
cat "${TMPD}/lookup.err" >&2
|
||||
echo "document-attach.sh: could not look up ref '${REF_IN}' on ${ENDPOINT} — refusing to upload blind" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
OID="$(python3 - "${TMPD}/lookup.json" "${REF_IN}" <<'PY'
|
||||
import json, sys
|
||||
try:
|
||||
rows = json.load(open(sys.argv[1]))
|
||||
except Exception:
|
||||
rows = []
|
||||
rows = rows if isinstance(rows, list) else []
|
||||
if len(rows) == 0:
|
||||
sys.exit("document-attach.sh: ref %r matched nothing on the target" % sys.argv[2])
|
||||
if len(rows) > 1:
|
||||
sys.exit("document-attach.sh: ref %r is ambiguous (%d matches) — pass object_id" % (sys.argv[2], len(rows)))
|
||||
print(int(rows[0]["id"]))
|
||||
PY
|
||||
)"
|
||||
REF="${REF_IN}"
|
||||
fi
|
||||
|
||||
# --- 3. List the object's GED directory (the dedupe source of truth) -----------
|
||||
list_documents() { # $1 = output file; 404 on a proven-existing object = "no docs yet"
|
||||
set +e
|
||||
"${W}" GET "/documents?modulepart=${MP}&id=${OID}" > "$1" 2> "${TMPD}/list.err"
|
||||
local rc=$?
|
||||
set -e
|
||||
if [[ ${rc} -ne 0 ]]; then
|
||||
if grep -q "HTTP 404" "${TMPD}/list.err"; then
|
||||
printf '[]' > "$1"
|
||||
else
|
||||
cat "${TMPD}/list.err" >&2
|
||||
echo "document-attach.sh: could not list documents for ${MP} id ${OID} — refusing to upload blind (dedupe impossible)" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
list_documents "${TMPD}/list.json"
|
||||
|
||||
# Find a same-named entry and derive its module-relative download path from
|
||||
# fullname (everything after the last "/facture/" — covers both moduleparts,
|
||||
# including the supplier-invoice get_exdir prefix like 0/3/REF/…).
|
||||
cat > "${TMPD}/match.py" <<'PY'
|
||||
import json, os, sys
|
||||
try:
|
||||
rows = json.load(open(sys.argv[1]))
|
||||
except Exception:
|
||||
rows = []
|
||||
rows = rows if isinstance(rows, list) else []
|
||||
fn = sys.argv[2]
|
||||
hits = []
|
||||
for r in rows:
|
||||
if str(r.get("type") or "") == "dir":
|
||||
continue
|
||||
name = str(r.get("relativename") or "").strip() \
|
||||
or os.path.basename(str(r.get("fullname") or ""))
|
||||
if name == fn:
|
||||
hits.append(r)
|
||||
if len(hits) > 1:
|
||||
sys.exit("document-attach.sh: ABORT — %d GED entries named %r on this object; "
|
||||
"refusing to guess" % (len(hits), fn))
|
||||
if hits:
|
||||
full = str(hits[0].get("fullname") or "")
|
||||
i = full.rfind("/facture/")
|
||||
if i < 0:
|
||||
sys.exit("document-attach.sh: ABORT — cannot derive the module-relative "
|
||||
"path from %r" % full)
|
||||
print(json.dumps({"relpath": full[i + len("/facture/"):],
|
||||
"size": hits[0].get("size")}))
|
||||
PY
|
||||
|
||||
remote_sha() { # $1 = module-relative path → prints sha256 of the stored bytes
|
||||
local enc dl
|
||||
enc="$(urlenc "$1")"
|
||||
if ! dl="$("${W}" GET "/documents/download?modulepart=${MP}&original_file=${enc}" 2> "${TMPD}/dl.err")"; then
|
||||
cat "${TMPD}/dl.err" >&2
|
||||
echo "document-attach.sh: could not download back '$1' to verify its sha256 — refusing to act blind" >&2
|
||||
exit 1
|
||||
fi
|
||||
python3 -c "
|
||||
import base64, hashlib, json, sys
|
||||
d = json.load(sys.stdin)
|
||||
print(hashlib.sha256(base64.b64decode(d['content'])).hexdigest())" <<<"${dl}"
|
||||
}
|
||||
|
||||
MATCH="$(python3 "${TMPD}/match.py" "${TMPD}/list.json" "${FILENAME}")"
|
||||
if [[ -n "${MATCH}" ]]; then
|
||||
RELPATH="$(python3 -c "import json,sys; print(json.load(sys.stdin)['relpath'])" <<<"${MATCH}")"
|
||||
SIZE="$(python3 -c "import json,sys; print(json.load(sys.stdin)['size'])" <<<"${MATCH}")"
|
||||
STORED_SHA="$(remote_sha "${RELPATH}")"
|
||||
if [[ "${STORED_SHA}" == "${LOCAL_SHA}" ]]; then
|
||||
echo "document-attach.sh: '${FILENAME}' already attached to ${REF} with identical sha256 — deduped, no upload" >&2
|
||||
python3 -c "
|
||||
import json, sys
|
||||
print(json.dumps({'object_id': int(sys.argv[1]), 'ref': sys.argv[2],
|
||||
'modulepart': sys.argv[3], 'filename': sys.argv[4],
|
||||
'sha256': sys.argv[5], 'size': int(sys.argv[6]),
|
||||
'deduped': True}))" "${OID}" "${REF}" "${MP}" "${FILENAME}" "${LOCAL_SHA}" "${SIZE}"
|
||||
exit 0
|
||||
fi
|
||||
echo "document-attach.sh: ABORT — '${FILENAME}' already exists on ${REF} with DIFFERENT content" >&2
|
||||
echo " stored sha256: ${STORED_SHA}" >&2
|
||||
echo " local sha256: ${LOCAL_SHA}" >&2
|
||||
echo " Same name + different bytes is a conflict for a human to resolve — never an overwrite." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- 4. Upload (no same-named entry) -------------------------------------------
|
||||
python3 - "${FILENAME}" "${MP}" "${REF}" "${TMPD}/b64.txt" > "${TMPD}/upload.json" <<'PY'
|
||||
import json, sys
|
||||
print(json.dumps({"filename": sys.argv[1], "modulepart": sys.argv[2],
|
||||
"ref": sys.argv[3], "filecontent": open(sys.argv[4]).read().strip(),
|
||||
"fileencoding": "base64", "overwriteifexists": "0"}))
|
||||
PY
|
||||
|
||||
RESP="$("${W}" POST /documents/upload @"${TMPD}/upload.json")"
|
||||
python3 - "${RESP}" "${FILENAME}" <<'PY'
|
||||
import json, sys
|
||||
try:
|
||||
got = json.loads(sys.argv[1])
|
||||
except Exception:
|
||||
got = None
|
||||
if got != sys.argv[2]:
|
||||
sys.exit("document-attach.sh: upload did not confirm the filename "
|
||||
"(expected %r, got %s)" % (sys.argv[2], sys.argv[1]))
|
||||
PY
|
||||
|
||||
# --- 5. Read-back: re-list + download + sha256 compare -------------------------
|
||||
list_documents "${TMPD}/list2.json"
|
||||
MATCH2="$(python3 "${TMPD}/match.py" "${TMPD}/list2.json" "${FILENAME}")"
|
||||
if [[ -z "${MATCH2}" ]]; then
|
||||
echo "document-attach.sh: read-back FAILED — '${FILENAME}' not listed on ${REF} after upload" >&2
|
||||
exit 1
|
||||
fi
|
||||
RELPATH2="$(python3 -c "import json,sys; print(json.load(sys.stdin)['relpath'])" <<<"${MATCH2}")"
|
||||
SIZE2="$(python3 -c "import json,sys; print(json.load(sys.stdin)['size'])" <<<"${MATCH2}")"
|
||||
STORED_SHA2="$(remote_sha "${RELPATH2}")"
|
||||
if [[ "${STORED_SHA2}" != "${LOCAL_SHA}" ]]; then
|
||||
echo "document-attach.sh: read-back FAILED — stored sha256 ${STORED_SHA2} != local ${LOCAL_SHA}" >&2
|
||||
exit 1
|
||||
fi
|
||||
python3 -c "
|
||||
import json, sys
|
||||
print(json.dumps({'object_id': int(sys.argv[1]), 'ref': sys.argv[2],
|
||||
'modulepart': sys.argv[3], 'filename': sys.argv[4],
|
||||
'sha256': sys.argv[5], 'size': int(sys.argv[6]),
|
||||
'deduped': False}))" "${OID}" "${REF}" "${MP}" "${FILENAME}" "${LOCAL_SHA}" "${SIZE2}"
|
||||
@@ -38,7 +38,8 @@ manifest_path, script_dir = sys.argv[1], sys.argv[2]
|
||||
ops = json.load(open(manifest_path))
|
||||
OP_SCRIPT = {"thirdparty": "thirdparty-create.sh", "invoice": "invoice-create.sh",
|
||||
"creditnote": "creditnote-create.sh", "payment": "payment-record.sh",
|
||||
"thirdparty_update": "thirdparty-update.sh", "contact": "contact-create.sh"}
|
||||
"thirdparty_update": "thirdparty-update.sh", "contact": "contact-create.sh",
|
||||
"attach": "document-attach.sh"}
|
||||
refmap = {}
|
||||
n_created = n_deduped = 0
|
||||
|
||||
@@ -100,6 +101,11 @@ for i, op in enumerate(ops, 1):
|
||||
if not script:
|
||||
sys.exit("promote-apply: unknown op '%s'" % t)
|
||||
inp = resolve(op.get("input", {}))
|
||||
# attach: a relative "file" is relative to the MANIFEST, not the CWD — the
|
||||
# manifest is the portable unit (replay packs carry their pdfs/ beside it).
|
||||
if t == "attach" and isinstance(inp.get("file"), str) and not os.path.isabs(inp["file"]):
|
||||
inp["file"] = os.path.normpath(os.path.join(
|
||||
os.path.dirname(os.path.abspath(manifest_path)), inp["file"]))
|
||||
r = subprocess.run([os.path.join(script_dir, script)], input=json.dumps(inp),
|
||||
capture_output=True, text=True, env=os.environ)
|
||||
if r.returncode != 0:
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
set -euo pipefail
|
||||
MANIFEST="${1:?usage: promote-plan.sh <manifest.json>}"
|
||||
python3 - "$MANIFEST" <<'PY'
|
||||
import json, sys
|
||||
import hashlib, json, os, sys
|
||||
ops = json.load(open(sys.argv[1]))
|
||||
print("Promote plan — %d operation(s) (symbolic refs resolve at apply time):\n" % len(ops))
|
||||
for i, op in enumerate(ops, 1):
|
||||
@@ -46,6 +46,23 @@ for i, op in enumerate(ops, 1):
|
||||
flds = inp.get("fields") or {}
|
||||
print(" socid=%s update %d dossier field(s): %s" % (inp.get("socid"), len(flds),
|
||||
", ".join(sorted(flds)) if flds else "NONE (will be refused)"))
|
||||
elif t == "attach":
|
||||
obj = inp.get("object_id") or inp.get("ref")
|
||||
f = inp.get("file")
|
||||
# A relative file resolves against the MANIFEST (same rule as apply).
|
||||
resolved = None
|
||||
if isinstance(f, str):
|
||||
resolved = f if os.path.isabs(f) else os.path.normpath(
|
||||
os.path.join(os.path.dirname(os.path.abspath(sys.argv[1])), f))
|
||||
print(" object=%s modulepart=%s file=%s" % (obj, inp.get("modulepart"), f))
|
||||
if resolved and os.path.isfile(resolved):
|
||||
data = open(resolved, "rb").read()
|
||||
print(" sha256=%s (%d bytes) (idempotent: dedupe by object+filename+sha256; "
|
||||
"same name + different content aborts)"
|
||||
% (hashlib.sha256(data).hexdigest(), len(data)))
|
||||
else:
|
||||
print(" !! FILE MISSING at plan time: %s — apply WILL fail; "
|
||||
"re-fetch the source first" % resolved)
|
||||
elif t == "contact":
|
||||
name = " ".join(x for x in (inp.get("firstname"), inp.get("lastname")) if x) or "?"
|
||||
print(" socid=%s contact %s%s%s (idempotent: dedupe by email, then lastname+firstname)"
|
||||
|
||||
Reference in New Issue
Block a user