Builds on the dedicated backup (erp#31).
Skip-if-unchanged: each half (DB / documents) carries a content fingerprint at
erp/<env>/.fp-{db,docs} and is dumped+uploaded only if it differs from the last
run — a quiet ERP day re-uploads nothing. Fingerprint = durable BUSINESS content
only: DB = count+max(tms) over tms tables EXCEPT volatile churn (llx_const,
llx_user, session/cron); docs EXCLUDE */temp/* (Dolibarr stats cache) — from both
the fingerprint and the tar. Proven live: 1st run uploads both, immediate 2nd run
skips both (uploaded=0).
Automation: the in-container logic moves to chart/files/backup-job.sh (single
source of truth, read by the orchestrator AND the chart). New
chart/templates/backup-cronjob.yaml renders a daily CronJob + ConfigMap +
VaultStaticSecret, gated by backup.enabled (default false). Helm-verified: off by
default (0 CronJobs), on renders correctly, env-aware (PREFIX erp/prod vs
erp/sandbox), script embedded.
Activation (documented): store GCS HMAC creds at kvv2/<backup.vaultS3Path>
(default erp/backup), grant the erp `auth` Vault role read on it (tools change),
set backup.enabled=true. Until then the orchestrator runs on demand.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The accounting data + issued documents are legally retained 10 years and warrant a
backup dedicated to Dolibarr. An audit found the generic Longhorn external backup
NEVER covered the erp volume (its Longhorn volume sits in the orphaned `default`
recurring-job group; the only job has groups=[] → serves nothing; lastBackupAt=never).
So /var/www/documents (invoice PDFs, supplier pieces, contracts, ECM) had zero
offsite copy — only in-cluster replicas.
ops/backup/dolibarr-backup.sh (orchestrator) + ops/backup/backup-job.sh (in-container
logic, env-driven, single source of truth):
- pg_dump -Fc of the DB + tar of the documents PVC (RWX, read-only mount) ->
s3://arcodange-backup/erp/<env>/{db,docs}/<ts>, then tiered prune (daily 30d /
monthly 12m / yearly 10y).
- prod is READ-only (dump+tar read; writes go only to the backup bucket); the DB is
read with the env's own dynamic creds; the GCS HMAC secret is copied transiently
(base64, deleted on exit) and never printed; the whole script ships base64.
- fixes the aws-cli v2.23+ default-checksum incompatibility with GCS/S3-compat
(SignatureDoesNotMatch) via AWS_*_CHECKSUM_*=when_required.
Proven live: sandbox end-to-end (dump+tar+upload+prune, verified in GCS, cleaned up)
and retention logic unit-tested (1100 daily -> 46 kept). The FIRST real prod backup
was taken (erp/prod/db 1.2 MB + erp/prod/docs 12.5 MB) — closing the gap now.
Automation (recurring CronJob in the chart + a dedicated erp Vault policy for its
own S3 creds) is the documented next step; the orchestrator works today on demand.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A skill + CLI group to drive the ADR-0003 sandbox lifecycle, instead of the manual
kubectl/deno/.env dance:
arcodange sandbox checkpoint status # liveness + is the write agent armed?
arcodange sandbox checkpoint refresh --yes # re-seed iso-prod (DESTRUCTIVE, gated)
arcodange sandbox checkpoint provision # re-create ai_agent_sandbox (Playwright) + relink
arcodange sandbox checkpoint relink-env # rewrite write skill .env from the key + verify
- refresh wraps ops/sandbox/sandbox-lifecycle.sh; requires --yes (it wipes the agent
too, since iso-prod overwrites llx_user). --db-only skips the documents sync.
- provision runs test/provisionSandbox.ts (you do the admin login — PROD creds,
iso-prod) then auto-relinks; relink-env writes .env mode 600 and verifies via
GET /users/info.
- scripts resolve the repo root from ARCO_ROOT (set by bin/arcodange) or their own
path, so they work via the CLI or standalone.
Tested: status reports armed/not-armed correctly; refresh refuses without --yes
(exit 3); relink-env errors with no key (exit 1); help/usage wired.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
refresh-from-prod was structurally broken and silently no-op'd the restore:
1. pg_restore lacked -U, so the postgres image connected as its OS user `root`
and auth-failed. The failure was swallowed by `|| echo "ignorable warnings"`,
so the script reported success while the DROP OWNED had already emptied the DB.
E2's original seed was a manual process, so this path had never really run.
Fix: pass `-h $PGHOST -U $SB_PGUSER`; don't trust pg_restore's exit code (it
returns non-zero on the harmless "schema public already exists" notice) — verify
by counting restored llx_* tables and FAIL the Job if < 250.
2. erp-sandbox is ArgoCD-managed with self-heal ON, which reverts the
`kubectl scale --replicas=0` within seconds — so the seed ran with Dolibarr
still connected. Fix: pause self-heal for the duration, re-arm it after; app
restore + self-heal restoration + secret cleanup are guarded by an EXIT trap so
an interrupt can't strand the sandbox at replicas=0 / self-heal off.
Validated end-to-end on the live sandbox: 295 llx tables, company=Arcodange,
owner=erp_sandbox_role, self-heal re-armed, pod 1/1. README documents the self-heal
pause and the iso-prod consequence (ai_agent_sandbox is wiped → re-provision).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The consumer side of erp#26/#27: now that a règlement stores its originating bank
transaction id (transaction_id -> llx_bank.num_chq), bank-match uses it.
New PASS 0 (exact), highest priority, before wire-ref and amt+date:
- carry each feed movement's own id (Qonto transaction id; Wise activity + transfer
resource id) as feed_ids, and each Dolibarr payment's num.
- match when a payment's num equals a feed id. Tagged [tx-id].
- DATE-WINDOW-INDEPENDENT — the id is proof, so it pairs movements whose bank
settlement and Dolibarr saisie are weeks apart (which amt+date would miss).
Pass 0 runs before the ref index is built, so its matches are excluded from the
later passes (no double-match).
Fixture-proven: a payment dated 15d off the bank movement (outside the ±7d window)
matches via [tx-id] when num carries the Qonto id, and correctly does NOT match
when num is empty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Make the originating bank transaction id a first-class input on payment-record.sh
so every règlement is tied to the real bank movement at write time.
- `transaction_id` is the canonical field (the Qonto/Wise feed tx id); `num` stays
as a back-compat alias. It's stored on the payment's bank line (llx_bank.num_chq),
the reconciliation key.
- Recording WITHOUT a transaction_id prints a stderr warning (still posts, but won't
auto-reconcile) — nudges the agent to always carry it.
- Output normalises to {id, bank_transaction_id, transaction_id}.
- Promote: manifests' payment ops carry transaction_id; promote-plan shows it
(tx=… or tx=MISSING).
Proven live: customer + supplier record with transaction_id; the `num` alias maps
to the same field; the no-tx warning fires; promote plan/apply carry it through.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A payment only returned its paiement id, which isn't what bank reconciliation
keys on. payment-record.sh now emits {id, bank_transaction_id, num}:
- bank_transaction_id = the Dolibarr bank line (llx_bank.fk_bank_line) the payment
created, resolved via GET /{invoices|supplierinvoices}/{id}/payments (correlated
by num, else the most recent line). Works for customer and supplier.
- num stores the originating bank tx id (Qonto/Wise) and lands on that bank line's
num_chq — so arcodange-bank-reco can match a règlement to a statement line by id
instead of fuzzy amount/date. Both ends captured at write time.
Proven live: customer {id:13,bank_transaction_id:35,num:QONTO-TX-1234},
supplier {id:16,bank_transaction_id:36,num:WISE-TX-5678}; llx_bank rows 35/36
carry the refs in num_chq. promote-apply still extracts .id unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two V9 follow-ups, both proven live on the sandbox.
- creditnote-create.sh: `kind:"supplier"` makes an avoir fournisseur on
/supplierinvoices (type=2 + fk_facture_source, carries ref_supplier); default
customer path unchanged. Proven: customer AVC002 (-240) + supplier AVF2026001
(-144, ref_supplier carried, linked to source, validated).
- bank-accounts.sh + `arcodange sandbox accounts`: list bank accounts (id/label/
bank) so a payment can pick its account_id. Needs `banque lire` (rights 111),
now added to the provisioner's WRITE_IDS so fresh runs include it; the existing
ai_agent_sandbox user was granted it live. GET /bankaccounts now returns the 3
accounts (QONTO, WISE EURO, Compte Courant Asso).
- SKILL.md: supplier-avoir example + accounts helper + updated banque-lire note.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the last promote gap: a manifest can now reference records it does NOT
create. A value like "#thirdparty:name=KissMetrics" (or :code=CL0007) is looked
up on the TARGET at apply time and resolved to that target's id — so the same
manifest is portable (sandbox id on --target sandbox, prod id on --target prod).
promote-apply.sh: resolve() gains a "#" branch + a lookup() helper that queries
the target via the GET wrapper with sqlfilters. Supports thirdparty
(name/code/supplier_code) and invoice/supplierinvoice (ref/ref_supplier). A
lookup matching nothing OR more than one record ABORTS the run — it never
guesses, so it cannot write to the wrong entity.
Proven live: "#thirdparty:name=ACME Conseil" resolved to the existing client and
invoiced it; a not-found code and an ambiguous (2-match) name both aborted with
exit 1. Combined with @refs, arbitrary self-contained-or-referential change-sets
now replay cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The human-gated path that carries a reviewed sandbox change to prod.
- promote-plan.sh: render a manifest (JSON array of write ops with symbolic @refs
instead of ids — portable sandbox->prod) as a human-readable change-set.
- promote-apply.sh <manifest> --target sandbox|prod: replay it, resolving each
@ref to the id actually created during the run (dependent ops wire up). sandbox
rehearses via dol-write.sh; prod via dol-prod-write.sh.
- dol-prod-write.sh: the ONLY prod-write path. Prod key read from the ENVIRONMENT
only (DOLIBARR_PROD_WRITE_KEY, never a stored .env); every write refused unless
ARCO_PROMOTE_CONFIRM=I-UNDERSTAND-THIS-WRITES-PROD.
- create scripts take a DOL_WRITE override so promote-apply reuses them per target.
- bin/arcodange: `promote {plan|apply}` group + example manifest.
- payment-record.sh: fixed supplier payments (payment_mode_id + closepaidinvoices).
Proven live: plan renders; apply --target sandbox replays a 3-op chain with refs
resolved (@tp1->id, invoice socid=@tp1, payment invoice=@inv1); --target prod
without the confirm flag is REFUSED before sending. Supplier payment now works
end-to-end via the script.
Limitation (documented): manifests reference entities they create (@ref);
pre-existing prod entities need business-key resolution (follow-up).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- dolibarr-sandbox-write/scripts/creditnote-create.sh: create a customer avoir
(credit note) — a customer invoice type=2 referencing source_invoice
(fk_facture_source); amounts negative, validates to an AVC… ref. Proven live.
- bin/arcodange: new `sandbox` command group wiring the write scripts —
`arcodange sandbox {thirdparty|invoice|payment|creditnote|write}` (JSON on
stdin). Header + usage updated to note the CLI now does host-guarded sandbox
writes (still read-only on prod).
- SKILL.md: avoir workflow + CLI notes.
Verified end-to-end through the CLI: thirdparty -> invoice (FAC…) -> avoir
(AVC…, total_ttc -240, fk_facture_source set); host-guard intact via the CLI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The write-capable companion to the read-only dolibarr* skills, scoped to the
erp-sandbox. Lets an AI agent rehearse bookkeeping writes against a copy of prod
(ADR-0003) before a human promotes the reviewed change to prod.
- scripts/dol-write.sh: write wrapper that REFUSES any host that is not
erp-sandbox.arcodange.lab (the structural prod-safety guarantee) using the
ai_agent_sandbox key from a gitignored .env.
- scripts/thirdparty-create.sh: create client/supplier fiches; codes auto-assign
via the elephant mask (code="-1").
- scripts/invoice-create.sh: customer (/invoices) or supplier (/supplierinvoices)
invoices with product/service lines + ref_supplier, optional validate.
- scripts/payment-record.sh: record a règlement (VIR/CB/CHQ/LIQ); customer pays
full + marks paid, supplier needs an amount.
- SKILL.md (safety model + workflows + the human-gated promote flow), .env.example,
example input.
Proven end-to-end live against the sandbox: client -> invoice (service+product
lines, HT 1100 / TTC 1320) -> validate -> payment (paid); supplier -> supplier
invoice (ref_supplier carried) -> validate. Host guard verified to refuse a prod
URL before sending.
Avoirs (credit notes) and bin/arcodange CLI wiring are planned follow-ups.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Validating ai_agent_sandbox's key against the sandbox API, /thirdparties
returned 404 (the voir_tous ACL trap) while /invoices, /products,
/supplierinvoices returned 200. The missing right is `societe client voir`
(id 262, "see all thirdparties") — prod's ai_agent has it. Added it to
WRITE_IDS so the list endpoint works; other modules' lists are fine with plain
`lire`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First real run against the sandbox revealed three issues in userSetup.ts:
1. generateApiKey generated the key client-side and read it into the file but
never submitted the edit form, so Dolibarr never persisted api_key (DB stayed
NULL → the key could not authenticate). Now it clicks Save after generating.
2. assignRights matched `rights=<id>` as an href substring, so a short id like
12 (facture creer) also matched rights=121 / rights=1232 and .first() clicked
the wrong link — facture creer was never granted. Anchored with a trailing
"&" (rights=<id>&) for an exact match.
3. createUser was not idempotent: a re-run hit the existing login and failed to
parse a new id. Added findUserId (look up by login via the user list) and
return the existing id instead of creating a duplicate.
Verified the symptoms live: ai_agent_sandbox (rowid 4) had api_key NULL and was
missing only facture/creer among the 11 intended rights.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After seeding erp-sandbox from prod, the home dashboard rendered a generic
"technical error" banner per box: prod mode ($dolibarr_main_prod=1, the image
default via DOLI_PROD) escalates the seed's minor non-fatal warnings into that
banner. Setting DOLI_PROD=0 for non-prod environments makes Dolibarr render
real errors inline (correct for a rehearsal env) and clears the banners.
config.yaml adds `DOLI_PROD: "0"` only when env != prod, so the prod configmap
is byte-identical (prod keeps the image default DOLI_PROD=1) — verified via
helm template diff. ArgoCD rolls only the sandbox pod.
Also corrects the test/README install.lock path: Dolibarr checks the DATA root
(/var/www/documents, a PVC — persists across restarts), not /var/www/html. And
notes that a prod-seeded sandbox still needs install.lock created (the seed +
documents/mycompany sync don't include it).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
provisionSandbox.ts now loads its own .env.sandbox (via @std/dotenv loadSync)
instead of the shared .env, so prod (main.ts → .env) and sandbox
(provisionSandbox.ts → .env.sandbox) configs don't collide. .gitignore widened
to .env* (keeping .env.example tracked). .env.example rewritten to document the
two-file convention + the per-env kubectl secret sources, including the caveat
that a prod-seeded sandbox uses PROD's admin password.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-existing (untracked) test/README documented creating Dolibarr's
install.lock after a fresh install — a non-obvious operational step missing from
the rewritten README. Preserve it (generalized to the per-env namespace/label,
with a note that a prod-seeded instance doesn't need it).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Productionizes the sandbox state-lifecycle mechanisms validated live against
erp-sandbox. `ops/sandbox/sandbox-lifecycle.sh`:
- refresh-from-prod: read-only pg_dump of prod erp (default_transaction_read_only)
-> DROP OWNED BY erp_sandbox_role CASCADE -> pg_restore into erp-sandbox, using
the sandbox's own membership creds (no DROP/CREATE DATABASE, no CREATEDB, no
superuser). Dumps the full public schema (so app helper functions + triggers
come over) and filters the provisioner-owned pgbouncer user_lookup function
from the restore TOC. Scales the pod to 0 for exclusive access; copies prod
creds into a transient secret that is deleted on exit.
- sync-documents: tar-pipe the documents/mycompany tree (company logo + uploads)
prod -> sandbox, since uploaded files live on the PVC, not the DB.
Prod integrity is structural: prod is read-only during dump; the restore can only
write erp-sandbox (erp_sandbox_role owns only the sandbox DB and cannot drop prod
erp/erp_role); the platform's only prod-capable superuser stays behind the
human-gated postgres.yaml CI and is never used here.
README documents the integrity guarantee, the encryption + PVC fidelity caveats,
the BDD reset loop, and the hardening backlog (dedicated read-only dump role,
golden-cache PVC).
Refs ADR-0003 (factory#19). Chart owner-role fix = erp#13.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extend the Deno + Playwright UI-automation POC to provision the erp-sandbox
Dolibarr for the AI agent:
- moduleSetup.ts: add enableApiModule(ctx) — toggles the REST API / Web services
module on /admin/modules.php (kanban). Resilient: tries the fr_FR card label
"API/Web services REST (serveur)" first, falls back to a /API.*REST|REST.*API/i
title match if the exact label is absent.
- userSetup.ts (new): createUser (returns the new numeric id), assignRights
(clicks each addrights link on /user/perms.php, idempotent), generateApiKey
(triggers Dolibarr's generate control on the user card and reads the value back).
- provisionSandbox.ts (new entrypoint, main.ts untouched): login → enable API →
create ai_agent_sandbox (non-admin) → grant write rights → generate API key,
then write the key to test/.ai_agent_sandbox.key (gitignored) instead of
printing it.
- .gitignore (new), .env.example + README.md: sandbox vars, the
deno run --allow-all provisionSandbox.ts command, and kubectl one-liners to
pull DOLI_ADMIN_PASSWORD (secretkv) / DOLI_DB_PASSWORD (vso-db-credentials)
from the erp-sandbox namespace.
Why UI not SQL: API keys are encrypted with the instance's DOLI_INSTANCE_UNIQUE_ID,
so the key must be generated by the sandbox itself, not INSERTed raw.
deno check passes for provisionSandbox.ts and scripts/admin/userSetup.ts.
NOT run end-to-end: the sandbox Dolibarr is not installed yet (empty DB / fresh
install wizard), so the selectors are best-effort Dolibarr 22 conventions and
must be confirmed on the first real run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Dolibarr before-start step `chart/scripts/update_ownership.sql` (embedded
into a ConfigMap by `chart/templates/scripts-config.yaml`) hardcoded the
Postgres owner role `erp_role`. It reassigns ownership of all public-schema
objects to that role after install. For any non-prod environment the owner
role differs — by the multi-env elision rule (ADR-0002/0003) it is snake-case
`<app>_role` for prod and `<app>_<env>_role` for non-prod, so the sandbox owner
role is `erp_sandbox_role`. With the literal `erp_role`, installing Dolibarr in
`erp-sandbox` would reassign sandbox tables to prod's `erp_role`, which (a)
breaks the sandbox runtime (its dynamic DB creds are a member of
`erp_sandbox_role`, not `erp_role`) and (b) breaks the ADR-0003 reset
(`DROP OWNED BY erp_sandbox_role`).
Fix: make the owner role env-aware via a new chart value `db.ownerRole`.
- values.yaml: default `ownerRole: erp_role` (prod).
- values-sandbox.yaml: override `ownerRole: erp_sandbox_role`.
- update_ownership.sql: all `'erp_role'` literals → `'{{ .Values.db.ownerRole }}'`.
- scripts-config.yaml: render that one SQL file through `tpl` so the value is
substituted (the other script has no template vars and stays on `.Files.Get`).
The SQL's `$$`, `%I`, `format(...)`, `RAISE NOTICE` are not Go-template syntax,
so `tpl` only substitutes the added `{{ .Values.db.ownerRole }}`.
Verified: the prod ConfigMap render (values.yaml only) is byte-identical to
origin/main (empty diff, still `erp_role`); the sandbox render
(-f values.yaml -f values-sandbox.yaml) now contains `erp_sandbox_role` and no
bare `erp_role`; `helm lint` passes (no worse than origin/main).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ADR-0002 Phase D, erp-repo layer. iac/main.tf iterates `envs = ["prod",
"sandbox"]` so the app_roles module + the admin-bootstrap resources are
materialised per environment:
- module.app_roles["sandbox"] → auth/kubernetes/role/erp-sandbox +
postgres/creds/erp-sandbox (dynamic role GRANTs erp_sandbox_role — the
snake-case owner role created in factory#17 — and REVOKEs on DATABASE
erp-sandbox; token_policies ["default","erp-sandbox"] = the policy from
tools#3).
- random_password.admin_initial_password["sandbox"], random_uuid.dolibarr_id
["sandbox"], and vault_kv_secret_v2.dolibarr_admin_setup["sandbox"]
(kvv2/erp-sandbox/config) → the sandbox Dolibarr's own admin password +
encryption id.
State migration via `moved` blocks: the pre-existing single-env resources are
re-keyed into the for_each map under "prod" so introducing for_each does NOT
destroy+recreate them. Critical for random_uuid.dolibarr_id (prevent_destroy =
true — prod encryption id + paid-module binding): a wrong/absent moved would
HARD-FAIL the apply rather than lose it. The module's internal moved
(role -> role[0]) chains with the module re-key. Verified the exact compound
(old-module state → new module count+moved + for_each, one apply) with two
standalone tofu plans: both show "X moved", sandbox created, 0 destroyed.
env=prod renders byte-identical to the single-env baseline (module elision
rule), so the prod erp Vault auth role, dynamic creds, admin secret + KV are
unchanged.
D3 of Phase D. D1 = factory#17 (DB+role, merged). D2 = tools#3 (Vault
policies, merged). D4 (ArgoCD Application) is next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase C of the multi-env evolution discussed in the runbook design thread
(see PR description). Pure refactor — the prod helm template render is
verified byte-identical (10857 bytes both before and after, diff exit 0).
What was hardcoded, now templated:
- chart/templates/vaultauth.yaml role: erp → role: {{ .Values.vault.k8sRole }}
- chart/templates/vaultdynamicsecret.yaml path: creds/erp → path: {{ .Values.vault.dynamicPath }}
- chart/templates/vaultsecret.yaml path: erp/config → path: {{ .Values.vault.staticPath }}
- chart/templates/config.yaml DOLI_DB_NAME: erp → DOLI_DB_NAME: {{ .Values.db.name }}
DOLI_URL_ROOT: https://erp..lab → DOLI_URL_ROOT: 'https://{{ .Values.host }}'
values.yaml gains a documented multi-env coordinate block with prod defaults
(env, instance, host, db.name, vault.k8sRole, vault.dynamicPath, vault.staticPath).
The elision rule (env=prod → no suffix, env=non-prod → "<app>-<env>" suffix)
guarantees the prod render is unchanged.
chart/values-sandbox.yaml is added as the ready-to-use overlay for Phase D.
It is NOT wired into any helm install / ArgoCD app today — the platform side
(factory/postgres/iac tfvars, tools/hashicorp-vault/iac module signature) is
not yet evolved. The file documents the convention so the Phase D commit can
just `helm install -f values.yaml -f values-sandbox.yaml`.
Also fixes .gitea/workflows/vault.yaml CI typo: the vault_step JWT role was
gitea_cicd_webapp (copy-paste from the template repo) instead of
gitea_cicd_erp. Real bug — the erp CI would have failed JWT auth against
Vault. Fix unrelated to multi-env but bundled here because it's small and
touches the same file family.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
email-list.sh gains two hard-exclusion filters (applied before the
candidate test, regardless of attachments):
- EXCLUDE_PATTERN matches subjects starting with Invitation: / Updated
invitation: / Canceled event: / Accepted: / Declined: / Tentative: /
Maybe: (after stripping Re:/Fwd:/Tr: prefixes). Filters Google Calendar
events that always carry an .ics attachment.
- EXCLUDE_SENDER matches updates.<domain>, noreply@*calendar, news@,
newsletter@. Filters newsletter blast traffic.
Effect on --all-folders --candidates-only baseline: 27 noisy → 12
actionable (calendar invites + the staying-ahead.ai newsletter blast
removed). Real supplier docs intact: Darnis F1042 in /Notification, 3 Free
Mobile factures in /Inbox/abonnements, Mistral + Anthropic in /Inbox/books.
The originally-planned --mark-ingested feature is deferred to V8.2:
flag-set requires the Zoho OAuth scope ZohoMail.messages.UPDATE which our
read-only refresh_token doesn't have. Documented in SKILL.md: once the
user opts in to the wider scope, --mark-ingested becomes a one-line flag
on email-inspect.sh and is_candidate() learns to skip flag_info messages.
Captured the new --all-folders baseline at examples/email-list-all-folders.txt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V8 — first inbound-side skill. Closes the loop from "bill arrives by email"
to "ready to enter in Dolibarr UI". Read-only at every layer.
What ships:
- arcodange-email-ingest/scripts/zoho-curl.sh OAuth wrapper with token cache
(50 min TTL, mode 600) — avoids
hitting Zoho OAuth rate limit on
every invocation.
- arcodange-email-ingest/scripts/email-list.sh List candidates in /Inbox/books
(where the books@ alias auto-
routes mail). --candidates-only
filter on supplier patterns or
attachments. --all-folders to
scan everything.
- arcodange-email-ingest/scripts/email-inspect.sh Pull message + attachments,
pdftotext on each PDF, heuristic
extract (supplier, ref, dates,
totals, VAT rate), emit Dolibarr
supplier-invoice draft JSON.
Architecture choice — Zoho API (not IMAP):
- books@arcodange.fr is an alias of gabrielradureau@arcodange.fr → one OAuth
refresh_token covers everything.
- Gmail folded in via forwarding (arcodange@gmail.com → books@) — no Google
API setup, no app-passwords, no second OAuth flow.
- Token-based auth, no SCA rabbit hole.
V8.0 baseline (in /Inbox/books):
- 3 candidates: Mistral AI facture, Anthropic Stripe receipt (Fwd Gmail),
INPI payment receipt (Fwd Gmail).
- Heuristic extraction is best-effort: works on amounts/refs for some
templates, misses others (Mistral PDF format, Stripe receipt layout).
- --save-pdf <DIR> lets the operator grab the PDFs for manual entry when
the heuristic falls short.
Rate-limit pitfall documented: Zoho OAuth refresh has an aggressive throttle
("too many requests continuously"). The cache file at $TMPDIR/zoho-access-$USER
(mode 600, 50 min TTL) prevents this; on 401 the wrapper auto-refreshes once
and retries.
V8.1+ ideas in SKILL.md out-of-scope:
- mark ingested emails (IMAP flag or Zoho label)
- body text extraction (inline-HTML invoices)
- per-template parsers or LLM-based extraction
- IMAP fallback for non-Zoho mailboxes
CLI: bin/arcodange email {list|inspect|curl} integrated.
Base updates: dolibarr/SKILL.md cross-link, dolibarr/README.md env schema
extended with ZOHO_CLIENT_ID/SECRET/REFRESH_TOKEN/DC.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three improvements that reduce the V6.1 exit-1 signal from 10 to 1 on
the current Arcodange baseline. Every bucket now has a single, clear
purpose; the only entry counted as a failure is a genuine action item.
A. fk_account context on dolibarr-only
- Fetches /bankaccounts and tags each dolibarr-only with the account
ref + label (e.g. "CCA1 (G.RADUREAU Compte Courant Asso)").
- Splits dolibarr-only into "on API-tracked accounts" (QON*/WIS* — real
gaps) vs "not in API scope" (CCA1 / personal — expected gaps).
- Personal-account entries no longer count toward the failure verdict.
B. Avoir-cycle netting
- Pairs AVC entries of -X on socid S with FAC entries of +X on the
same socid within ±5d.
- Both surface in a dedicated AVOIR-NETTED bucket and are excluded from
dolibarr-only, since the bank only sees the net of the cycle.
- Resolves the V6.1 noise where AVC001-CL0001001 + FAC001-CL00001
appeared as fake gaps for a 510€ cancel-and-reissue dance.
C. Wire-reference strong matching (--enrich flag, opt-in)
- When --enrich is passed, bank-match.sh fetches /v1/transfers/{id}
per Wise TRANSFER and reads the wire `reference` field.
- References containing a FAC\d+(CL\d+)? pattern strong-match against
the corresponding Dolibarr customer invoice (annotated [wire-ref]
vs the loose [amt+date] kind).
- Verified on FAC002 5100€: KM's wire memo "FOR INVOICE FAC002CL0001002"
gives an unambiguous match independent of date drift.
Baseline (Jan-May 2026, --enrich on):
6 matched · 1 internal · 2 avoir-netted · 7 bank-known · 1 bank-UNKNOWN
0 dol-only-API · 7 dol-only-personal
→ exit-1 count = 1 (just the +2147€ KM Wise 2026-05-29 to record).
The CLI (bin/arcodange) gains --enrich on the match subcommand. The
SKILL.md has a new "V7 bucket structure" section explaining the seven
buckets and a before/after table showing the signal/noise improvement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
