Gabriel Radureau
8ec8fde67e
The accounting data + issued documents are legally retained 10 years and warrant a
backup dedicated to Dolibarr. An audit found the generic Longhorn external backup
NEVER covered the erp volume (its Longhorn volume sits in the orphaned `default`
recurring-job group; the only job has groups=[] → serves nothing; lastBackupAt=never).
So /var/www/documents (invoice PDFs, supplier pieces, contracts, ECM) had zero
offsite copy — only in-cluster replicas.
ops/backup/dolibarr-backup.sh (orchestrator) + ops/backup/backup-job.sh (in-container
logic, env-driven, single source of truth):
- pg_dump -Fc of the DB + tar of the documents PVC (RWX, read-only mount) ->
s3://arcodange-backup/erp/<env>/{db,docs}/<ts>, then tiered prune (daily 30d /
monthly 12m / yearly 10y).
- prod is READ-only (dump+tar read; writes go only to the backup bucket); the DB is
read with the env's own dynamic creds; the GCS HMAC secret is copied transiently
(base64, deleted on exit) and never printed; the whole script ships base64.
- fixes the aws-cli v2.23+ default-checksum incompatibility with GCS/S3-compat
(SignatureDoesNotMatch) via AWS_*_CHECKSUM_*=when_required.
Proven live: sandbox end-to-end (dump+tar+upload+prune, verified in GCS, cleaned up)
and retention logic unit-tested (1100 daily -> 46 kept). The FIRST real prod backup
was taken (erp/prod/db 1.2 MB + erp/prod/docs 12.5 MB) — closing the gap now.
Automation (recurring CronJob in the chart + a dedicated erp Vault policy for its
own S3 creds) is the documented next step; the orchestrator works today on demand.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
57 lines
2.6 KiB
Bash
Executable File
57 lines
2.6 KiB
Bash
Executable File
#!/bin/sh
|
|
# In-container backup logic for Dolibarr — the single source of truth shared by the
|
|
# manual orchestrator (ops/backup/dolibarr-backup.sh) and the scheduled CronJob
|
|
# (chart/templates/backup-cronjob.yaml). Driven entirely by environment:
|
|
# BUCKET PREFIX DB PGHOST (config)
|
|
# PGUSER PGPASSWORD (DB creds, from vso-db-credentials)
|
|
# AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_ENDPOINTS (S3 creds)
|
|
# It dumps the DB (pg_dump -Fc) + tars the documents mounted at /docs, pushes both
|
|
# to s3://$BUCKET/$PREFIX/{db,docs}/, then prunes to a tiered retention.
|
|
set -eu
|
|
apk add --no-cache aws-cli tar gzip >/dev/null 2>&1 || { echo "ABORT apk add"; exit 1; }
|
|
: "${BUCKET:?}"; : "${PREFIX:?}"; : "${DB:?}"; : "${PGHOST:?}"
|
|
export AWS_DEFAULT_REGION="${AWS_DEFAULT_REGION:-us-east-1}"
|
|
# GCS / S3-compatible stores reject aws-cli v2.23+ default integrity checksums
|
|
# ("SignatureDoesNotMatch / Invalid argument") — only sign/validate when required.
|
|
export AWS_REQUEST_CHECKSUM_CALCULATION=when_required
|
|
export AWS_RESPONSE_CHECKSUM_VALIDATION=when_required
|
|
S3() { aws --endpoint-url "$AWS_ENDPOINTS" s3 "$@"; }
|
|
|
|
TS=$(date -u +%Y-%m-%dT%H-%M-%SZ)
|
|
echo "timestamp=$TS db=$DB -> s3://$BUCKET/$PREFIX"
|
|
pg_dump -h "$PGHOST" -U "$PGUSER" -d "$DB" -Fc -f /tmp/db.dump
|
|
echo "db.dump $(wc -c < /tmp/db.dump) bytes"
|
|
tar -C /docs -czf /tmp/docs.tar.gz . 2>/dev/null
|
|
echo "docs.tar.gz $(wc -c < /tmp/docs.tar.gz) bytes"
|
|
S3 cp /tmp/db.dump "s3://$BUCKET/$PREFIX/db/$TS.dump"
|
|
S3 cp /tmp/docs.tar.gz "s3://$BUCKET/$PREFIX/docs/$TS.tar.gz"
|
|
echo "uploaded to s3://$BUCKET/$PREFIX/{db,docs}/$TS.*"
|
|
|
|
# tiered retention: daily 30d / monthly 12m (latest per month) / yearly ~10y
|
|
cat > /tmp/prune.py <<'PY'
|
|
import sys, datetime
|
|
keys=[k.strip() for k in open(sys.argv[1]) if k.strip()]
|
|
now=datetime.datetime.strptime(sys.argv[2][:10], "%Y-%m-%d").date()
|
|
def d(k):
|
|
try: return datetime.datetime.strptime(k[:10], "%Y-%m-%d").date()
|
|
except Exception: return None
|
|
dated=sorted([(d(k),k) for k in keys if d(k)], key=lambda x:x[0])
|
|
keep=set(); bymonth={}; byyear={}
|
|
for dt,k in dated:
|
|
age=(now-dt).days
|
|
if age <= 30: keep.add(k)
|
|
elif age <= 365: bymonth[(dt.year,dt.month)]=k
|
|
elif age <= 3660: byyear[dt.year]=k
|
|
keep |= set(bymonth.values()) | set(byyear.values())
|
|
for dt,k in dated:
|
|
if k not in keep: print(k)
|
|
PY
|
|
for SUB in db docs; do
|
|
S3 ls "s3://$BUCKET/$PREFIX/$SUB/" | awk '{print $4}' > /tmp/keys.$SUB || true
|
|
python3 /tmp/prune.py "/tmp/keys.$SUB" "$TS" > /tmp/del.$SUB || true
|
|
while read -r DK; do
|
|
[ -n "$DK" ] && S3 rm "s3://$BUCKET/$PREFIX/$SUB/$DK" && echo "pruned $SUB/$DK"
|
|
done < /tmp/del.$SUB
|
|
done
|
|
echo "DONE."
|