[vibe](../README.md) > [Runbooks](README.md) > **Set up a new app** # Set up a new app > **Status:** ✅ Active > **Audience:** platform operator + agents (English). For the detailed human-facing procedure see the French [new-web-app runbook](../../doc/runbooks/new-web-app/README.md). > **Last Updated:** 2026-06-23 ## TL;DR > [!TIP] > Standing up a brand-new application touches **three repos** — the app's own Gitea repo, [`factory`](../../argocd/values.yaml), and [`tools`](../guidebooks/tools/secrets-and-vso.md) — with a **strict ordering dependency**. An agent may write every file and open every PR (`[AGENT]`), but each **merge/apply is `[HUMAN]`-gated**. The single rule that everything else hangs on: the **factory** Postgres DB+role and the **tools** Vault JWT role MUST be applied **before** the app's own `iac/` runs. Ship the app in **degraded mode first** (no DB/Vault), wire the platform sides, then turn on dynamic credentials last. The detailed companion is the French [new-web-app runbook](../../doc/runbooks/new-web-app/README.md); this page is its agent-oriented English mirror. > [!CAUTION] > **Ordering is load-bearing — do not reorder the phases.** > - The app's own `iac/` (Phase 6) calls the shared `app_roles` module, which issues `GRANT _role TO …` on every dynamic credential and authenticates to Vault as `gitea_cicd_`. So **both** of these must already exist: > - the Postgres role `_role` + database `` → created by the **factory** side (Phase 4). > - the Vault JWT role `gitea_cicd_` + policies `` / `-ops` → created by the **tools** side (Phase 5). > - The app's `vault.yaml` CI needs the **`TERRAFORM_SSH_KEY`** Actions secret (the `tofu_module_reader` SSH key from Vault) or `terraform init` cannot clone the `app_roles` module over `git::ssh://`. This is the canonical pitfall — it sank the first `iac/` push and was fixed in [dance-lessons-coach PR #100](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/100). > Apply Phases 4 and 5 **before** merging Phase 6. ## Scope This runbook covers standing up a **brand-new application** end-to-end: its own Gitea repo, a Helm `chart/`, CI/CD with IaC (`iac/` + `.gitea/workflows/`), and database access — all deployed by factory **ArgoCD** into a dedicated namespace. Systems touched: **Gitea** (repo + Actions + container registry), **Postgres** (DB + owner role via factory), **Vault** (JWT CI role, policies, dynamic DB creds via tools + app), **k3s** (namespace, pod, SA), **ArgoCD** (Application sync + image-updater), and **Traefik** (ingress). It does **not** cover: writing the application code itself, the one-time platform foundations (Vault mounts, the Vault→Postgres connection, the `gitea_cicd` bootstrap JWT role, the `tofu_module_reader` bot, org-level Actions secrets — all already in place), or adding a non-application platform component (see [Set up a new tool](new-tool.md)). The reference onboarding is **`dance-lessons-coach`** (verified from its merged PRs), with **[webapp](../guidebooks/applications/webapp.md)** as the canonical app to clone. ## Preconditions - [ ] Working in a worktree under `.claude/worktrees//` (never the trunk). - [ ] You can create a Gitea repo under `arcodange-org` (default) or `arcodange` (for some apps). - [ ] Local clones of `factory` and `tools` are available and on synced `main`. - [ ] The `` name is chosen — **kebab-case, lowercase**. This is the **universal join key**: the same string is reused verbatim across Gitea, Postgres, Vault, Kubernetes, ArgoCD, GCS, and DNS. One typo silently breaks the chain. See [naming-conventions](../guidebooks/lab-ecosystem/naming-conventions.md) and the FR [conventions](../../doc/runbooks/new-web-app/conventions.md). - [ ] The platform foundations exist (Vault mounts `kvv2`/`postgres`/`transit` + auth `kubernetes`, the Vault→Postgres connection via `credentials_editor`, the bootstrap `gitea_cicd` role, the `tofu_module_reader` SSH bot, and org Actions secrets `HOMELAB_CA_CERT` / `vault_oauth__sh_b64` / `PACKAGES_TOKEN`). ## The three-repo onboarding (ordering) ```mermaid %%{init: {'theme':'base','themeVariables':{'fontSize':'14px'}}}%% flowchart TB classDef app fill:#2563eb,stroke:#1e40af,color:#ffffff classDef plat fill:#059669,stroke:#047857,color:#ffffff classDef tools fill:#7c3aed,stroke:#6d28d9,color:#ffffff classDef run fill:#b45309,stroke:#92400e,color:#ffffff P1["Phase 1-3 · APP repo
chart/ degraded + Vault-ready (gated) + TLS
(serves, no DB/Vault yet)"]:::app P4["Phase 4 · FACTORY repo
argocd/values.yaml + postgres/iac
→ DB <app> + role <app>_role"]:::plat P5["Phase 5 · TOOLS repo
hashicorp-vault/iac
→ gitea_cicd_<app> + policies"]:::tools P6["Phase 6 · APP repo
iac/ (app_roles module) + vault.yaml
+ TERRAFORM_SSH_KEY secret"]:::app P7["Phase 7-8 · APP repo
vault.enabled=true + dockerimage.yaml
→ dynamic creds on, image rollout"]:::run P1 --> P4 P1 --> P5 P4 --> P6 P5 --> P6 P6 --> P7 ``` 1. **Phases 1-3 (app repo):** ship the chart in degraded mode, make it Vault-ready behind a default-off gate, and set the right ingress — none of this needs the platform sides yet. 2. **Phase 4 (factory) and Phase 5 (tools)** are independent of each other but **both** must be applied before Phase 6. 3. **Phase 6 (app repo)** applies the app's own `iac/`, which depends on the role/JWT created in 4 and 5, and needs the `TERRAFORM_SSH_KEY` secret. 4. **Phases 7-8 (app repo)** flip `vault.enabled=true` for live dynamic DB creds, then add the image-build CI so ArgoCD's image-updater rolls out releases. ## Procedure ### Phase 0 — Choose the name and create the repo 1. **[HUMAN]** Fix `` (kebab-case) and the Gitea org. Default org is **`arcodange-org`**; some apps live under **`arcodange`** (e.g. `dance-lessons-coach`, `telegram-gateway`). Create the empty repo under the chosen org. Inheriting org-level Actions secrets is why the org choice matters. ### Phase 1 — App in degraded mode Mirrors [dance-lessons-coach PR #89](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/89). Clone the [webapp](../guidebooks/applications/webapp.md) pattern. 2. **[AGENT]** Add a `Dockerfile` and a Helm `chart/` (`deployment`, `service`, `ingress`, `serviceaccount`, `configmap`, `_helpers.tpl`, `NOTES.txt`) with **no DB/Vault wiring**. Set: - ingress host `.arcodange.lab` (internal) and/or `.arcodange.fr` (public) — TLS details land in Phase 3; - a `nodeSelector` of `kubernetes.io/hostname: pi1` (network entrypoint, preserves the user IP, avoids NAT); - `/healthz` (or the app's real path, e.g. `dance-lessons-coach` uses `/api/healthz`) for **both** liveness and readiness probes; - leave any DB host empty so the pod serves in degraded mode. ```bash # [AGENT] lint + render before opening the PR — safe, no cluster contact helm lint chart/ helm template chart/ --set image.repository=test --set image.tag=v1 ``` 3. **[HUMAN]** Open and merge the PR. Verify the app serves in degraded mode (binary + health endpoint reachable once ArgoCD picks it up in Phase 4+). ### Phase 2 — Make the chart Vault-ready (gated, default off) Mirrors [dance-lessons-coach PR #97](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/97). 4. **[AGENT]** Add `VaultAuth`, `VaultStaticSecret`, and `VaultDynamicSecret` templates, each **gated behind `.Values.vault.enabled`** (default `false`) so a plain `helm install` keeps working. The reference `values.yaml` exposes: ```yaml # chart/values.yaml — gate + the three Vault join keys (all derived from ) vault: enabled: false role: # k8s auth backend role (matches iac/main.tf) kvv2Path: /config # KVv2 secret path postgresPath: creds/ # postgres dynamic creds path ``` The `VaultAuth` targets the k8s role `` with the app's ServiceAccount and audience `vault`; the `VaultDynamicSecret` reads `postgres/creds/` into a `db-credentials` Secret and `rolloutRestartTargets` the Deployment. 5. **[HUMAN]** Open and merge the PR. The chart is now Vault-ready without activating any Vault dependency. ### Phase 3 — Ingress / TLS Mirrors [dance-lessons-coach PR #98](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/98). Pick by host suffix: 6. **[AGENT]** For a **`.lab`** host: `traefik.../router.entrypoints: websecure` + `router.tls: "true"` + `router.tls.certresolver: letsencrypt` (with `router.tls.domains.0.main: arcodange.lab` and `…sans: .arcodange.lab`) + `router.middlewares: localIp@file`. For a **`.fr`** host: `router.entrypoints: web` + `router.middlewares: kube-system-crowdsec@kubernetescrd`. (Convention: `.lab` = internal, websecure + localIp + letsencrypt; `.fr` = public, web + crowdsec.) 7. **[HUMAN]** Merge the PR. ### Phase 4 — FACTORY side (DB + role, ArgoCD enrollment) Mirrors [factory PR #1](https://gitea.arcodange.lab/arcodange-org/factory/pulls/1) (ArgoCD) and [factory PR #2](https://gitea.arcodange.lab/arcodange-org/factory/pulls/2) (Postgres). Link: [postgres-iac](../guidebooks/factory-provisioning/opentofu/postgres-iac.md), [ci-apply-flow](../guidebooks/factory-provisioning/opentofu/ci-apply-flow.md). 8. **[AGENT]** Enroll `` in [`argocd/values.yaml`](../../argocd/values.yaml) under `gitea_applications`. The [apps template](../../argocd/templates/apps.yaml) defaults the org to `arcodange-org` (`{{- $org := default "arcodange-org" $app_attr.org -}}`), so add `org: arcodange` **only** if the app is not under `arcodange-org`. Add image-updater annotations for digest-based rollout: ```yaml # argocd/values.yaml — under gitea_applications : org: arcodange # ← ONLY if not arcodange-org annotations: argocd-image-updater.argoproj.io/image-list: =gitea.arcodange.lab//:latest argocd-image-updater.argoproj.io/.update-strategy: digest ``` 9. **[AGENT]** Add `""` to the `applications` list in [`postgres/iac/terraform.tfvars`](../../postgres/iac/terraform.tfvars). This creates the `` database, the non-login owner role `_role`, and the pgbouncer `user_lookup()` function. ```hcl # postgres/iac/terraform.tfvars applications = [ "webapp", "erp", "crowdsec", "plausible", "dance-lessons-coach", "", # ← add ] ``` 10. **[HUMAN]** Merge both PRs. Factory CI (`postgres.yaml`) applies — the DB + role now exist. ArgoCD creates the Application and deploys the degraded chart into namespace ``. ### Phase 5 — TOOLS side (Vault JWT role + policies) Mirrors [tools PR #1](https://gitea.arcodange.lab/arcodange-org/tools/pulls/1). Link: [tools secrets-and-vso](../guidebooks/tools/secrets-and-vso.md), [tools components](../guidebooks/tools/components.md). 11. **[AGENT]** Add `{ name = "" }` to the `applications` list in [`tools/hashicorp-vault/iac/terraform.tfvars`](https://gitea.arcodange.lab/arcodange-org/tools/src/branch/main/hashicorp-vault/iac/terraform.tfvars). Via the `app_policy` / `app_roles` modules this creates the `gitea_cicd_` JWT role, the `` (runtime) and `-ops` (CI) policies, the `-ops` identity group, and the k8s auth role. ```hcl # tools/hashicorp-vault/iac/terraform.tfvars applications = [ { name = "webapp" }, { name = "erp" }, { name = "" }, # ← add # optional fields when needed: # { name = "", ops_policies = ["…"], service_account_names = ["…"], service_account_namespaces = ["tools"] } ] ``` 12. **[HUMAN]** Merge the PR. Tools CI (`vault.yaml`) applies — `gitea_cicd_` and the policies now exist. ### Phase 6 — App IaC + Vault workflow Mirrors [dance-lessons-coach PR #99](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/99) and the [#100 fix](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/100). See [05-app-terraform](../../doc/runbooks/new-web-app/05-app-terraform.md) for the module contract. > [!CAUTION] > **Phases 4 and 5 must already be applied** before merging this phase, or the first `tofu apply` fails (no `_role` to GRANT, or Vault auth fails on the missing `gitea_cicd_` role). 13. **[AGENT]** Add the app's `iac/`: - `providers.tf` — Vault provider with `auth_login_jwt { mount = "gitea_jwt", role = "gitea_cicd_" }`. - `backend.tf` — GCS backend `bucket = "arcodange-tf"`, `prefix = "/main"`. - `main.tf` — call the shared module (the exact source string used by every app): ```hcl module "app_roles" { source = "git::ssh://git@192.168.1.202:2222/arcodange-org/tools.git//hashicorp-vault/iac/modules/app_roles?depth=1&ref=main" name = "" } ``` This provisions `postgres/creds/` (dynamic DB role inheriting `_role`) and the k8s auth role ``. Add any app-specific `kvv2//config` secrets alongside. 14. **[AGENT]** Add `.gitea/workflows/vault.yaml` that authenticates via Gitea OIDC and runs `tofu apply iac/`. The `vault-action` step's `role:` and `providers.tf`'s `role` **must both** be `gitea_cicd_` (the copy-paste trap — `erp` still carries a stale `gitea_cicd_webapp`). The secrets block must read the SSH key: ```yaml # .gitea/workflows/vault.yaml — vault-action secrets block secrets: | kvv1/google/credentials credentials | GOOGLE_BACKEND_CREDENTIALS ; kvv1/gitea/tofu_module_reader ssh_private_key | TERRAFORM_SSH_KEY ; ``` 15. **[HUMAN]** Add the **`TERRAFORM_SSH_KEY`** secret (the `tofu_module_reader` SSH key, read from Vault at `kvv1/gitea/tofu_module_reader`) to the app repo's **Actions secrets**. Without it, `terraform init` cannot clone the `app_roles` module over `git::ssh://` — the canonical pitfall fixed in [PR #100](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/100). 16. **[HUMAN]** Merge the PR. The app's `vault.yaml` runs `tofu apply` — `postgres/creds/` and the k8s role `` now exist. ### Phase 7 — Turn on dynamic DB credentials 17. **[AGENT]** Set `vault.enabled=true` in `chart/values.yaml` (and point the app's DB env at `pgbouncer.tools:5432`). On next ArgoCD sync, VSO authenticates with the k8s role ``, fetches dynamic Postgres creds from `postgres/creds/` into the `db-credentials` Secret, and the pod reaches the DB through **pgbouncer.tools** with a short-lived user that inherits `_role`. See [webapp](../guidebooks/applications/webapp.md) and [erp](../guidebooks/erp/README.md) for the consumption pattern. 18. **[HUMAN]** Merge the PR. ### Phase 8 — Image CI + deploy 19. **[AGENT]** Add `.gitea/workflows/dockerimage.yaml` that builds the image and pushes it to the Gitea registry (`gitea.arcodange.lab//:latest` + branch tag), logging in with `PACKAGES_TOKEN`. No deploy step is needed — the ArgoCD image-updater annotations from Phase 4 watch `latest` (digest strategy) and roll it out. Skip this phase entirely for apps that run a public upstream image (e.g. `erp`/Dolibarr). 20. **[HUMAN]** Merge the PR. ## Verification The convention chain must resolve end-to-end (this is the same parity check the [safe-env PRD](../ADR/0001-safe-prod-like-environment.md) rehearses in the sandbox). All checks below are **[AGENT]** read-only: ```bash # [AGENT] Gitea repo exists under the chosen org git ls-remote https://gitea.arcodange.lab// &>/dev/null && echo "repo OK" # [AGENT] Postgres DB + owner role exist (run from a host with psql access to the engine) psql -h 192.168.1.202 -U credentials_editor -tAc \ "SELECT datname FROM pg_database WHERE datname='';" psql -h 192.168.1.202 -U credentials_editor -tAc \ "SELECT rolname FROM pg_roles WHERE rolname='_role';" # [AGENT] Vault: dynamic role, policies, and CI JWT role exist vault read postgres/roles/ vault policy read vault policy read -ops vault read auth/gitea_jwt/role/gitea_cicd_ # [AGENT] ArgoCD Application is Synced + Healthy kubectl --context -n argocd get application \ -o jsonpath='{.status.sync.status}/{.status.health.status}' # expected: Synced/Healthy # [AGENT] VSO created the db-credentials Secret + pod is Running + ingress resolves kubectl --context -n get secret db-credentials kubectl --context -n get pods curl -fsS https://.arcodange.lab/healthz # or the app's real health path ``` Expected: repo present; PG `` DB + `_role` exist; Vault `postgres/creds/` + policies ``/`-ops` + `gitea_cicd_` exist; ArgoCD Application `Synced/Healthy`; the `db-credentials` Secret was created by VSO; the pod is `Running`; the ingress resolves. ## Rollback Revert the per-repo PRs **in reverse order**: app → tools → factory. Tag each undo just like the procedure. 1. **[HUMAN]** App repo: revert Phase 8 → 7 → 6 PRs. Reverting the Phase 6 `iac/` removes `postgres/creds/` and the k8s role on the next CI run; setting `vault.enabled=false` returns the chart to degraded mode. 2. **[HUMAN]** Tools repo: remove the `{ name = "" }` entry; tools CI prunes `gitea_cicd_` + policies. 3. **[HUMAN]** Factory repo: remove the `` entry from `argocd/values.yaml` — ArgoCD **prunes the Application** (and its namespace) — and remove `""` from `postgres/iac/terraform.tfvars` to drop the DB + role. 4. **[HUMAN]** For a full cluster-level recovery (power-cut, lost unseal key) consult `CLUSTER_RECOVERY.md`. > [!WARNING] > Removing the Postgres entry **drops the database** `` and its data. Back up first if the app already holds state. ## References - French human-operator procedure: [new-web-app runbook](../../doc/runbooks/new-web-app/README.md) + [conventions](../../doc/runbooks/new-web-app/conventions.md) (the universal `` join key). - Exemplars: [webapp](../guidebooks/applications/webapp.md) (in-house image + DB) and [erp](../guidebooks/erp/README.md) (public image + DB). - Platform mechanics: [tools secrets-and-vso](../guidebooks/tools/secrets-and-vso.md), [tools components](../guidebooks/tools/components.md), [postgres-iac](../guidebooks/factory-provisioning/opentofu/postgres-iac.md), [ci-apply-flow](../guidebooks/factory-provisioning/opentofu/ci-apply-flow.md), [naming-conventions](../guidebooks/lab-ecosystem/naming-conventions.md), [secrets-and-vault](../guidebooks/lab-ecosystem/secrets-and-vault.md). - Companion runbook: [Set up a new tool](new-tool.md). - Parity rehearsal: [safe-prod-like-environment ADR/PRD](../ADR/0001-safe-prod-like-environment.md). - Factory files: [argocd/values.yaml](../../argocd/values.yaml), [argocd/templates/apps.yaml](../../argocd/templates/apps.yaml), [postgres/iac/terraform.tfvars](../../postgres/iac/terraform.tfvars). - Reference PRs (verified, all merged): - app `dance-lessons-coach`: [#89 degraded](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/89) · [#97 Vault-ready gate](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/97) · [#98 TLS ingress](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/98) · [#99 iac + workflow](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/99) · [#100 TERRAFORM_SSH_KEY fix](https://gitea.arcodange.lab/arcodange/dance-lessons-coach/pulls/100) - factory: [#1 ArgoCD enroll + org override](https://gitea.arcodange.lab/arcodange-org/factory/pulls/1) · [#2 Postgres DB + role](https://gitea.arcodange.lab/arcodange-org/factory/pulls/2) - tools: [#1 Vault JWT role + policy](https://gitea.arcodange.lab/arcodange-org/tools/pulls/1)