arcodange-org/factory
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(cicd): cap act_runner jobs (3g/2cpu/pids) and capacity 2→1 — a build can no longer take down pi1

Browse Source 
Incident 2026-07-23: an uncapped nuxt generate (3.5G RSS) on pi1 starved the
k3s control-plane and traefik (load >150, no swap, no OOM-kill) — every
*.arcodange.lab endpoint went dark, Gitea included, while Gitea itself was
healthy on pi2. Job containers are spawned via the host docker socket, so
cgroup caps on the job container are the only guardrail.

Applied live on pi1+pi3 via 03_cicd.yml on 2026-07-24 (both runners
re-registered; pi3's runner was down and is back in service).

Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
arcodange
2026-07-24 00:09:47 +02:00
co-authored by Claude Fable 5
parent 1f22669ac6
commit 961691d6d2
1 changed files with 5 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ansible/arcodange/factory/playbooks/03_cicd.yml
+5 -2
View File
@@ -64,7 +64,8 @@
# Where to store the registration result. # Where to store the registration result.
file: .runner file: .runner
# Execute how many tasks concurrently at the same time. # Execute how many tasks concurrently at the same time.
capacity: 2 # 1 seul job à la fois : les hôtes (8 Go, control-plane k3s sur pi1) ne survivent pas à 2 builds lourds simultanés.
capacity: 1
# Extra environment variables to run jobs. # Extra environment variables to run jobs.
envs: envs:
A_TEST_ENV_NAME_1: a_test_env_value_1 A_TEST_ENV_NAME_1: a_test_env_value_1
@@ -117,7 +118,9 @@
# Whether to use privileged mode or not when launching task containers (privileged mode is required for Docker-in-Docker). # Whether to use privileged mode or not when launching task containers (privileged mode is required for Docker-in-Docker).
privileged: false privileged: false
# And other options to be used when the container is started (eg, --add-host=my.gitea.url:host-gateway). # And other options to be used when the container is started (eg, --add-host=my.gitea.url:host-gateway).
options: # Plafonds durs : un build ne doit jamais pouvoir affamer l'hôte (incident 2026-07-23 :
# nuxt generate à 3,5 Go RSS sur pi1 → load 150, ingress+API k3s morts → gitea.arcodange.lab injoignable).
options: "--memory=3g --memory-swap=3g --cpus=2 --pids-limit=512"
# The parent directory of a job's working directory. # The parent directory of a job's working directory.
# NOTE: There is no need to add the first '/' of the path as act_runner will add it automatically. # NOTE: There is no need to add the first '/' of the path as act_runner will add it automatically.
# If the path starts with '/', the '/' will be trimmed. # If the path starts with '/', the '/' will be trimmed.