From e84383ee3410db8c9b21255ed035f5114897ccea Mon Sep 17 00:00:00 2001 From: Gabriel Radureau Date: Mon, 27 Jul 2026 13:34:26 +0200 Subject: [PATCH] =?UTF-8?q?feat(miroirs)=20=E2=80=94=20un=20d=C3=A9p=C3=B4?= =?UTF-8?q?t=20personnel=20n'est=20pas=20une=20organisation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Le rôle gitea_repo ne savait viser qu'un propriétaire : l'organisation, des deux côtés à la fois. Les dépôts qui vivent sous le compte personnel `arcodange` ne pouvaient donc pas sortir du homelab — ni être balayés par gitea_sync, qui n'interroge que /orgs//repos. Trois séparations, toutes rétrocompatibles (les défauts reconduisent le comportement org-vers-org des dix dépôts déjà en miroir) : - le propriétaire côté Gitea (`gitea_repo_owner`) n'est plus le même objet que celui d'en face (`github_owner`, `gitlab_owner`) ; - un compte personnel n'est pas une organisation : GitHub ne crée pas le dépôt au même endroit, d'où `github_owner_is_org` qui route vers POST /user/repos ; - GitLab devient facultatif (`gitea_mirror_gitlab`). Il ne l'était pas : sa création attendait un 201 sans ignore_errors, si bien qu'un échec GitLab avortait l'itération — y compris la moitié GitHub, qui n'y était pour rien. Deux défauts corrigés au passage, tous deux silencieux : - les trois listages de gitea_sync ne paginaient pas (30 chez GitHub, 20 chez GitLab). Sous la taille d'une page tout va bien ; au-delà, la différence entre forges désigne de FAUX dépôts manquants et le rôle les « répare » ; - la migration entrante posait `repo_owner: github_organization` pour désigner le propriétaire DANS Gitea. Et un piège découvert en exécutant : un dépôt GitHub créé vide adopte comme branche par défaut la PREMIÈRE branche que le miroir lui pousse — `kadans` a atterri sur `arcodange/adr-ddd-front`. Le rôle réaligne désormais sur la branche par défaut de Gitea ; le miroir étant asynchrone, l'alignement échoue au run qui crée le dépôt et réussit au suivant, d'où le failed_when permissif. Ce qui sort du homelab reste un CHOIX : playbooks/07_mirrors.yml parcourt une liste explicite et relue (`gitea_mirrored_repos`) plutôt que la différence automatique entre forges, qui recréerait un dépôt supprimé exprès. Co-Authored-By: Claude Opus 5 (1M context) --- .../inventory/group_vars/all/gitea.yml | 39 ++++++++ .../factory/playbooks/07_mirrors.yml | 60 ++++++++++++ .../roles/gitea_repo/defaults/main.yml | 36 ++++++- .../factory/roles/gitea_repo/tasks/main.yml | 97 ++++++++++++++----- .../factory/roles/gitea_repo/vars/main.yml | 8 +- .../roles/gitea_sync/defaults/main.yml | 16 ++- .../factory/roles/gitea_sync/tasks/main.yml | 49 ++++++++-- .../factory/roles/gitea_sync/vars/main.yml | 6 +- .../ansible/07-mirrors.md | 81 ++++++++++++++++ .../factory-provisioning/ansible/README.md | 4 +- .../factory-provisioning/ansible/roles.md | 4 +- 11 files changed, 357 insertions(+), 43 deletions(-) create mode 100644 ansible/arcodange/factory/playbooks/07_mirrors.yml create mode 100644 vibe/guidebooks/factory-provisioning/ansible/07-mirrors.md diff --git a/ansible/arcodange/factory/inventory/group_vars/all/gitea.yml b/ansible/arcodange/factory/inventory/group_vars/all/gitea.yml index 837e292..d255af1 100644 --- a/ansible/arcodange/factory/inventory/group_vars/all/gitea.yml +++ b/ansible/arcodange/factory/inventory/group_vars/all/gitea.yml @@ -9,3 +9,42 @@ # so the secret propagation playbook iterates over this list. gitea_secret_propagation_users: - arcodange + +# Dépôts mis en miroir vers GitHub (et GitLab) par playbooks/06_mirrors.yml. +# Gitea reste la source ; les forges publiques ne reçoivent qu'une copie poussée. +# +# `gitea_sync` ne balaie qu'UN propriétaire à la fois et déduit les manques en +# comparant les forges : utile pour l'organisation, inadapté ici, où l'on choisit +# dépôt par dépôt ce qui sort du homelab. D'où cette liste, explicite et relue. +# +# owner : propriétaire côté Gitea +# github_owner : propriétaire côté GitHub (un compte personnel n'est pas une +# organisation — voir github_owner_is_org) +# gitlab_namespace : ID numérique du groupe ou du compte GitLab d'accueil +gitea_mirrored_repos: + - name: kadans + owner: arcodange + github_owner: arcodange + github_owner_is_org: false + description: application d'entrainement social de danse + - name: kadans-api + owner: arcodange + github_owner: arcodange + github_owner_is_org: false + - name: kadans-dossier + owner: arcodange + github_owner: arcodange + github_owner_is_org: false + - name: kadans-jobs + owner: arcodange + github_owner: arcodange + github_owner_is_org: false + - name: video_analysis + owner: arcodange + github_owner: arcodange + github_owner_is_org: false + +# Espace GitLab qui accueille les dépôts du compte personnel. À renseigner avec +# l'ID du namespace « arcodange » sur gitlab.com (Settings → General) : sans lui, +# la création GitLab retomberait dans le groupe arcodange-org. +gitlab_personal_namespace_id: ~ diff --git a/ansible/arcodange/factory/playbooks/07_mirrors.yml b/ansible/arcodange/factory/playbooks/07_mirrors.yml new file mode 100644 index 0000000..cec9c3f --- /dev/null +++ b/ansible/arcodange/factory/playbooks/07_mirrors.yml @@ -0,0 +1,60 @@ +--- +# Miroirs push Gitea → GitHub (et GitLab). +# +# Gitea est la source ; chaque dépôt listé dans `gitea_mirrored_repos` +# (inventory/group_vars/all/gitea.yml) reçoit un miroir push rafraîchi toutes les +# 8 h ET à chaque commit. Les dépôts créés en face le sont en PRIVÉ. +# +# uv run ansible-playbook -i ansible/arcodange/factory/inventory \ +# ansible/arcodange/factory/playbooks/06_mirrors.yml +# +# GitHub seulement (tant que l'espace GitLab personnel n'est pas renseigné) : +# … -e gitea_mirror_gitlab=false +# +# Le jeton Gitea est frappé pour la durée du run puis révoqué en post_tasks. + +- name: Mettre les dépôts Gitea en miroir sur GitHub et GitLab + hosts: localhost + gather_facts: true # gitea_token date son jeton avec ansible_date_time + + roles: + - role: arcodange.factory.gitea_token + tags: + - gitea_mirrors + + tasks: + - name: Poser le miroir de chaque dépôt déclaré + tags: gitea_mirrors + include_role: + name: arcodange.factory.gitea_repo + apply: + tags: gitea_mirrors + vars: + gitea_repo_name: "{{ mirrored_repo.name }}" + gitea_repo_owner: "{{ mirrored_repo.owner }}" + gitea_repo_description: "{{ mirrored_repo.description | default('') }}" + github_owner: "{{ mirrored_repo.github_owner | default(mirrored_repo.owner) }}" + github_owner_is_org: "{{ mirrored_repo.github_owner_is_org | default(true) }}" + gitlab_owner: "{{ mirrored_repo.gitlab_owner | default(mirrored_repo.owner) }}" + gitlab_namespace_id: >- + {{ mirrored_repo.gitlab_namespace + | default(gitlab_personal_namespace_id) + | default(89826881, true) }} + # Ce qui sort du homelab reste privé en face. + github_repo_private: true + gitlab_repo_visibility: private + loop: "{{ gitea_mirrored_repos }}" + loop_control: + loop_var: mirrored_repo + label: "{{ mirrored_repo.owner }}/{{ mirrored_repo.name }}" + + post_tasks: + - name: Révoquer le jeton Gitea du run + tags: + - gitea_mirrors + include_role: + name: arcodange.factory.gitea_token + apply: + tags: gitea_mirrors + vars: + gitea_token_delete: true diff --git a/ansible/arcodange/factory/roles/gitea_repo/defaults/main.yml b/ansible/arcodange/factory/roles/gitea_repo/defaults/main.yml index a07a87e..75c7dd9 100644 --- a/ansible/arcodange/factory/roles/gitea_repo/defaults/main.yml +++ b/ansible/arcodange/factory/roles/gitea_repo/defaults/main.yml @@ -6,4 +6,38 @@ gitea_username: arcodange gitea_organization: arcodange-org # URL de base du serveur Gitea -gitea_base_url: http://{{ groups.gitea[0] }}:3000 \ No newline at end of file +gitea_base_url: http://{{ groups.gitea[0] }}:3000 + +# Propriétaire du dépôt CÔTÉ GITEA. Par défaut l'organisation, pour ne rien +# changer aux dépôts déjà en miroir ; à surcharger (« arcodange ») pour les +# dépôts qui vivent sous le compte personnel. +gitea_repo_owner: "{{ gitea_organization }}" + +# Propriétaires en FACE, forge par forge. Ils suivent le propriétaire Gitea par +# défaut, mais un dépôt personnel peut viser un compte personnel. +github_owner: "{{ github_organization }}" +gitlab_owner: "{{ gitlab_root_group }}" + +# Un compte personnel n'est pas une organisation : GitHub ne crée pas un dépôt +# au même endroit (POST /user/repos contre POST /orgs//repos). +github_owner_is_org: true + +# Identifiant du groupe OU de l'utilisateur GitLab qui accueille le projet. +# https://gitlab.com/groups/arcodange-org/-/edit +gitlab_namespace_id: 89826881 + +# Quelles forges recevoir en miroir. GitLab devient facultatif : sans ça, un +# échec côté GitLab avorte toute l'itération, y compris la partie GitHub. +gitea_mirror_github: true +gitea_mirror_gitlab: true + +# Le miroir pousse ; c'est le dépôt d'en face qui doit être privé. +github_repo_private: true +gitlab_repo_visibility: private + +# Nom d'utilisateur porté par le miroir push (le mot de passe est le jeton). +github_mirror_username: "{{ gitea_username }}" +gitlab_mirror_username: "{{ gitea_username }}" + +# Cadence de rafraîchissement des miroirs, en plus du push à chaque commit. +gitea_mirror_interval: "8h" diff --git a/ansible/arcodange/factory/roles/gitea_repo/tasks/main.yml b/ansible/arcodange/factory/roles/gitea_repo/tasks/main.yml index 2dcd952..57ab1f0 100644 --- a/ansible/arcodange/factory/roles/gitea_repo/tasks/main.yml +++ b/ansible/arcodange/factory/roles/gitea_repo/tasks/main.yml @@ -1,6 +1,6 @@ - name: Vérifier si le dépôt existe dans Gitea uri: - url: "{{ gitea_base_url }}/api/v1/repos/{{ gitea_organization }}/{{ gitea_repo_name }}" + url: "{{ gitea_base_url }}/api/v1/repos/{{ gitea_repo_owner }}/{{ gitea_repo_name }}" method: GET headers: Authorization: "token {{ gitea_api_token }}" @@ -10,26 +10,36 @@ - name: Vérifier si le dépôt existe sur GitLab uri: - url: "https://gitlab.com/api/v4/projects/{{ gitlab_root_group }}%2F{{ gitea_repo_name }}" + url: "https://gitlab.com/api/v4/projects/{{ gitlab_owner }}%2F{{ gitea_repo_name }}" method: GET headers: Authorization: "Bearer {{ gitlab_api_token }}" status_code: 200 register: gitlab_repo_check ignore_errors: yes + when: gitea_mirror_gitlab | bool - name: Vérifier si le dépôt existe sur GitHub uri: - url: "https://api.github.com/repos/{{ github_organization }}/{{ gitea_repo_name }}" + url: "https://api.github.com/repos/{{ github_owner }}/{{ gitea_repo_name }}" method: GET headers: Authorization: "token {{ github_api_token }}" status_code: 200 register: github_repo_check ignore_errors: yes + when: gitea_mirror_github | bool + +# Une tâche sautée n'enregistre pas de « status » : sans le default(0), la +# condition suivante explose dès qu'une forge est désactivée. +- name: Retenir l'état de chaque forge + set_fact: + gitlab_repo_present: "{{ (gitlab_repo_check.status | default(0)) == 200 }}" + github_repo_present: "{{ (github_repo_check.status | default(0)) == 200 }}" + gitea_repo_present: "{{ (gitea_repo_check.status | default(0)) == 200 }}" - name: Importer un dépôt GitLab/GitHub vers Gitea - when: gitea_repo_check.status != 200 and (gitlab_repo_check.status == 200 or github_repo_check.status == 200) + when: not gitea_repo_present and (gitlab_repo_present or github_repo_present) uri: url: "{{ gitea_base_url }}/api/v1/repos/migrate" method: POST @@ -38,16 +48,16 @@ status_code: 201 body_format: json body: - service: "{{ (gitlab_repo_check.status == 200) | ternary('gitlab','github') }}" + service: "{{ gitlab_repo_present | ternary('gitlab','github') }}" # URL du dépôt GitHub/GitLab clone_addr: >- - {{ (gitlab_repo_check.status == 200) | ternary(gitlab_mirror_url,github_mirror_url) }} + {{ gitlab_repo_present | ternary(gitlab_mirror_url,github_mirror_url) }} auth_username: "{{ gitea_username }}" # Nom d'utilisateur pour l'authentification si nécessaire # token d'accès auth_token: >- - {{ (gitlab_repo_check.status == 200) | ternary(gitlab_api_token,github_api_token) }} + {{ gitlab_repo_present | ternary(gitlab_api_token,github_api_token) }} repo_name: "{{ gitea_repo_name }}" # Nom du dépôt dans Gitea - repo_owner: "{{ github_organization }}" # Propriétaire du dépôt dans Gitea (utilisateur ou organisation + repo_owner: "{{ gitea_repo_owner }}" # Propriétaire du dépôt dans Gitea (utilisateur ou organisation) mirror: false # Activer le mirroring pour synchroniser les changements register: migration_result @@ -66,15 +76,20 @@ body: name: "{{ gitea_repo_name }}" path: "{{ gitea_repo_name }}" - namespace_id: "{{ gitlab_namespace_id }}" # Remplacez par l'ID du groupe ou de l'utilisateur où le projet doit être créé - visibility: "{{ gitlab_repo_visibility | default('private') }}" # Définir la visibilité (private, internal, public) + namespace_id: "{{ gitlab_namespace_id }}" # ID du groupe ou de l'utilisateur où le projet doit être créé + visibility: "{{ gitlab_repo_visibility }}" # Définir la visibilité (private, internal, public) description: "{{ gitea_repo_description | default('') }}" status_code: 201 - when: gitlab_repo_check.status != 200 + when: (gitea_mirror_gitlab | bool) and not gitlab_repo_present +# Un compte personnel n'a pas d'endpoint /orgs//repos : GitHub crée alors +# le dépôt sous le compte porteur du jeton, via POST /user/repos. - name: Créer un dépôt sur GitHub si nécessaire uri: - url: "https://api.github.com/orgs/{{ github_organization }}/repos" + url: >- + {{ (github_owner_is_org | bool) + | ternary('https://api.github.com/orgs/' ~ github_owner ~ '/repos', + 'https://api.github.com/user/repos') }} method: POST headers: Authorization: "token {{ github_api_token }}" @@ -82,13 +97,13 @@ body: name: "{{ gitea_repo_name }}" description: "{{ gitea_repo_description | default('') }}" - private: "{{ github_repo_private | default(true) }}" # Définir si le dépôt est privé ou public + private: "{{ github_repo_private | bool }}" # Définir si le dépôt est privé ou public status_code: 201 - when: github_repo_check.status != 200 + when: (gitea_mirror_github | bool) and not github_repo_present - name: Vérifier l'existence des miroirs push sur GitHub et GitLab uri: - url: "{{ gitea_base_url }}/api/v1/repos/{{ gitea_organization }}/{{ gitea_repo_name }}/push_mirrors" + url: "{{ gitea_base_url }}/api/v1/repos/{{ gitea_repo_owner }}/{{ gitea_repo_name }}/push_mirrors" method: GET headers: Authorization: "token {{ gitea_api_token }}" @@ -97,32 +112,68 @@ - name: Ajouter un miroir push vers GitHub si nécessaire uri: - url: "{{ gitea_base_url }}/api/v1/repos/{{ gitea_organization }}/{{ gitea_repo_name }}/push_mirrors" + url: "{{ gitea_base_url }}/api/v1/repos/{{ gitea_repo_owner }}/{{ gitea_repo_name }}/push_mirrors" method: POST headers: Authorization: "token {{ gitea_api_token }}" body_format: json body: - interval: "8h" + interval: "{{ gitea_mirror_interval }}" remote_address: "{{ github_mirror_url }}" - remote_username: "{{ gitea_username }}" + remote_username: "{{ github_mirror_username }}" remote_password: "{{ github_api_token }}" sync_on_commit: true status_code: 200 - when: "github_mirror_url not in existing_mirrors.json | map(attribute='remote_address') | list" + when: + - gitea_mirror_github | bool + - github_mirror_url not in existing_mirrors.json | map(attribute='remote_address') | list - name: Ajouter un miroir push vers GitLab si nécessaire uri: - url: "{{ gitea_base_url }}/api/v1/repos/{{ gitea_organization }}/{{ gitea_repo_name }}/push_mirrors" + url: "{{ gitea_base_url }}/api/v1/repos/{{ gitea_repo_owner }}/{{ gitea_repo_name }}/push_mirrors" method: POST headers: Authorization: "token {{ gitea_api_token }}" body_format: json body: - interval: "8h" + interval: "{{ gitea_mirror_interval }}" remote_address: "{{ gitlab_mirror_url }}" - remote_username: "{{ gitea_username }}" + remote_username: "{{ gitlab_mirror_username }}" remote_password: "{{ gitlab_api_token }}" sync_on_commit: true status_code: 200 - when: "gitlab_mirror_url not in existing_mirrors.json | map(attribute='remote_address') | list" + when: + - gitea_mirror_gitlab | bool + - gitlab_mirror_url not in existing_mirrors.json | map(attribute='remote_address') | list + +# Un dépôt GitHub créé vide adopte comme branche par défaut la PREMIÈRE branche +# que le miroir lui pousse — souvent une branche de travail, pas « main ». Le +# miroir étant asynchrone, l'alignement échoue au run qui crée le dépôt et +# réussit au suivant : d'où le failed_when permissif plutôt qu'un blocage. +- name: Aligner la branche par défaut de GitHub sur celle de Gitea + uri: + url: "https://api.github.com/repos/{{ github_owner }}/{{ gitea_repo_name }}" + method: PATCH + headers: + Authorization: "token {{ github_api_token }}" + body_format: json + body: + default_branch: "{{ gitea_repo_check.json.default_branch }}" + status_code: 200 + register: github_default_branch + failed_when: false + when: + - gitea_mirror_github | bool + - gitea_repo_present + - (github_repo_check.json.default_branch | default('')) != gitea_repo_check.json.default_branch + +- name: Signaler une branche par défaut encore désalignée + debug: + msg: >- + La branche par défaut de github.com/{{ github_owner }}/{{ gitea_repo_name }} + n'a pas pu être alignée sur « {{ gitea_repo_check.json.default_branch }} » : + le miroir ne l'a probablement pas encore poussée. Relancer après la synchro. + when: + - github_default_branch is defined + - github_default_branch is not skipped + - (github_default_branch.status | default(0)) != 200 diff --git a/ansible/arcodange/factory/roles/gitea_repo/vars/main.yml b/ansible/arcodange/factory/roles/gitea_repo/vars/main.yml index 2226e28..1bc7dc6 100644 --- a/ansible/arcodange/factory/roles/gitea_repo/vars/main.yml +++ b/ansible/arcodange/factory/roles/gitea_repo/vars/main.yml @@ -3,8 +3,8 @@ gitlab_api_token: '{{ hostvars[groups.gitea[0]].gitea_vault.gitlab_api_token }}' github_organization: '{{ gitea_organization }}' gitlab_root_group: '{{ gitea_organization }}' -gitlab_namespace_id: 89826881 # https://gitlab.com/groups/arcodange-org/-/edit -# URLs des miroirs sur GitLab et GitHub -gitlab_mirror_url: "https://gitlab.com/{{ gitlab_root_group | default(gitlab_username | default(gitea_username)) }}/{{ gitea_repo_name }}.git" -github_mirror_url: "https://github.com/{{ github_organization | default(github_username | default(gitea_username)) }}/{{ gitea_repo_name }}.git" +# URLs des miroirs sur GitLab et GitHub — elles suivent le propriétaire visé sur +# chaque forge (cf. github_owner / gitlab_owner dans defaults/). +gitlab_mirror_url: "https://gitlab.com/{{ gitlab_owner }}/{{ gitea_repo_name }}.git" +github_mirror_url: "https://github.com/{{ github_owner }}/{{ gitea_repo_name }}.git" diff --git a/ansible/arcodange/factory/roles/gitea_sync/defaults/main.yml b/ansible/arcodange/factory/roles/gitea_sync/defaults/main.yml index 5bc0185..614ba79 100644 --- a/ansible/arcodange/factory/roles/gitea_sync/defaults/main.yml +++ b/ansible/arcodange/factory/roles/gitea_sync/defaults/main.yml @@ -4,4 +4,18 @@ gitea_organization: arcodange-org # URL de base du serveur Gitea gitea_base_url: http://{{ groups.gitea[0] }}:3000 -gitea_token_fact_name: arcodange_factory_gitea_sync_token \ No newline at end of file +gitea_token_fact_name: arcodange_factory_gitea_sync_token + +# Propriétaire balayé. Par défaut l'organisation ; mettre « arcodange » et +# gitea_sync_owner_is_org à false pour balayer le compte personnel. +gitea_sync_owner: "{{ gitea_organization }}" +gitea_sync_owner_is_org: true + +# Les trois API paginent (30 par défaut chez GitHub, 20 chez GitLab). Sous la +# taille d'une page, la différence entre forges désigne de faux dépôts manquants. +gitea_sync_page_size: 100 + +# Forges comparées. Balayer une forge qu'on ne veut pas alimenter ferait passer +# tous ses dépôts pour « incomplets ». +gitea_mirror_github: true +gitea_mirror_gitlab: true diff --git a/ansible/arcodange/factory/roles/gitea_sync/tasks/main.yml b/ansible/arcodange/factory/roles/gitea_sync/tasks/main.yml index a54b5df..f254f56 100644 --- a/ansible/arcodange/factory/roles/gitea_sync/tasks/main.yml +++ b/ansible/arcodange/factory/roles/gitea_sync/tasks/main.yml @@ -1,40 +1,71 @@ +# Un compte personnel n'est pas une organisation : ni GitHub ni GitLab ne +# servent ses dépôts au même endroit. - name: Lister les dépôts de l'organisation GitHub uri: - url: "https://api.github.com/orgs/{{ github_organization }}/repos" + url: >- + {{ (gitea_sync_owner_is_org | bool) + | ternary('https://api.github.com/orgs/' ~ github_owner ~ '/repos', + 'https://api.github.com/users/' ~ github_owner ~ '/repos') + }}?per_page={{ gitea_sync_page_size }} method: GET headers: Authorization: "token {{ github_api_token }}" status_code: 200 register: github_repos + when: gitea_mirror_github | bool - name: Lister les dépôts du groupe GitLab uri: - url: "https://gitlab.com/api/v4/groups/{{ gitlab_root_group }}/projects" + url: >- + {{ (gitea_sync_owner_is_org | bool) + | ternary('https://gitlab.com/api/v4/groups/' ~ gitlab_owner ~ '/projects', + 'https://gitlab.com/api/v4/users/' ~ gitlab_owner ~ '/projects') + }}?per_page={{ gitea_sync_page_size }} method: GET headers: Authorization: "Bearer {{ gitlab_api_token }}" status_code: 200 register: gitlab_repos + when: gitea_mirror_gitlab | bool - name: Lister les dépôts de l'organisation Gitea uri: - url: "{{ gitea_base_url }}/api/v1/orgs/{{ gitea_organization }}/repos" + url: >- + {{ (gitea_sync_owner_is_org | bool) + | ternary(gitea_base_url ~ '/api/v1/orgs/' ~ gitea_sync_owner ~ '/repos', + gitea_base_url ~ '/api/v1/users/' ~ gitea_sync_owner ~ '/repos') + }}?limit={{ gitea_sync_page_size }} method: GET headers: Authorization: "token {{ gitea_api_token }}" status_code: 200 register: gitea_repos +# Une forge désactivée ne doit pas peser dans la différence : on la remplace par +# la liste Gitea elle-même, qui la rend neutre à l'intersection. +- name: Établir la liste des dépôts incomplets + set_fact: + gitea_repo_names: "{{ gitea_repos.json | map(attribute='name') | list }}" + github_repo_names: >- + {{ (gitea_mirror_github | bool) + | ternary(github_repos.json | default([]) | map(attribute='name') | list, + gitea_repos.json | map(attribute='name') | list) }} + gitlab_repo_names: >- + {{ (gitea_mirror_gitlab | bool) + | ternary(gitlab_repos.json | default([]) | map(attribute='name') | list, + gitea_repos.json | map(attribute='name') | list) }} + +- name: Réduire aux dépôts absents d'au moins une forge + set_fact: + repos_incomplete: >- + {{ (github_repo_names | union(gitlab_repo_names) | union(gitea_repo_names)) + | difference(github_repo_names | intersect(gitlab_repo_names) | intersect(gitea_repo_names)) }} + - name: Synchroniser include_role: name: arcodange.factory.gitea_repo vars: - github_repo_names: "{{ github_repos.json | map(attribute='name') | list }}" - gitlab_repo_names: "{{ gitlab_repos.json | map(attribute='name') | list }}" - gitea_repo_names: "{{ gitea_repos.json | map(attribute='name') | list }}" - all_repos: "{{ github_repo_names | union(gitlab_repo_names) | union(gitea_repo_names) }}" - repos_common_to_all: "{{ github_repo_names | intersect(gitlab_repo_names) | intersect(gitea_repo_names) }}" - repos_incomplete: "{{ all_repos | difference(repos_common_to_all) }}" + gitea_repo_owner: "{{ gitea_sync_owner }}" loop: "{{ repos_incomplete }}" loop_control: loop_var: gitea_repo_name diff --git a/ansible/arcodange/factory/roles/gitea_sync/vars/main.yml b/ansible/arcodange/factory/roles/gitea_sync/vars/main.yml index 8b44b9b..e08cadb 100644 --- a/ansible/arcodange/factory/roles/gitea_sync/vars/main.yml +++ b/ansible/arcodange/factory/roles/gitea_sync/vars/main.yml @@ -2,4 +2,8 @@ github_api_token: '{{ hostvars[groups.gitea[0]].gitea_vault.github_api_token }}' gitlab_api_token: '{{ hostvars[groups.gitea[0]].gitea_vault.gitlab_api_token }}' github_organization: '{{ gitea_organization }}' -gitlab_root_group: '{{ gitea_organization }}' \ No newline at end of file +gitlab_root_group: '{{ gitea_organization }}' + +# Les propriétaires en face suivent celui qu'on balaie côté Gitea. +github_owner: '{{ gitea_sync_owner }}' +gitlab_owner: '{{ gitea_sync_owner }}' diff --git a/vibe/guidebooks/factory-provisioning/ansible/07-mirrors.md b/vibe/guidebooks/factory-provisioning/ansible/07-mirrors.md new file mode 100644 index 0000000..227fa7c --- /dev/null +++ b/vibe/guidebooks/factory-provisioning/ansible/07-mirrors.md @@ -0,0 +1,81 @@ +[vibe](../../../README.md) > [Guidebooks](../../README.md) > [Factory provisioning](../README.md) > [Ansible](README.md) > **07 · Mirrors** + +# 07 · Mirrors — Gitea → GitHub / GitLab + +> [!NOTE] +> **Status:** ✅ active · **Last Updated:** 2026-07-27 +> **Upstream:** [Ansible sub-hub](README.md) · [Factory provisioning hub](../README.md) +> **Downstream:** [Roles reference](roles.md) — `gitea_repo`, `gitea_sync`, `gitea_token` +> **Related:** [Inventory & variables](inventory.md) · [03 · CI/CD](03-cicd.md) + +Gitea is the **source of truth**; GitHub and GitLab hold a pushed copy. [`playbooks/07_mirrors.yml`](../../../../ansible/arcodange/factory/playbooks/07_mirrors.yml) walks the repos declared in `gitea_mirrored_repos` ([`inventory/group_vars/all/gitea.yml`](../../../../ansible/arcodange/factory/inventory/group_vars/all/gitea.yml)) and, for each, calls [`gitea_repo`](../../../../ansible/arcodange/factory/roles/gitea_repo): create the counterpart repo **private** if it is missing, then attach a push mirror refreshed every **8 h** *and* on **every commit**. + +Nothing is pulled back. A mirror only ever pushes Gitea → forge, so a change made on GitHub is overwritten at the next sync. + +```sh +uv run ansible-playbook -i ansible/arcodange/factory/inventory \ + ansible/arcodange/factory/playbooks/07_mirrors.yml + +# GitHub only — while the personal GitLab namespace is still unset: +… -e gitea_mirror_gitlab=false +``` + +The Gitea token is minted for the run by `gitea_token` and **revoked in `post_tasks`**. Everything is tagged `gitea_mirrors`. + +--- + +## Two ways to pick repos, and when each fits + +| | [`gitea_sync`](../../../../ansible/arcodange/factory/roles/gitea_sync) | `gitea_mirrored_repos` + `07_mirrors.yml` | +| --- | --- | --- | +| Selection | Automatic: diffs the three forges for **one owner**, reconciles whatever is missing somewhere | Explicit list, reviewed in the inventory | +| Fits | The organisation, where every repo is meant to exist everywhere | The personal account, where each repo leaving the homelab is a deliberate call | +| Blind spot | `repos_incomplete = all − common` says nothing about *why* a repo is missing — a repo deleted on purpose from GitHub is recreated | Anything absent from the list is silently never mirrored | + +Both drive the same `gitea_repo` role, so the mirror they produce is identical. + +--- + +## Owner mapping + +A Gitea repo owned by the **user** `arcodange` does not belong on the GitHub **organisation** — and GitHub does not even create it the same way (`POST /user/repos` instead of `POST /orgs//repos`). Hence three knobs, all defaulting to the previous org-only behaviour: + +| Var | Default | Meaning | +| --- | --- | --- | +| `gitea_repo_owner` | `gitea_organization` | Owner **on Gitea** | +| `github_owner` / `gitlab_owner` | `github_organization` / `gitlab_root_group` | Owner **on the far forge** | +| `github_owner_is_org` | `true` | `false` routes creation to `POST /user/repos` | +| `gitea_mirror_github` / `gitea_mirror_gitlab` | `true` | Turn a forge off entirely | + +> [!IMPORTANT] +> GitLab was **not optional** before. Its create call expected `201` with no `ignore_errors`, so a GitLab failure aborted the iteration — including the GitHub half that had nothing to do with it. `gitea_mirror_gitlab: false` is the way out. + +> [!WARNING] +> A GitHub repo created **empty** adopts as its default branch the *first branch the mirror pushes*, which is routinely a work branch rather than `main`. The role realigns it against Gitea's default branch, but the mirror is asynchronous: the alignment fails on the run that creates the repo and succeeds on the next one. Run the playbook twice, or fix the branch by hand. + +--- + +## Current state (2026-07-27) + +| Owner | Repos mirrored | Target | +| --- | --- | --- | +| `arcodange-org` | 10 (`factory`, `tools`, `erp`, `cms`, `webapp`, `url-shortener`, `docker.tofu`, `docker-build-workflow`, `super-linter-workflow`, `vault-action`) | `github.com/arcodange-org/*` + GitLab | +| `arcodange` (user) | 5 (`kadans`, `kadans-api`, `kadans-dossier`, `kadans-jobs`, `video_analysis`) — all **private** | `github.com/arcodange/*` | + +Not mirrored, deliberately left out of `gitea_mirrored_repos`: `documents`, `studio`, `prospection`, `kissmetrics_contract_proposal` (org) and `.profile`, `DanceVideos`, `SecondBrain`, `dance-lessons-coach`, `frame-sdk`, `telegram-gateway` (user). + +> [!NOTE] +> The personal repos have **no GitLab mirror yet**: `gitlab_personal_namespace_id` is still `~`. Fill it with the numeric namespace ID of the `arcodange` account on gitlab.com, otherwise creation would land the project in the `arcodange-org` group. + +--- + +## Reading the truth from Gitea + +The push mirrors live in Gitea, not in this repo — the playbook is idempotent precisely because it asks first: + +```sh +curl -s -H "Authorization: token $GITEA_TOKEN" \ + https://gitea.arcodange.lab/api/v1/repos/arcodange/kadans/push_mirrors +``` + +`last_update` tells you when the mirror last pushed. A repo with no entry has no mirror, whatever this page claims. diff --git a/vibe/guidebooks/factory-provisioning/ansible/README.md b/vibe/guidebooks/factory-provisioning/ansible/README.md index 9ec705c..8d8df8f 100644 --- a/vibe/guidebooks/factory-provisioning/ansible/README.md +++ b/vibe/guidebooks/factory-provisioning/ansible/README.md @@ -5,7 +5,7 @@ > [!NOTE] > **Status:** ✅ active · **Last Updated:** 2026-06-23 > **Upstream:** [Factory provisioning hub](../README.md) · [Lab ecosystem · 01 factory](../../lab-ecosystem/01-factory.md) -> **Downstream:** [01 · System](01-system.md) · [02 · Setup](02-setup.md) · [03 · CI/CD](03-cicd.md) · [04 · Tools](04-tools.md) · [05 · Backup](05-backup.md) · [06 · Recover](06-recover.md) · [Inventory & variables](inventory.md) · [Roles reference](roles.md) +> **Downstream:** [01 · System](01-system.md) · [02 · Setup](02-setup.md) · [03 · CI/CD](03-cicd.md) · [04 · Tools](04-tools.md) · [05 · Backup](05-backup.md) · [06 · Recover](06-recover.md) · [07 · Mirrors](07-mirrors.md) · [Inventory & variables](inventory.md) · [Roles reference](roles.md) > **Related:** [Secrets & Vault](../../lab-ecosystem/secrets-and-vault.md) · [Storage & recovery](../../lab-ecosystem/storage-and-recovery.md) · [Naming conventions](../../lab-ecosystem/naming-conventions.md) · [ADR-0001 safe prod-like environment](../../../ADR/0001-safe-prod-like-environment.md) Ansible is the **imperative half** of the factory: it takes three bare Raspberry Pis (`pi1`, `pi2`, `pi3`) and turns them into a running K3s cluster with Docker, Longhorn storage, Gitea CI runners, CrowdSec, and Vault. OpenTofu (the declarative half) then provisions everything that lives *outside* the cluster — see the [OpenTofu sub-hub](../opentofu/README.md). @@ -22,7 +22,7 @@ Everything ships as a single Ansible **collection** committed under [`ansible/ar | `requirements.yml` | [`ansible/requirements.yml`](../../../../ansible/requirements.yml) | External dependencies pulled at install time (see table below). | | `ansible.cfg` | [`ansible/arcodange/factory/ansible.cfg`](../../../../ansible/arcodange/factory/ansible.cfg) | `collections_path = ~/.ansible/collections` and `scp_if_ssh = True` for the SSH connection plugin. | | `inventory/` | [`ansible/arcodange/factory/inventory/`](../../../../ansible/arcodange/factory/inventory) | `hosts.yml` + `group_vars/`. Detailed in [Inventory & variables](inventory.md). | -| `playbooks/` | [`ansible/arcodange/factory/playbooks/`](../../../../ansible/arcodange/factory/playbooks) | The numbered pipeline `01..05` plus the `recover/` branch. | +| `playbooks/` | [`ansible/arcodange/factory/playbooks/`](../../../../ansible/arcodange/factory/playbooks) | The numbered pipeline `01..05`, the `recover/` branch, and the on-demand [`07_mirrors.yml`](../../../../ansible/arcodange/factory/playbooks/07_mirrors.yml). | | `roles/` | [`ansible/arcodange/factory/roles/`](../../../../ansible/arcodange/factory/roles) | Seven reusable roles. Detailed in [Roles reference](roles.md). | ### External dependencies (`requirements.yml`) diff --git a/vibe/guidebooks/factory-provisioning/ansible/roles.md b/vibe/guidebooks/factory-provisioning/ansible/roles.md index b334e83..07cc159 100644 --- a/vibe/guidebooks/factory-provisioning/ansible/roles.md +++ b/vibe/guidebooks/factory-provisioning/ansible/roles.md @@ -126,10 +126,10 @@ Smaller roles, mostly Gitea/forge plumbing and one-shot helpers. Shared roles li | Role | Purpose | Key vars / notes | Secrets | | --- | --- | --- | --- | -| [`gitea_repo`](../../../../ansible/arcodange/factory/roles/gitea_repo) | Ensure a repo exists across Gitea + GitHub + GitLab and add **8h push mirrors** (`sync_on_commit: true`) to GitHub/GitLab. | Creates missing repos on each forge; mirror URLs + namespace IDs in [`vars/main.yml`](../../../../ansible/arcodange/factory/roles/gitea_repo/vars/main.yml). | `github_api_token`, `gitlab_api_token` (from `gitea_vault`). | +| [`gitea_repo`](../../../../ansible/arcodange/factory/roles/gitea_repo) | Ensure a repo exists across Gitea + GitHub + GitLab and add **8h push mirrors** (`sync_on_commit: true`) to GitHub/GitLab. | Creates missing repos on each forge (**private** by default). Owner is per-forge — `gitea_repo_owner` / `github_owner` / `gitlab_owner`, with `github_owner_is_org: false` for a personal account. Each forge can be switched off (`gitea_mirror_github` / `gitea_mirror_gitlab`). See [07 · Mirrors](07-mirrors.md). | `github_api_token`, `gitlab_api_token` (from `gitea_vault`). | | [`gitea_token`](../../../../ansible/arcodange/factory/roles/gitea_token) | Generate / replace / delete a Gitea access token via `docker exec … gitea admin user generate-access-token`. | Stores the raw token in the fact named by `gitea_token_fact_name`; `gitea_token_replace` / `gitea_token_delete` toggles; scopes default to `write:admin,organization,package,repository,user`. | The minted token itself (a fact, not persisted). | | [`gitea_secret`](../../../../ansible/arcodange/factory/roles/gitea_secret) | `PUT` a Gitea **Actions secret** at user or org scope. | `gitea_secret_name` / `_value`; `gitea_owner_type` (`user`\|`org`) selects the API path. | `gitea_api_token` (Authorization). | -| [`gitea_sync`](../../../../ansible/arcodange/factory/roles/gitea_sync) | List repos on all **three forges**, diff them, and call `gitea_repo` for the repos missing somewhere. | Computes `repos_incomplete = all − common`; loops `gitea_repo` over the gaps. | GitHub/GitLab/Gitea API tokens. | +| [`gitea_sync`](../../../../ansible/arcodange/factory/roles/gitea_sync) | List repos on all **three forges** for **one owner**, diff them, and call `gitea_repo` for the repos missing somewhere. | Computes `repos_incomplete = all − common`; loops `gitea_repo` over the gaps. `gitea_sync_owner` + `gitea_sync_owner_is_org` pick the owner (a user is not served at the same API paths). **Not currently invoked by any playbook** — the explicit list in [07 · Mirrors](07-mirrors.md) is what runs. | GitHub/GitLab/Gitea API tokens. | | [`traefik_certs`](../../../../ansible/arcodange/factory/roles/traefik_certs) | Extract the live **`*.arcodange.lab`** cert from Traefik's `acme.json`. | `kubectl exec` into Traefik → `jq` the LetsEncrypt wildcard cert → `traefik_cert_pem` fact; no-op if already set. | — (reads in-cluster acme.json). | | [`playwright`](../../../../ansible/arcodange/factory/roles/playwright) | Run a Playwright browser-automation script in Docker. | Builds `playwright:` (default `1.47.0`) from `files/`, runs the script with `playwright_env` injected as `-e`; default script `loginGitea.js`. Used by `hashicorp_vault` for the OIDC app setup. | Script-specific env (e.g. Gitea admin creds). | | [`deploy_gitea`](../../../../ansible/arcodange/factory/playbooks/setup/roles/deploy_gitea) | Deploy Gitea: template [`app.ini.j2`](../../../../ansible/arcodange/factory/playbooks/setup/roles/deploy_gitea/tasks/main.yml), `docker compose up`, then **health-check `:3000`** until ready. | Compose source is `/home/pi/arcodange/docker_composes/gitea`; admin user `arcodange`. | (consumes the vaulted Gitea compose env). | -- 2.54.0