Compare commits
42 Commits
be0a31a525
...
1138d35eeb
| Author | SHA1 | Date | |
|---|---|---|---|
| 1138d35eeb | |||
| 4473a55ea8 | |||
| 473dcbdbbb | |||
| 6a295bccc9 | |||
| 930e9ac159 | |||
| e27ed5b6ca | |||
| a9e2f08a28 | |||
| 623f822fde | |||
| 411e7210be | |||
| 264c4fe7df | |||
| c8d3e86986 | |||
| f4b34b688d | |||
| ef1e5c0757 | |||
| 706848c3d5 | |||
| 9f8b090164 | |||
| 37f839009c | |||
| 2404983627 | |||
| 9937f814f6 | |||
| ab9bdef81d | |||
| 8ac9261a81 | |||
| 1843bc968f | |||
| 968d9956e9 | |||
| 3cafc7bcb2 | |||
| dd1e95bbd5 | |||
| 07b34ab1e7 | |||
| 1c63530580 | |||
| 1cd3235404 | |||
| 4559ea3b82 | |||
| c5e1b61eb4 | |||
| 1f8c5450d5 | |||
| ebc131f33b | |||
| e98b082ec5 | |||
| 870529964c | |||
| 6055d444f8 | |||
| 13f32378e6 | |||
| 28002070bc | |||
| 4a7edf5218 | |||
| a8533a7a15 | |||
| 7e8a1f3ae7 | |||
| 385b016703 | |||
| 7805b3d2ad | |||
| af5165c2a7 |
1
.gitea/test_workflow_trigger.md
Normal file
1
.gitea/test_workflow_trigger.md
Normal file
@@ -0,0 +1 @@
|
||||
# Workflow test
|
||||
234
.gitea/workflows/README.md
Normal file
234
.gitea/workflows/README.md
Normal file
@@ -0,0 +1,234 @@
|
||||
# CI/CD Workflow Architecture
|
||||
|
||||
## 🗺️ Overview
|
||||
|
||||
The dance-lessons-coach project uses a **multi-workflow architecture** for better separation of concerns, maintainability, and flexibility.
|
||||
|
||||
## 📁 Workflow Files
|
||||
|
||||
### 1. `ci-cd.yaml` - Main CI/CD Pipeline
|
||||
|
||||
**Purpose**: Run tests, build binaries, and generate documentation
|
||||
|
||||
**Triggers**:
|
||||
- Push to `main`, `ci/**`, `feature/**`, `fix/**`, `refactor/**` branches
|
||||
- Pull requests to `main` branch
|
||||
- Manual workflow dispatch
|
||||
|
||||
**Jobs**:
|
||||
1. **build-cache** - Build and cache Docker build environment
|
||||
2. **ci-pipeline** - Run tests, build binaries, generate Swagger docs
|
||||
3. **trigger-docker-push** - Trigger separate Docker workflow on main branch
|
||||
|
||||
**Key Features**:
|
||||
- Runs in container environment with all build tools
|
||||
- Generates Swagger documentation
|
||||
- Runs BDD and unit tests with PostgreSQL
|
||||
- Updates badges and version information
|
||||
- Triggers Docker workflow only on main branch
|
||||
|
||||
### 2. `docker-push.yaml` - Docker Image Publishing
|
||||
|
||||
**Purpose**: Build and push Docker images to registry
|
||||
|
||||
**Triggers**:
|
||||
- Manual workflow dispatch only (no automatic triggers)
|
||||
- Triggered by `ci-cd.yaml` on main branch
|
||||
|
||||
**Jobs**:
|
||||
1. **docker-push** - Build production Docker image and push to registry
|
||||
|
||||
**Key Features**:
|
||||
- Runs on host environment (access to Docker daemon)
|
||||
- Uses dependency hash from build-cache
|
||||
- Builds minimal Alpine-based production image
|
||||
- Pushes multiple tags (version, latest, commit SHA)
|
||||
|
||||
## 🔧 Architecture Benefits
|
||||
|
||||
### 1. Clear Separation of Concerns
|
||||
- **CI/CD Pipeline**: Testing and artifact generation
|
||||
- **Docker Publishing**: Image building and registry operations
|
||||
|
||||
### 2. Proper Environment Isolation
|
||||
- **CI jobs run in container**: Consistent build environment
|
||||
- **Docker jobs run on host**: Access to Docker daemon
|
||||
|
||||
### 3. Flexible Testing
|
||||
- Can trigger Docker workflow independently for testing
|
||||
- No complex conditional logic in main workflow
|
||||
- Easier to debug and maintain
|
||||
|
||||
### 4. Better Security
|
||||
- Docker operations isolated in separate workflow
|
||||
- Clear dependency between test success and deployment
|
||||
- Manual trigger capability for emergency situations
|
||||
|
||||
## 🚀 Usage Examples
|
||||
|
||||
### Trigger Full CI/CD Pipeline
|
||||
```bash
|
||||
# Automatically triggered on push to main branch
|
||||
# Or manually:
|
||||
./scripts/gitea-client.sh trigger-workflow arcodange dance-lessons-coach ci-cd.yaml main
|
||||
```
|
||||
|
||||
### Trigger Docker Push Manually
|
||||
```bash
|
||||
# Get dependency hash from build-cache job first
|
||||
DEPS_HASH="abc123def456"
|
||||
|
||||
# Trigger Docker workflow manually
|
||||
./scripts/gitea-client.sh trigger-workflow arcodange dance-lessons-coach docker-push.yaml main --deps_hash $DEPS_HASH
|
||||
```
|
||||
|
||||
### Workflow Dispatch Parameters (docker-push.yaml)
|
||||
- `deps_hash` (required): Dependency hash from build-cache job
|
||||
- `ref` (optional): Git reference (branch/tag), defaults to current
|
||||
|
||||
## 🔗 Workflow Dependencies
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Push to main] --> B[ci-cd.yaml]
|
||||
B --> C[build-cache job]
|
||||
B --> D[ci-pipeline job]
|
||||
D --> E[trigger-docker-push job]
|
||||
E --> F[docker-push.yaml]
|
||||
F --> G[docker-push job]
|
||||
G --> H[Docker Registry]
|
||||
```
|
||||
|
||||
## 📋 Best Practices
|
||||
|
||||
### 1. Always Run CI First
|
||||
- Docker workflow should only be triggered after CI passes
|
||||
- Maintains quality gate before deployment
|
||||
|
||||
### 2. Use Dependency Hash
|
||||
- Ensures consistent builds across workflows
|
||||
- Pass hash from build-cache to docker-push
|
||||
|
||||
### 3. Manual Testing
|
||||
- Use separate Docker workflow for testing image builds
|
||||
- Avoids polluting main branch with test images
|
||||
|
||||
### 4. Monitor Both Workflows
|
||||
- CI/CD workflow for test results and artifacts
|
||||
- Docker workflow for image build and push status
|
||||
|
||||
## 🎯 Docker Build Strategy Decision
|
||||
|
||||
### 🏆 Chosen Approach: Attempt 2 (Standard Dockerfile)
|
||||
|
||||
After extensive testing of multiple approaches, we selected **Attempt 2** as the optimal Docker build strategy.
|
||||
|
||||
#### ⚡ Why Attempt 2 Won:
|
||||
|
||||
**1. Simplicity (60% smaller workflow)**
|
||||
- 73 lines vs 158 lines in complex approaches
|
||||
- No inline Dockerfile generation
|
||||
- Standard `docker build -f docker/Dockerfile .` command
|
||||
|
||||
**2. Better Performance**
|
||||
- No artifact/cache action overhead
|
||||
- Natural Docker layer caching works optimally
|
||||
- Faster execution without complex variable substitutions
|
||||
|
||||
**3. Superior Reliability**
|
||||
- Proven standard Docker build process
|
||||
- Easier to debug and maintain
|
||||
- Fewer moving parts = fewer failures
|
||||
|
||||
**4. Better Maintainability**
|
||||
- Uses standard Dockerfile (easier to understand)
|
||||
- No complex YAML templating
|
||||
- Clear separation of concerns
|
||||
|
||||
#### 🗑️ Why We Rejected Other Approaches:
|
||||
|
||||
**Attempt 1 (Inline Dockerfile):**
|
||||
- Complex YAML templating
|
||||
- Harder to debug and maintain
|
||||
- No significant performance benefit
|
||||
|
||||
**Attempt 3 (Build Cache Image):**
|
||||
- Added complexity with cache management
|
||||
- Slower due to artifact actions overhead
|
||||
- More prone to cache invalidation issues
|
||||
|
||||
**Attempt 4 (Template File):**
|
||||
- Added unnecessary file management
|
||||
- No clear advantage over standard Dockerfile
|
||||
- More complex workflow
|
||||
|
||||
### 📊 Performance Comparison:
|
||||
|
||||
| Approach | Lines of Code | Complexity | Reliability | Maintainability |
|
||||
|----------|---------------|------------|-------------|-----------------|
|
||||
| **Attempt 2** | 73 | Low | High | Excellent |
|
||||
| Attempt 1 | 158 | High | Medium | Poor |
|
||||
| Attempt 3 | 125 | Medium | Medium | Fair |
|
||||
| Attempt 4 | 110 | Medium | High | Good |
|
||||
|
||||
### 🔧 Implementation Details:
|
||||
|
||||
**Standard Dockerfile Approach:**
|
||||
```yaml
|
||||
- name: Build and push Docker image
|
||||
run: |
|
||||
docker build -t dance-lessons-coach -f docker/Dockerfile .
|
||||
docker tag dance-lessons-coach "$IMAGE_NAME"
|
||||
docker push "$IMAGE_NAME"
|
||||
```
|
||||
|
||||
**Key Benefits:**
|
||||
- Uses multi-stage builds for optimization
|
||||
- Standard Docker layer caching works naturally
|
||||
- Easy to understand and modify
|
||||
- Proven reliability in production
|
||||
|
||||
## 🎯 Future Enhancements
|
||||
|
||||
### Potential Improvements:
|
||||
- Add workflow status badges to README
|
||||
- Implement workflow chaining with outputs
|
||||
- Add matrix builds for multiple architectures
|
||||
- Implement canary deployment workflow
|
||||
- Add rollback capability
|
||||
|
||||
### Architecture Considerations:
|
||||
- Keep workflows focused on single responsibilities
|
||||
- Maintain clear separation between test and deploy
|
||||
- Document all workflow triggers and conditions
|
||||
- Monitor workflow execution times and optimize
|
||||
|
||||
## 📝 Maintenance
|
||||
|
||||
### Adding New Jobs:
|
||||
- Add to appropriate workflow based on responsibility
|
||||
- CI-related jobs → `ci-cd.yaml`
|
||||
- Docker-related jobs → `docker-push.yaml`
|
||||
|
||||
### Modifying Triggers:
|
||||
- Update trigger conditions in respective workflow files
|
||||
- Test changes thoroughly before merging
|
||||
|
||||
### Debugging:
|
||||
- Check workflow logs in Gitea Actions
|
||||
- Use `gitea-client.sh diagnose-job` for detailed analysis
|
||||
- Monitor workflow dependencies and execution order
|
||||
|
||||
## 🔒 Security
|
||||
|
||||
### Secrets Management:
|
||||
- Docker registry credentials stored in Gitea secrets
|
||||
- Never hardcode credentials in workflow files
|
||||
- Use GitHub token for workflow dispatch
|
||||
|
||||
### Access Control:
|
||||
- Only authorized users can trigger workflows
|
||||
- Manual approval required for production deployments
|
||||
- Audit logs available for all workflow executions
|
||||
|
||||
This architecture provides a clean, maintainable, and secure CI/CD pipeline that scales well with project growth while maintaining clear separation of concerns.
|
||||
@@ -132,7 +132,8 @@ jobs:
|
||||
name: CI Pipeline
|
||||
needs: build-cache
|
||||
runs-on: ubuntu-latest-ca
|
||||
if: "!contains(github.event.head_commit.message, '[skip ci]') && github.actor != 'ci-bot'"
|
||||
# Skip conditions: standard skip ci + actor check + respect skip_ci input
|
||||
if: "!contains(github.event.head_commit.message, '[skip ci]') && github.actor != 'ci-bot' && (!github.event.inputs.skip_ci || github.event.inputs.skip_ci == 'false')"
|
||||
|
||||
container:
|
||||
image: ${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}-build-cache:${{ needs.build-cache.outputs.deps_hash }}
|
||||
@@ -304,47 +305,23 @@ jobs:
|
||||
echo "ℹ️ No changes to push"
|
||||
fi
|
||||
|
||||
# Docker build and push (main branch only)
|
||||
- name: Login to Gitea Container Registry
|
||||
if: github.ref == 'refs/heads/main'
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.CI_REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.PACKAGES_TOKEN }}
|
||||
|
||||
- name: Build and push Docker image
|
||||
if: github.ref == 'refs/heads/main'
|
||||
run: |
|
||||
source VERSION
|
||||
IMAGE_VERSION="$MAJOR.$MINOR.$PATCH${PRERELEASE:+-$PRERELEASE}"
|
||||
|
||||
# Use the template file with proper dependency hash replacement
|
||||
DEPS_HASH="${{ needs.build-cache.outputs.deps_hash }}"
|
||||
echo "Using dependency hash: $DEPS_HASH"
|
||||
|
||||
# Create Dockerfile.prod from template
|
||||
sed "s/{{DEPS_HASH}}/$DEPS_HASH/g" docker/Dockerfile.prod.template > docker/Dockerfile.prod
|
||||
|
||||
TAGS="$IMAGE_VERSION latest ${{ github.sha }}"
|
||||
echo "Building Docker image with tags: $TAGS"
|
||||
|
||||
# Build the production image
|
||||
docker build -t dance-lessons-coach -f docker/Dockerfile.prod .
|
||||
|
||||
for TAG in $TAGS; do
|
||||
IMAGE_NAME="${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}:$TAG"
|
||||
echo "Tagging and pushing: $IMAGE_NAME"
|
||||
docker tag dance-lessons-coach "$IMAGE_NAME"
|
||||
docker push "$IMAGE_NAME"
|
||||
done
|
||||
|
||||
- name: Show published images
|
||||
if: github.ref == 'refs/heads/main'
|
||||
|
||||
# Trigger Docker push workflow on main branch
|
||||
trigger-docker-push:
|
||||
name: Trigger Docker Push
|
||||
needs: [build-cache, ci-pipeline]
|
||||
runs-on: ubuntu-latest-ca
|
||||
if: "!contains(github.event.head_commit.message, '[skip ci]') && github.actor != 'ci-bot' && github.ref == 'refs/heads/main'"
|
||||
|
||||
steps:
|
||||
- name: Trigger Docker Push Workflow
|
||||
run: |
|
||||
source VERSION
|
||||
IMAGE_VERSION="$MAJOR.$MINOR.$PATCH${PRERELEASE:+-$PRERELEASE}"
|
||||
echo "📦 Published Docker images:"
|
||||
echo " - ${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}:$IMAGE_VERSION"
|
||||
echo " - ${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}:latest"
|
||||
echo " - ${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}:${{ github.sha }}"
|
||||
echo "🚀 Triggering Docker Push workflow..."
|
||||
curl -X POST \
|
||||
-H "Authorization: token ${{ secrets.GITEA_TOKEN || secrets.PACKAGES_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
"${{ env.GITEA_INTERNAL }}api/v1/repos/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}/actions/workflows/docker-push.yaml/dispatches" \
|
||||
-d '{"ref":"${{ github.ref }}"}'
|
||||
echo "✅ Docker Push workflow triggered successfully!"
|
||||
|
||||
73
.gitea/workflows/docker-push.yaml
Normal file
73
.gitea/workflows/docker-push.yaml
Normal file
@@ -0,0 +1,73 @@
|
||||
---
|
||||
# dance-lessons-coach Docker Push Workflow
|
||||
# Separate workflow for Docker image building and pushing
|
||||
# Can be triggered manually or by CI/CD workflow
|
||||
|
||||
name: Docker Push
|
||||
|
||||
on:
|
||||
# Manual trigger for testing or production
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
ref:
|
||||
description: 'Git reference (branch/tag)'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
|
||||
# Environment variables
|
||||
env:
|
||||
GITEA_INTERNAL: "https://gitea.arcodange.lab/"
|
||||
GITEA_EXTERNAL: "https://gitea.arcodange.fr/"
|
||||
GITEA_ORG: "arcodange"
|
||||
GITEA_REPO: "dance-lessons-coach"
|
||||
CI_REGISTRY: "gitea.arcodange.lab"
|
||||
|
||||
jobs:
|
||||
docker-push:
|
||||
name: Docker Push
|
||||
runs-on: ubuntu-latest-ca
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.event.inputs.ref || github.ref }}
|
||||
|
||||
- name: Login to Gitea Container Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.CI_REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.PACKAGES_TOKEN }}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
- name: Build and push Docker image
|
||||
run: |
|
||||
source VERSION
|
||||
IMAGE_VERSION="$MAJOR.$MINOR.$PATCH${PRERELEASE:+-$PRERELEASE}"
|
||||
|
||||
TAGS="$IMAGE_VERSION latest ${{ github.sha }}"
|
||||
echo "Building Docker image with tags: $TAGS"
|
||||
|
||||
# Build using the standard Dockerfile (Attempt 2 - simplest approach)
|
||||
docker build -t dance-lessons-coach -f docker/Dockerfile .
|
||||
|
||||
for TAG in $TAGS; do
|
||||
IMAGE_NAME="${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}:$TAG"
|
||||
echo "Tagging and pushing: $IMAGE_NAME"
|
||||
docker tag dance-lessons-coach "$IMAGE_NAME"
|
||||
docker push "$IMAGE_NAME"
|
||||
done
|
||||
|
||||
- name: Show published images
|
||||
run: |
|
||||
source VERSION
|
||||
IMAGE_VERSION="$MAJOR.$MINOR.$PATCH${PRERELEASE:+-$PRERELEASE}"
|
||||
echo "📦 Published Docker images:"
|
||||
echo " - ${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}:$IMAGE_VERSION"
|
||||
echo " - ${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}:latest"
|
||||
echo " - ${{ env.CI_REGISTRY }}/${{ env.GITEA_ORG }}/${{ env.GITEA_REPO }}:${{ github.sha }}"
|
||||
@@ -32,12 +32,12 @@ RUN go install github.com/swaggo/swag/cmd/swag@latest && \
|
||||
go install golang.org/x/tools/cmd/goimports@latest && \
|
||||
go install honnef.co/go/tools/cmd/staticcheck@latest
|
||||
|
||||
# Simple build environment - source code is mounted at runtime
|
||||
WORKDIR /build
|
||||
|
||||
# Copy only go.mod and go.sum first for dependency caching
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download && go mod verify
|
||||
|
||||
# Simple build environment - source code is mounted at runtime
|
||||
WORKDIR /workspace
|
||||
|
||||
# Pre-download common Go tools (already installed in base)
|
||||
# RUN go install github.com/swaggo/swag/cmd/swag@latest
|
||||
@@ -2,9 +2,13 @@
|
||||
# Minimal image using pre-built binary from CI cache
|
||||
# Template: Replace {{DEPS_HASH}} with actual dependency hash
|
||||
|
||||
# Use the build cache image as base
|
||||
# Use the build cache image as base for building
|
||||
FROM gitea.arcodange.lab/arcodange/dance-lessons-coach-build-cache:{{DEPS_HASH}} AS builder
|
||||
|
||||
# Set working directory and build the binary
|
||||
WORKDIR /workspace
|
||||
RUN go build -o dance-lessons-coach ./cmd/server
|
||||
|
||||
# Final minimal image
|
||||
FROM alpine:3.18
|
||||
|
||||
|
||||
1
issue_data.json
Normal file
1
issue_data.json
Normal file
@@ -0,0 +1 @@
|
||||
{"title": "Move Docker push steps to separate job", "body": "The current CI/CD workflow has Docker push steps in the second job that runs in a container. The container doesn't have Docker commands available. Need to move these steps to a new third job that runs on ubuntu-latest-ca without using a container."}
|
||||
@@ -48,3 +48,5 @@ func (s *Service) Greet(ctx context.Context, name string) string {
|
||||
}
|
||||
return "Hello " + name + "!"
|
||||
}
|
||||
|
||||
// Test workflow trigger
|
||||
|
||||
54
scripts/test-docker-push.sh
Executable file
54
scripts/test-docker-push.sh
Executable file
@@ -0,0 +1,54 @@
|
||||
#!/bin/bash
|
||||
# Test Docker push workflow locally using act
|
||||
# Usage: scripts/test-docker-push.sh
|
||||
|
||||
set -e
|
||||
|
||||
echo "🧪 Testing Docker Push Workflow Locally"
|
||||
echo "======================================"
|
||||
echo ""
|
||||
|
||||
# Check requirements
|
||||
if ! command -v act >/dev/null 2>&1; then
|
||||
echo "❌ act not found. Please install act first."
|
||||
echo " brew install act"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
echo "❌ Docker not found. Please install Docker first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "✅ Requirements met"
|
||||
echo ""
|
||||
|
||||
# Test the Docker push workflow
|
||||
echo "1. Testing Docker push workflow with act..."
|
||||
echo " This may take a few minutes..."
|
||||
echo ""
|
||||
|
||||
# Run act with the docker-push workflow
|
||||
act \
|
||||
-W .gitea/workflows/docker-push.yaml \
|
||||
--job docker-push \
|
||||
--secret GITEA_TOKEN=dummy_token \
|
||||
--secret PACKAGES_TOKEN=dummy_token \
|
||||
--env GITEA_INTERNAL=https://gitea.arcodange.lab/ \
|
||||
--env GITEA_EXTERNAL=https://gitea.arcodange.fr/ \
|
||||
--env GITEA_ORG=arcodange \
|
||||
--env GITEA_REPO=dance-lessons-coach \
|
||||
--env CI_REGISTRY=gitea.arcodange.lab
|
||||
|
||||
echo ""
|
||||
echo "✅ Local testing complete!"
|
||||
echo ""
|
||||
echo "💡 If the test failed, check:"
|
||||
echo " - Dockerfile.prod.template syntax"
|
||||
echo " - Dependency hash calculation"
|
||||
echo " - Docker build context"
|
||||
echo " - Go module dependencies"
|
||||
echo ""
|
||||
echo "💡 To test specific scenarios:"
|
||||
echo " act -W .gitea/workflows/docker-push.yaml --list"
|
||||
echo " act -W .gitea/workflows/docker-push.yaml --job docker-push --dryrun"
|
||||
Reference in New Issue
Block a user