🧪 test: add JWT secret rotation BDD scenarios and step implementations (#12)
✨ merge: implement JWT secret rotation with BDD scenario isolation - Implement JWT secret rotation mechanism (closes #8) - Add per-scenario state isolation for BDD tests (closes #14) - Validate password reset workflow via BDD tests (closes #7) - Fix port conflicts in test validation - Add state tracer for debugging test execution - Document BDD isolation strategies in ADR 0025 - Fix PostgreSQL configuration environment variables Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai> Co-authored-by: Gabriel Radureau <arcodange@gmail.com> Co-committed-by: Gabriel Radureau <arcodange@gmail.com>
This commit was merged in pull request #12.
This commit is contained in:
320
bdd_implementation_plan.md
Normal file
320
bdd_implementation_plan.md
Normal file
@@ -0,0 +1,320 @@
|
||||
# BDD Implementation Plan - Iterative Approach
|
||||
|
||||
Based on ADR 0024: BDD Test Organization and Isolation Strategy
|
||||
|
||||
## Phase 1: Refactor Current Tests (1-2 weeks)
|
||||
|
||||
### Objective: Split monolithic feature files into modular, isolated components
|
||||
|
||||
### Tasks:
|
||||
1. **Split feature files by business domain**
|
||||
- Create `features/auth/` directory
|
||||
- Create `features/config/` directory
|
||||
- Create `features/greet/` directory
|
||||
- Create `features/health/` directory
|
||||
- Create `features/jwt/` directory
|
||||
|
||||
2. **Implement feature-specific isolation**
|
||||
- Add config file patterns: `features/{domain}/{domain}-test-config.yaml`
|
||||
- Implement database naming: `dance_lessons_coach_{domain}_test`
|
||||
- Assign unique ports per feature group
|
||||
|
||||
3. **Create feature-specific test scripts**
|
||||
- Implement `scripts/test-feature.sh` with feature parameter
|
||||
- Add environment setup/teardown logic
|
||||
- Implement resource cleanup routines
|
||||
|
||||
### Deliverables:
|
||||
- ✅ Modular feature directory structure
|
||||
- ✅ Feature-specific configuration files
|
||||
- ✅ Basic isolation mechanisms
|
||||
- ✅ Feature-level test scripts
|
||||
|
||||
## Phase 2: Enhance Test Infrastructure (2-3 weeks)
|
||||
|
||||
### Objective: Add synchronization and lifecycle management
|
||||
|
||||
### Tasks:
|
||||
1. **Implement synchronization helpers**
|
||||
- Add `waitForServerReady()` with timeout
|
||||
- Add `waitForConfigReload()` with event-based detection
|
||||
- Add `waitForCondition()` helper function
|
||||
|
||||
2. **Add Godog context management**
|
||||
- Create feature-specific context structs
|
||||
- Implement `InitializeFeatureSuite()`
|
||||
- Implement `CleanupFeatureSuite()`
|
||||
|
||||
3. **Add tag-based test selection**
|
||||
- Implement `@smoke`, `@auth`, `@config` tags
|
||||
- Add tag filtering to test scripts
|
||||
- Document tag usage in README
|
||||
|
||||
### Deliverables:
|
||||
- ✅ Robust synchronization mechanisms
|
||||
- ✅ Proper context lifecycle management
|
||||
- ✅ Tag-based test execution
|
||||
- ✅ Improved test reliability
|
||||
|
||||
## Phase 3: Parallel Testing (Optional - 1 week)
|
||||
|
||||
### Objective: Enable safe parallel test execution
|
||||
|
||||
### Tasks:
|
||||
1. **Implement port management**
|
||||
- Add port allocation system
|
||||
- Implement port conflict detection
|
||||
- Add parallel execution flags
|
||||
|
||||
2. **Add resource monitoring**
|
||||
- Implement resource usage tracking
|
||||
- Add timeout detection
|
||||
- Implement cleanup on failure
|
||||
|
||||
3. **Update CI/CD pipeline**
|
||||
- Add parallel test execution
|
||||
- Implement resource limits
|
||||
- Add test isolation validation
|
||||
|
||||
### Deliverables:
|
||||
- ✅ Parallel test execution capability
|
||||
- ✅ Resource monitoring and limits
|
||||
- ✅ Updated CI/CD configuration
|
||||
|
||||
## Implementation Timeline
|
||||
|
||||
### Week 1-2: Phase 1 - Test Refactoring
|
||||
- Day 1-2: Create feature directory structure
|
||||
- Day 3-4: Implement feature-specific configs
|
||||
- Day 5-7: Create test scripts and isolation
|
||||
- Day 8-10: Test and validate refactoring
|
||||
|
||||
### Week 3-5: Phase 2 - Infrastructure Enhancement
|
||||
- Day 11-12: Add synchronization helpers
|
||||
- Day 13-14: Implement context management
|
||||
- Day 15-17: Add tag-based selection
|
||||
- Day 18-21: Test and validate infrastructure
|
||||
|
||||
### Week 6: Phase 3 - Parallel Testing (Optional)
|
||||
- Day 22-24: Implement port management
|
||||
- Day 25-26: Add resource monitoring
|
||||
- Day 27-28: Update CI/CD pipeline
|
||||
- Day 29-30: Test and validate parallel execution
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Phase 1 Success:
|
||||
- ✅ All tests pass in new structure
|
||||
- ✅ Feature isolation working correctly
|
||||
- ✅ Test scripts functional
|
||||
- ✅ No regression in test coverage
|
||||
|
||||
### Phase 2 Success:
|
||||
- ✅ Synchronization working reliably
|
||||
- ✅ Context management implemented
|
||||
- ✅ Tag filtering operational
|
||||
- ✅ Test reliability >95%
|
||||
|
||||
### Phase 3 Success:
|
||||
- ✅ Parallel tests execute safely
|
||||
- ✅ Resource usage within limits
|
||||
- ✅ CI/CD pipeline updated
|
||||
- ✅ Test execution time reduced
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
### Phase 1 Risks:
|
||||
- **Test failures during refactoring**: Maintain old structure until new is validated
|
||||
- **Isolation issues**: Implement gradual rollout with validation
|
||||
|
||||
### Phase 2 Risks:
|
||||
- **Synchronization complexity**: Start with simple timeouts, enhance gradually
|
||||
- **Context management bugs**: Add comprehensive logging and debugging
|
||||
|
||||
### Phase 3 Risks:
|
||||
- **Resource conflicts**: Implement strict resource limits and monitoring
|
||||
- **CI/CD instability**: Test parallel execution locally before pipeline update
|
||||
|
||||
## Monitoring and Validation
|
||||
|
||||
### Phase 1 Validation:
|
||||
```bash
|
||||
# Test each feature independently
|
||||
./scripts/test-feature.sh auth
|
||||
./scripts/test-feature.sh config
|
||||
./scripts/test-feature.sh greet
|
||||
|
||||
# Verify isolation
|
||||
./scripts/validate-isolation.sh
|
||||
```
|
||||
|
||||
### Phase 2 Validation:
|
||||
```bash
|
||||
# Test synchronization
|
||||
./scripts/test-synchronization.sh
|
||||
|
||||
# Test tag filtering
|
||||
godog --tags=@smoke features/
|
||||
|
||||
# Test context management
|
||||
./scripts/test-context-lifecycle.sh
|
||||
```
|
||||
|
||||
### Phase 3 Validation:
|
||||
```bash
|
||||
# Test parallel execution
|
||||
./scripts/test-all-features-parallel.sh
|
||||
|
||||
# Monitor resource usage
|
||||
./scripts/monitor-test-resources.sh
|
||||
|
||||
# Validate CI/CD changes
|
||||
./scripts/validate-ci-cd.sh
|
||||
```
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
### Phase 1 Rollback:
|
||||
```bash
|
||||
# Revert to original structure
|
||||
git checkout HEAD~1 -- features/
|
||||
|
||||
# Restore original test scripts
|
||||
git checkout HEAD~1 -- scripts/test-*.sh
|
||||
```
|
||||
|
||||
### Phase 2 Rollback:
|
||||
```bash
|
||||
# Remove synchronization helpers
|
||||
git checkout HEAD~1 -- pkg/bdd/helpers/
|
||||
|
||||
# Restore original context management
|
||||
git checkout HEAD~1 -- pkg/bdd/context/
|
||||
```
|
||||
|
||||
### Phase 3 Rollback:
|
||||
```bash
|
||||
# Disable parallel execution
|
||||
sed -i 's/parallel=true/parallel=false/' scripts/test-all-features-parallel.sh
|
||||
|
||||
# Revert CI/CD changes
|
||||
git checkout HEAD~1 -- .github/workflows/
|
||||
```
|
||||
|
||||
## Documentation Updates
|
||||
|
||||
### Phase 1 Documentation:
|
||||
- ✅ Update README with new test structure
|
||||
- ✅ Document feature organization conventions
|
||||
- ✅ Add test execution instructions
|
||||
|
||||
### Phase 2 Documentation:
|
||||
- ✅ Document synchronization patterns
|
||||
- ✅ Add context management guide
|
||||
- ✅ Document tag usage and filtering
|
||||
|
||||
### Phase 3 Documentation:
|
||||
- ✅ Add parallel testing guide
|
||||
- ✅ Document resource limits
|
||||
- ✅ Update CI/CD documentation
|
||||
|
||||
## Team Communication
|
||||
|
||||
### Phase 1:
|
||||
- Team meeting to explain new structure
|
||||
- Hands-on workshop for test refactoring
|
||||
- Daily standups to track progress
|
||||
|
||||
### Phase 2:
|
||||
- Technical deep dive on synchronization
|
||||
- Code review sessions for context management
|
||||
- Pair programming for complex scenarios
|
||||
|
||||
### Phase 3:
|
||||
- Performance testing workshop
|
||||
- CI/CD pipeline review
|
||||
- Resource monitoring training
|
||||
|
||||
## Continuous Improvement
|
||||
|
||||
### Post-Phase 1:
|
||||
- Gather feedback on new structure
|
||||
- Identify pain points in isolation
|
||||
- Optimize test execution times
|
||||
|
||||
### Post-Phase 2:
|
||||
- Monitor test reliability metrics
|
||||
- Identify flaky tests for fixing
|
||||
- Optimize synchronization patterns
|
||||
|
||||
### Post-Phase 3:
|
||||
- Monitor parallel execution performance
|
||||
- Identify resource bottlenecks
|
||||
- Optimize CI/CD pipeline timing
|
||||
|
||||
## Metrics Tracking
|
||||
|
||||
### Test Reliability:
|
||||
```
|
||||
# Track pass rate over time
|
||||
./scripts/track-test-reliability.sh
|
||||
```
|
||||
|
||||
### Test Execution Time:
|
||||
```
|
||||
# Monitor execution times
|
||||
./scripts/monitor-execution-time.sh
|
||||
```
|
||||
|
||||
### Resource Usage:
|
||||
```
|
||||
# Track resource consumption
|
||||
./scripts/monitor-resource-usage.sh
|
||||
```
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Post-Phase 3:
|
||||
- Test impact analysis
|
||||
- Flaky test detection
|
||||
- Performance benchmarking
|
||||
- Test coverage visualization
|
||||
|
||||
### Long-term:
|
||||
- AI-assisted test generation
|
||||
- Automated test optimization
|
||||
- Predictive test failure analysis
|
||||
- Intelligent test prioritization
|
||||
|
||||
## Implementation Checklist
|
||||
|
||||
### Phase 1: Test Refactoring
|
||||
- [ ] Create feature directories
|
||||
- [ ] Split feature files
|
||||
- [ ] Implement config isolation
|
||||
- [ ] Add database isolation
|
||||
- [ ] Create test scripts
|
||||
- [ ] Test and validate
|
||||
|
||||
### Phase 2: Infrastructure Enhancement
|
||||
- [ ] Add synchronization helpers
|
||||
- [ ] Implement context management
|
||||
- [ ] Add tag filtering
|
||||
- [ ] Test and validate
|
||||
|
||||
### Phase 3: Parallel Testing
|
||||
- [ ] Implement port management
|
||||
- [ ] Add resource monitoring
|
||||
- [ ] Update CI/CD pipeline
|
||||
- [ ] Test and validate
|
||||
|
||||
## Notes
|
||||
|
||||
- Each phase builds on the previous one
|
||||
- Phase 3 is optional and can be deferred
|
||||
- Focus on reliability before performance
|
||||
- Maintain backward compatibility where possible
|
||||
- Document all changes thoroughly
|
||||
- Gather team feedback at each phase
|
||||
- Monitor metrics continuously
|
||||
- Celebrate milestones and successes
|
||||
Reference in New Issue
Block a user