diff --git a/bdd_implementation_plan.md b/bdd_implementation_plan.md index 77a34c4..c6061b4 100644 --- a/bdd_implementation_plan.md +++ b/bdd_implementation_plan.md @@ -1,31 +1,320 @@ -Pending BDD Tests Implementation Plan +# BDD Implementation Plan - Iterative Approach -Implementation Plan: +Based on ADR 0024: BDD Test Organization and Isolation Strategy -**Configuration & Validation** (LOW priority): -- `iSetRetentionFactorTo()` - Dynamic configuration -- `iTryToStartTheServer()` - Server validation -- `iShouldReceiveConfigurationValidationError()` - Error handling -- `theErrorShouldMention()` - Error message validation +## Phase 1: Refactor Current Tests (1-2 weeks) -**Monitoring & Metrics** (LOW priority): -- `iShouldSeeMetricIncrement()` - Already implemented ✅ -- `iShouldSeeMetricDecrease()` - Already implemented ✅ -- `iShouldSeeHistogramUpdate()` - Already implemented ✅ +### Objective: Split monolithic feature files into modular, isolated components -**Performance & Scalability** (LOW priority): -- `iHaveJWTSecrets()` - Bulk secret management -- `ofThemAreExpired()` - Expiration tracking -- `itShouldCompleteWithinMilliseconds()` - Performance validation -- `andNotImpactServerPerformance()` - Performance monitoring +### Tasks: +1. **Split feature files by business domain** + - Create `features/auth/` directory + - Create `features/config/` directory + - Create `features/greet/` directory + - Create `features/health/` directory + - Create `features/jwt/` directory -**Advanced Features** (LOW priority): -- Various edge case and advanced scenarios +2. **Implement feature-specific isolation** + - Add config file patterns: `features/{domain}/{domain}-test-config.yaml` + - Implement database naming: `dance_lessons_coach_{domain}_test` + - Assign unique ports per feature group -Next Steps: +3. **Create feature-specific test scripts** + - Implement `scripts/test-feature.sh` with feature parameter + - Add environment setup/teardown logic + - Implement resource cleanup routines -1. Add configuration validation and monitoring -2. Implement step definitions for pending scenarios -3. Run full test suite to verify all scenarios pass +### Deliverables: +- ✅ Modular feature directory structure +- ✅ Feature-specific configuration files +- ✅ Basic isolation mechanisms +- ✅ Feature-level test scripts -Estimated Time: 2-3 days +## Phase 2: Enhance Test Infrastructure (2-3 weeks) + +### Objective: Add synchronization and lifecycle management + +### Tasks: +1. **Implement synchronization helpers** + - Add `waitForServerReady()` with timeout + - Add `waitForConfigReload()` with event-based detection + - Add `waitForCondition()` helper function + +2. **Add Godog context management** + - Create feature-specific context structs + - Implement `InitializeFeatureSuite()` + - Implement `CleanupFeatureSuite()` + +3. **Add tag-based test selection** + - Implement `@smoke`, `@auth`, `@config` tags + - Add tag filtering to test scripts + - Document tag usage in README + +### Deliverables: +- ✅ Robust synchronization mechanisms +- ✅ Proper context lifecycle management +- ✅ Tag-based test execution +- ✅ Improved test reliability + +## Phase 3: Parallel Testing (Optional - 1 week) + +### Objective: Enable safe parallel test execution + +### Tasks: +1. **Implement port management** + - Add port allocation system + - Implement port conflict detection + - Add parallel execution flags + +2. **Add resource monitoring** + - Implement resource usage tracking + - Add timeout detection + - Implement cleanup on failure + +3. **Update CI/CD pipeline** + - Add parallel test execution + - Implement resource limits + - Add test isolation validation + +### Deliverables: +- ✅ Parallel test execution capability +- ✅ Resource monitoring and limits +- ✅ Updated CI/CD configuration + +## Implementation Timeline + +### Week 1-2: Phase 1 - Test Refactoring +- Day 1-2: Create feature directory structure +- Day 3-4: Implement feature-specific configs +- Day 5-7: Create test scripts and isolation +- Day 8-10: Test and validate refactoring + +### Week 3-5: Phase 2 - Infrastructure Enhancement +- Day 11-12: Add synchronization helpers +- Day 13-14: Implement context management +- Day 15-17: Add tag-based selection +- Day 18-21: Test and validate infrastructure + +### Week 6: Phase 3 - Parallel Testing (Optional) +- Day 22-24: Implement port management +- Day 25-26: Add resource monitoring +- Day 27-28: Update CI/CD pipeline +- Day 29-30: Test and validate parallel execution + +## Success Criteria + +### Phase 1 Success: +- ✅ All tests pass in new structure +- ✅ Feature isolation working correctly +- ✅ Test scripts functional +- ✅ No regression in test coverage + +### Phase 2 Success: +- ✅ Synchronization working reliably +- ✅ Context management implemented +- ✅ Tag filtering operational +- ✅ Test reliability >95% + +### Phase 3 Success: +- ✅ Parallel tests execute safely +- ✅ Resource usage within limits +- ✅ CI/CD pipeline updated +- ✅ Test execution time reduced + +## Risk Mitigation + +### Phase 1 Risks: +- **Test failures during refactoring**: Maintain old structure until new is validated +- **Isolation issues**: Implement gradual rollout with validation + +### Phase 2 Risks: +- **Synchronization complexity**: Start with simple timeouts, enhance gradually +- **Context management bugs**: Add comprehensive logging and debugging + +### Phase 3 Risks: +- **Resource conflicts**: Implement strict resource limits and monitoring +- **CI/CD instability**: Test parallel execution locally before pipeline update + +## Monitoring and Validation + +### Phase 1 Validation: +```bash +# Test each feature independently +./scripts/test-feature.sh auth +./scripts/test-feature.sh config +./scripts/test-feature.sh greet + +# Verify isolation +./scripts/validate-isolation.sh +``` + +### Phase 2 Validation: +```bash +# Test synchronization +./scripts/test-synchronization.sh + +# Test tag filtering +godog --tags=@smoke features/ + +# Test context management +./scripts/test-context-lifecycle.sh +``` + +### Phase 3 Validation: +```bash +# Test parallel execution +./scripts/test-all-features-parallel.sh + +# Monitor resource usage +./scripts/monitor-test-resources.sh + +# Validate CI/CD changes +./scripts/validate-ci-cd.sh +``` + +## Rollback Plan + +### Phase 1 Rollback: +```bash +# Revert to original structure +git checkout HEAD~1 -- features/ + +# Restore original test scripts +git checkout HEAD~1 -- scripts/test-*.sh +``` + +### Phase 2 Rollback: +```bash +# Remove synchronization helpers +git checkout HEAD~1 -- pkg/bdd/helpers/ + +# Restore original context management +git checkout HEAD~1 -- pkg/bdd/context/ +``` + +### Phase 3 Rollback: +```bash +# Disable parallel execution +sed -i 's/parallel=true/parallel=false/' scripts/test-all-features-parallel.sh + +# Revert CI/CD changes +git checkout HEAD~1 -- .github/workflows/ +``` + +## Documentation Updates + +### Phase 1 Documentation: +- ✅ Update README with new test structure +- ✅ Document feature organization conventions +- ✅ Add test execution instructions + +### Phase 2 Documentation: +- ✅ Document synchronization patterns +- ✅ Add context management guide +- ✅ Document tag usage and filtering + +### Phase 3 Documentation: +- ✅ Add parallel testing guide +- ✅ Document resource limits +- ✅ Update CI/CD documentation + +## Team Communication + +### Phase 1: +- Team meeting to explain new structure +- Hands-on workshop for test refactoring +- Daily standups to track progress + +### Phase 2: +- Technical deep dive on synchronization +- Code review sessions for context management +- Pair programming for complex scenarios + +### Phase 3: +- Performance testing workshop +- CI/CD pipeline review +- Resource monitoring training + +## Continuous Improvement + +### Post-Phase 1: +- Gather feedback on new structure +- Identify pain points in isolation +- Optimize test execution times + +### Post-Phase 2: +- Monitor test reliability metrics +- Identify flaky tests for fixing +- Optimize synchronization patterns + +### Post-Phase 3: +- Monitor parallel execution performance +- Identify resource bottlenecks +- Optimize CI/CD pipeline timing + +## Metrics Tracking + +### Test Reliability: +``` +# Track pass rate over time +./scripts/track-test-reliability.sh +``` + +### Test Execution Time: +``` +# Monitor execution times +./scripts/monitor-execution-time.sh +``` + +### Resource Usage: +``` +# Track resource consumption +./scripts/monitor-resource-usage.sh +``` + +## Future Enhancements + +### Post-Phase 3: +- Test impact analysis +- Flaky test detection +- Performance benchmarking +- Test coverage visualization + +### Long-term: +- AI-assisted test generation +- Automated test optimization +- Predictive test failure analysis +- Intelligent test prioritization + +## Implementation Checklist + +### Phase 1: Test Refactoring +- [ ] Create feature directories +- [ ] Split feature files +- [ ] Implement config isolation +- [ ] Add database isolation +- [ ] Create test scripts +- [ ] Test and validate + +### Phase 2: Infrastructure Enhancement +- [ ] Add synchronization helpers +- [ ] Implement context management +- [ ] Add tag filtering +- [ ] Test and validate + +### Phase 3: Parallel Testing +- [ ] Implement port management +- [ ] Add resource monitoring +- [ ] Update CI/CD pipeline +- [ ] Test and validate + +## Notes + +- Each phase builds on the previous one +- Phase 3 is optional and can be deferred +- Focus on reliability before performance +- Maintain backward compatibility where possible +- Document all changes thoroughly +- Gather team feedback at each phase +- Monitor metrics continuously +- Celebrate milestones and successes diff --git a/features/auth/auth_test.go b/features/auth/auth_test.go new file mode 100644 index 0000000..0381dde --- /dev/null +++ b/features/auth/auth_test.go @@ -0,0 +1,29 @@ +package auth + +import ( + "os" + "testing" + + "dance-lessons-coach/pkg/bdd" + "github.com/cucumber/godog" +) + +func TestAuthBDD(t *testing.T) { + // Set FEATURE environment variable for feature-specific configuration + os.Setenv("FEATURE", "auth") + + suite := godog.TestSuite{ + Name: "dance-lessons-coach BDD Tests - Auth Feature", + TestSuiteInitializer: bdd.InitializeTestSuite, + ScenarioInitializer: bdd.InitializeScenario, + Options: &godog.Options{ + Format: "progress", + Paths: []string{"."}, + TestingT: t, + }, + } + + if suite.Run() != 0 { + t.Fatal("non-zero status returned, failed to run auth BDD tests") + } +} diff --git a/features/user_authentication.feature b/features/auth/user_authentication.feature similarity index 100% rename from features/user_authentication.feature rename to features/auth/user_authentication.feature diff --git a/features/bdd_test.go b/features/bdd_test.go index bfe4df8..dd6e267 100644 --- a/features/bdd_test.go +++ b/features/bdd_test.go @@ -1,6 +1,7 @@ package features import ( + "os" "testing" "dance-lessons-coach/pkg/bdd" @@ -8,13 +9,35 @@ import ( ) func TestBDD(t *testing.T) { + // Get feature name from environment variable or default to all features + feature := os.Getenv("FEATURE") + + var paths []string + var suiteName string + + if feature == "" { + // Run all features + suiteName = "dance-lessons-coach BDD Tests - All Features" + paths = []string{ + "features/auth", + "features/config", + "features/greet", + "features/health", + "features/jwt", + } + } else { + // Run specific feature + suiteName = "dance-lessons-coach BDD Tests - " + feature + " Feature" + paths = []string{"features/" + feature} + } + suite := godog.TestSuite{ - Name: "dance-lessons-coach BDD Tests", + Name: suiteName, TestSuiteInitializer: bdd.InitializeTestSuite, ScenarioInitializer: bdd.InitializeScenario, Options: &godog.Options{ Format: "progress", - Paths: []string{"."}, + Paths: paths, TestingT: t, }, } diff --git a/features/config_hot_reloading.feature b/features/config/config_hot_reloading.feature similarity index 100% rename from features/config_hot_reloading.feature rename to features/config/config_hot_reloading.feature diff --git a/features/config/config_test.go b/features/config/config_test.go new file mode 100644 index 0000000..320d039 --- /dev/null +++ b/features/config/config_test.go @@ -0,0 +1,29 @@ +package config + +import ( + "os" + "testing" + + "dance-lessons-coach/pkg/bdd" + "github.com/cucumber/godog" +) + +func TestConfigBDD(t *testing.T) { + // Set FEATURE environment variable for feature-specific configuration + os.Setenv("FEATURE", "config") + + suite := godog.TestSuite{ + Name: "dance-lessons-coach BDD Tests - Config Feature", + TestSuiteInitializer: bdd.InitializeTestSuite, + ScenarioInitializer: bdd.InitializeScenario, + Options: &godog.Options{ + Format: "progress", + Paths: []string{"."}, + TestingT: t, + }, + } + + if suite.Run() != 0 { + t.Fatal("non-zero status returned, failed to run config BDD tests") + } +} diff --git a/features/greet.feature b/features/greet/greet.feature similarity index 97% rename from features/greet.feature rename to features/greet/greet.feature index 5daf96f..4b2aa40 100644 --- a/features/greet.feature +++ b/features/greet/greet.feature @@ -1,17 +1,21 @@ # features/greet.feature +@greet @smoke Feature: Greet Service The greet service should return appropriate greetings + @basic Scenario: Default greeting Given the server is running When I request the default greeting Then the response should be "{\"message\":\"Hello world!\"}" + @basic Scenario: Personalized greeting Given the server is running When I request a greeting for "John" Then the response should be "{\"message\":\"Hello John!\"}" + @v2 @api Scenario: v2 greeting with JSON POST request Given the server is running with v2 enabled When I send a POST request to v2 greet with name "John" diff --git a/features/greet/greet_test.go b/features/greet/greet_test.go new file mode 100644 index 0000000..a0bbb49 --- /dev/null +++ b/features/greet/greet_test.go @@ -0,0 +1,29 @@ +package greet + +import ( + "os" + "testing" + + "dance-lessons-coach/pkg/bdd" + "github.com/cucumber/godog" +) + +func TestGreetBDD(t *testing.T) { + // Set FEATURE environment variable for feature-specific configuration + os.Setenv("FEATURE", "greet") + + suite := godog.TestSuite{ + Name: "dance-lessons-coach BDD Tests - Greet Feature", + TestSuiteInitializer: bdd.InitializeTestSuite, + ScenarioInitializer: bdd.InitializeScenario, + Options: &godog.Options{ + Format: "progress", + Paths: []string{"."}, + TestingT: t, + }, + } + + if suite.Run() != 0 { + t.Fatal("non-zero status returned, failed to run greet BDD tests") + } +} diff --git a/features/health.feature b/features/health/health.feature similarity index 86% rename from features/health.feature rename to features/health/health.feature index 3ba8b65..c897150 100644 --- a/features/health.feature +++ b/features/health/health.feature @@ -1,7 +1,9 @@ # features/health.feature +@health @smoke @critical Feature: Health Endpoint The health endpoint should indicate server status + @basic @critical Scenario: Health check returns healthy status Given the server is running When I request the health endpoint diff --git a/features/health/health_test.go b/features/health/health_test.go new file mode 100644 index 0000000..390f578 --- /dev/null +++ b/features/health/health_test.go @@ -0,0 +1,29 @@ +package health + +import ( + "os" + "testing" + + "dance-lessons-coach/pkg/bdd" + "github.com/cucumber/godog" +) + +func TestHealthBDD(t *testing.T) { + // Set FEATURE environment variable for feature-specific configuration + os.Setenv("FEATURE", "health") + + suite := godog.TestSuite{ + Name: "dance-lessons-coach BDD Tests - Health Feature", + TestSuiteInitializer: bdd.InitializeTestSuite, + ScenarioInitializer: bdd.InitializeScenario, + Options: &godog.Options{ + Format: "progress", + Paths: []string{"."}, + TestingT: t, + }, + } + + if suite.Run() != 0 { + t.Fatal("non-zero status returned, failed to run health BDD tests") + } +} diff --git a/features/jwt_secret_retention.feature b/features/jwt/jwt_secret_retention.feature similarity index 100% rename from features/jwt_secret_retention.feature rename to features/jwt/jwt_secret_retention.feature diff --git a/features/jwt_secret_rotation.feature b/features/jwt/jwt_secret_rotation.feature similarity index 100% rename from features/jwt_secret_rotation.feature rename to features/jwt/jwt_secret_rotation.feature diff --git a/features/jwt/jwt_test.go b/features/jwt/jwt_test.go new file mode 100644 index 0000000..0404691 --- /dev/null +++ b/features/jwt/jwt_test.go @@ -0,0 +1,29 @@ +package jwt + +import ( + "os" + "testing" + + "dance-lessons-coach/pkg/bdd" + "github.com/cucumber/godog" +) + +func TestJWTBDD(t *testing.T) { + // Set FEATURE environment variable for feature-specific configuration + os.Setenv("FEATURE", "jwt") + + suite := godog.TestSuite{ + Name: "dance-lessons-coach BDD Tests - JWT Feature", + TestSuiteInitializer: bdd.InitializeTestSuite, + ScenarioInitializer: bdd.InitializeScenario, + Options: &godog.Options{ + Format: "progress", + Paths: []string{"."}, + TestingT: t, + }, + } + + if suite.Run() != 0 { + t.Fatal("non-zero status returned, failed to run jwt BDD tests") + } +} diff --git a/pkg/bdd/README.md b/pkg/bdd/README.md index 4bffa69..6a23f30 100644 --- a/pkg/bdd/README.md +++ b/pkg/bdd/README.md @@ -1,96 +1,327 @@ -# BDD Testing with Godog +# BDD Testing Framework -This package implements Behavior-Driven Development (BDD) testing using the Godog framework. +This directory contains the Behavior-Driven Development (BDD) testing framework for the dance-lessons-coach project, implementing the architecture described in ADR 0024. -## Important Requirements for Step Definitions +## 🗺️ Architecture Overview -### Step Pattern Matching +The BDD framework follows a modular, isolated test suite architecture with these key components: -Godog has **very specific requirements** for step pattern matching. To avoid "undefined" warnings: +### 📁 Directory Structure -1. **Use the exact regex pattern** that Godog suggests in its error messages -2. **Use the exact parameter names** that Godog suggests (`arg1, arg2`, etc.) -3. **Match the feature file syntax exactly** including quotes and JSON formatting - -### Example - -**Feature file step:** -```gherkin -Then the response should be "{\"message\":\"Hello world!\"}" +``` +pkg/bdd/ +├── README.md # This file +├── context/ # Feature-specific test contexts +│ ├── auth_context.go # Authentication test context +│ └── config_context.go # Configuration test context +├── helpers/ # Test synchronization helpers +│ └── synchronization.go # Wait functions and utilities +├── parallel/ # Parallel test execution +│ ├── port_manager.go # Port allocation system +│ └── resource_monitor.go # Resource tracking +├── steps/ # Step definitions +│ ├── auth_steps.go # Authentication steps +│ ├── config_steps.go # Configuration steps +│ ├── greet_steps.go # Greeting steps +│ ├── health_steps.go # Health check steps +│ ├── jwt_retention_steps.go # JWT retention steps +│ └── steps.go # Main step registration +├── suite.go # Test suite initialization +├── suite_feature.go # Feature-specific suite support +└── testserver/ # Test server implementation + ├── client.go # HTTP test client + └── server.go # Test server with config ``` -**Correct step definition:** +## 🎯 Core Components + +### 1. Test Server + +**Location:** `pkg/bdd/testserver/` + +The test server provides a real HTTP server instance for black-box testing: + +- **Hybrid Testing**: Runs in-process (not external process) +- **Configuration**: Loads feature-specific configs from `features/*/*-test-config.yaml` +- **Database**: Manages PostgreSQL connections with proper isolation +- **Port Management**: Uses feature-specific ports (9192-9196) + +**Key Functions:** +- `NewServer()` - Creates test server instance +- `Start()` - Starts server with feature-specific configuration +- `initDBConnection()` - Initializes database connection +- `createTestConfig()` - Loads feature-specific configuration + +### 2. Step Definitions + +**Location:** `pkg/bdd/steps/` + +Step definitions implement the Gherkin scenarios using Godog: + +- **Domain-Specific**: Organized by feature area (auth, config, greet, etc.) +- **Reusable**: Common patterns in `common_steps.go` +- **Exact Matching**: Uses Godog's exact regex patterns + +**Example:** ```go -ctx.Step(`^the response should be "{\"([^"]*)\":\"([^"]*)\"}"$`, func(arg1, arg2 string) error { - // Implementation here - return nil -}) +// greet_steps.go +func (gs *GreetSteps) iRequestAGreetingFor(name string) error { + return gs.client.Request("GET", fmt.Sprintf("/api/v1/greet/%s", name), nil) +} ``` -**Incorrect patterns that cause "undefined" warnings:** +### 3. Synchronization Helpers + +**Location:** `pkg/bdd/helpers/` + +Helpers provide robust waiting mechanisms for async operations: + +- **Timeout Support**: All functions include timeout parameters +- **Polling**: Uses context-based polling with configurable intervals +- **Common Patterns**: Covers server readiness, config reload, API availability + +**Available Helpers:** +- `waitForServerReady()` - Waits for server to be ready +- `waitForConfigReload()` - Detects configuration changes +- `waitForCondition()` - Generic condition waiting +- `waitForV2APIEnabled()` - Checks v2 API availability + +### 4. Parallel Testing + +**Location:** `pkg/bdd/parallel/` + +Parallel execution infrastructure for CI/CD optimization: + +- **Port Management**: `PortManager` allocates unique ports +- **Resource Monitoring**: Tracks memory, goroutines, CPU usage +- **Controlled Parallelism**: `ParallelTestRunner` limits concurrency + +**Key Features:** +- Thread-safe port allocation +- Resource limit enforcement +- Timeout detection +- Comprehensive monitoring + +### 5. Feature Contexts + +**Location:** `pkg/bdd/context/` + +Feature-specific test contexts for better organization: + +- **AuthContext**: User management and authentication +- **ConfigContext**: Configuration file handling +- **Extensible**: Easy to add new feature contexts + +## 🚀 Test Execution + +### Running All Tests + +```bash +# Default: Run all features sequentially +go test ./features/... + +# With environment variables +DLC_DATABASE_HOST=localhost DLC_DATABASE_PORT=5432 \ +DLC_DATABASE_USER=postgres DLC_DATABASE_PASSWORD=postgres \ +DLC_DATABASE_NAME=dance_lessons_coach_bdd_test \ +DLC_DATABASE_SSL_MODE=disable \ +go test ./features/... +``` + +### Feature-Specific Testing + +```bash +# Test specific feature +./scripts/test-feature.sh greet + +# Test with specific tags +./scripts/test-by-tag.sh @smoke greet +``` + +### Parallel Testing + +```bash +# Run all features in parallel +./scripts/test-all-features-parallel.sh + +# Run specific features in parallel +# (Requires PostgreSQL container running) +``` + +### Tag-Based Testing + +```bash +# List available tags +./scripts/run-bdd-tests.sh list-tags + +# Run smoke tests +./scripts/run-bdd-tests.sh run @smoke + +# Run critical tests for auth +./scripts/run-bdd-tests.sh run @critical @auth +``` + +## 📋 Test Organization + +### Feature Structure + +Each feature follows this structure: + +``` +features/{feature}/ +├── {feature}.feature # Gherkin scenarios +├── {feature}-test-config.yaml # Feature-specific config +└── {feature}_test.go # Go test runner +``` + +### Configuration Files + +Feature-specific YAML files define test environment: + +```yaml +# features/greet/greet-test-config.yaml +server: + host: "127.0.0.1" + port: 9194 + +database: + host: "localhost" + port: 5432 + name: "dance_lessons_coach_greet_test" + +api: + v2_enabled: true +``` + +### Tagging System + +Comprehensive tagging for selective test execution: + +- **Feature Tags**: `@auth`, `@config`, `@greet`, `@health`, `@jwt` +- **Priority Tags**: `@smoke`, `@critical`, `@basic`, `@advanced` +- **Component Tags**: `@api`, `@v2`, `@database`, `@security` + +See `features/BDD_TAGS.md` for complete documentation. + +## 🔧 Database Management + +### Database Creation + +The framework handles database creation automatically: + +1. **PostgreSQL Container**: Uses Docker (`dance-lessons-coach-postgres`) +2. **Feature Databases**: Creates `dance_lessons_coach_{feature}_test` per feature +3. **Cleanup**: Automatically drops databases after tests + +**Database Creation Flow:** +1. Check if database exists +2. Create if missing (`createdb` command) +3. Run tests with isolated database +4. Cleanup (`dropdb` command) + +### Configuration + +Database settings come from: +- Environment variables (`DLC_DATABASE_*`) +- Feature-specific config files +- Default values for development + +## 🧪 Best Practices + +### Step Definition Patterns + ```go -// Wrong: Different regex pattern -ctx.Step(`^the response should be "{\"message\":\"([^"]*)\"}"$`, func(message string) error { - // ... -}) +// ✅ DO: Use Godog's exact regex patterns +ctx.Step(`^I request a greeting for "([^"]*)"$`, sc.iRequestAGreetingFor) -// Wrong: Different parameter names -ctx.Step(`^the response should be "{\"([^"]*)\":\"([^"]*)\"}"$`, func(key, value string) error { - // ... -}) +// ❌ DON'T: Use different patterns +ctx.Step(`^I request greeting "(.*)"$`, sc.iRequestAGreetingFor) ``` -## Current Implementation +### Test Isolation -### Step Definition Strategy +- Each feature has unique port and database +- No shared state between features +- Cleanup after each test run +- Feature-specific configuration -1. **First eliminate "undefined" warnings** by using Godog's exact suggested patterns -2. **Return `godog.ErrPending`** initially to confirm pattern matching works -3. **Then implement actual validation** logic +### Synchronization -### Files +```go +// ✅ DO: Use helpers for async operations +helpers.waitForServerReady(client, 30*time.Second) -- `suite.go`: Test suite initialization and server management -- `testserver/`: Test server and client implementation -- `steps/`: Step definitions for each feature +// ❌ DON'T: Use fixed sleep times +time.Sleep(5 * time.Second) +``` -## Debugging "Undefined" Steps +### Context Management -If you see "undefined" warnings: +```go +// ✅ DO: Use feature-specific contexts +switch featureName { +case "auth": + authCtx = context.NewAuthContext(client) + context.InitializeAuthContext(ctx, client) +} +``` -1. Run the tests to see Godog's suggested pattern: - ```bash - go test ./features/... -v - ``` +## 📈 Performance Optimization -2. Copy the **exact regex pattern** from the error message -3. Copy the **exact parameter names** (`arg1, arg2`, etc.) -4. Update your step definition to match exactly +### Parallel Execution -## Common Mistakes +- Use `scripts/test-all-features-parallel.sh` for CI/CD +- Limit parallelism based on system resources +- Monitor resource usage with `ResourceMonitor` -The "undefined" warnings are **not a Godog bug** - they occur when step definitions don't match Godog's expected patterns exactly: +### Selective Testing -- Using different regex patterns than what Godog suggests -- Using descriptive parameter names instead of `arg1, arg2` -- Not escaping quotes properly in JSON patterns -- Trying to be "clever" with regex optimization +- Run only relevant tests with tag filtering +- Use `@smoke` for quick validation +- Use `@critical` for essential path testing -**Solution**: Always use the exact pattern and parameter names that Godog suggests in its error messages. +### Resource Management -## Best Practices +- Set appropriate timeouts +- Limit maximum goroutines +- Monitor memory usage +- Cleanup resources promptly -1. **Follow Godog's suggestions exactly** - Copy-paste the pattern and parameter names -2. **Test pattern matching first** - Use `godog.ErrPending` to verify patterns work -3. **Then implement logic** - Replace `godog.ErrPending` with actual validation -4. **Don't over-optimize regex** - Use the patterns Godog provides, even if they seem verbose -5. **One pattern per step type** - Use generic patterns to cover similar steps +## 🔧 Troubleshooting -## Why This Matters +### Common Issues -Godog's step matching is **very specific by design**: -- It needs to reliably match feature file steps to code -- It provides exact patterns to ensure consistency -- Following its suggestions guarantees your steps will be recognized +| Issue | Cause | Solution | +|-------|-------|----------| +| Undefined steps | Step pattern mismatch | Use Godog's exact suggested patterns | +| Port conflicts | Multiple servers | Check port allocation in config files | +| Database connection | PostgreSQL not running | Start with `docker compose up -d postgres` | +| Test isolation | Shared state | Verify unique ports/databases per feature | -**Remember**: The "undefined" warnings are Godog telling you exactly how to fix your step definitions! \ No newline at end of file +### Debugging + +```bash +# Verbose output +go test ./features/... -v + +# Check specific feature +cd features/greet && go test -v . + +# List available tags +./scripts/run-bdd-tests.sh list-tags +``` + +## 📚 Documentation + +- **ADR 0024**: BDD Test Organization and Isolation Strategy +- **BDD_TAGS.md**: Complete tag reference +- **Godog Documentation**: https://github.com/cucumber/godog + +## 🎯 Future Enhancements + +- **Test Impact Analysis**: Track which tests are affected by code changes +- **Flaky Test Detection**: Automatically identify and quarantine flaky tests +- **Performance Benchmarking**: Monitor test execution times +- **AI-Assisted Testing**: Automated test generation and optimization + +This BDD framework provides a robust foundation for behavior-driven development in the dance-lessons-coach project, ensuring test reliability, maintainability, and scalability. \ No newline at end of file diff --git a/pkg/bdd/context/auth_context.go b/pkg/bdd/context/auth_context.go index 6ece164..5057661 100644 --- a/pkg/bdd/context/auth_context.go +++ b/pkg/bdd/context/auth_context.go @@ -2,6 +2,7 @@ package context import ( "dance-lessons-coach/pkg/bdd/testserver" + "github.com/cucumber/godog" ) diff --git a/pkg/bdd/context/config_context.go b/pkg/bdd/context/config_context.go index 0c89632..09ce938 100644 --- a/pkg/bdd/context/config_context.go +++ b/pkg/bdd/context/config_context.go @@ -2,6 +2,7 @@ package context import ( "dance-lessons-coach/pkg/bdd/testserver" + "github.com/cucumber/godog" ) diff --git a/pkg/bdd/helpers/synchronization.go b/pkg/bdd/helpers/synchronization.go index 5bb696d..864130f 100644 --- a/pkg/bdd/helpers/synchronization.go +++ b/pkg/bdd/helpers/synchronization.go @@ -6,6 +6,7 @@ import ( "time" "dance-lessons-coach/pkg/bdd/testserver" + "github.com/rs/zerolog/log" ) diff --git a/pkg/bdd/steps/config_steps.go b/pkg/bdd/steps/config_steps.go index af57f68..bb28cf2 100644 --- a/pkg/bdd/steps/config_steps.go +++ b/pkg/bdd/steps/config_steps.go @@ -18,9 +18,19 @@ type ConfigSteps struct { } func NewConfigSteps(client *testserver.Client) *ConfigSteps { + // Get feature-specific config path + feature := os.Getenv("FEATURE") + var configFilePath string + + if feature != "" { + configFilePath = fmt.Sprintf("features/%s/%s-test-config.yaml", feature, feature) + } else { + configFilePath = "test-config.yaml" + } + return &ConfigSteps{ client: client, - configFilePath: "test-config.yaml", + configFilePath: configFilePath, } } diff --git a/pkg/bdd/testserver/server.go b/pkg/bdd/testserver/server.go index 2cded52..dfa2f0b 100644 --- a/pkg/bdd/testserver/server.go +++ b/pkg/bdd/testserver/server.go @@ -6,6 +6,7 @@ import ( "fmt" "net/http" "os" + "strconv" "strings" "time" @@ -27,8 +28,37 @@ type Server struct { } func NewServer() *Server { + // Get feature-specific port from configuration + feature := os.Getenv("FEATURE") + port := 9191 // Default port + + if feature != "" { + // Try to read port from feature-specific config + configPath := fmt.Sprintf("features/%s/%s-test-config.yaml", feature, feature) + if _, statErr := os.Stat(configPath); statErr == nil { + // Read config file to get port + content, err := os.ReadFile(configPath) + if err == nil { + // Simple YAML parsing to extract port + lines := strings.Split(string(content), "\n") + for _, line := range lines { + if strings.Contains(line, "port:") { + parts := strings.Split(line, ":") + if len(parts) >= 2 { + portStr := strings.TrimSpace(parts[1]) + if p, err := strconv.Atoi(portStr); err == nil { + port = p + break + } + } + } + } + } + } + } + return &Server{ - port: 9191, + port: port, } } @@ -71,7 +101,16 @@ func (s *Server) Start() error { // monitorConfigFile monitors the test config file for changes and reloads configuration func (s *Server) monitorConfigFile() { - testConfigPath := "test-config.yaml" + // Get feature-specific config path + feature := os.Getenv("FEATURE") + var testConfigPath string + + if feature != "" { + testConfigPath = fmt.Sprintf("features/%s/%s-test-config.yaml", feature, feature) + } else { + testConfigPath = "test-config.yaml" + } + lastModTime := time.Time{} fileExists := false @@ -151,7 +190,39 @@ func (s *Server) ReloadConfig() error { // initDBConnection initializes a direct database connection for cleanup operations func (s *Server) initDBConnection() error { - cfg := createTestConfig(s.port) + // Get feature-specific configuration + feature := os.Getenv("FEATURE") + var cfg *config.Config + + if feature != "" { + // Try to load feature-specific config + configPath := fmt.Sprintf("features/%s/%s-test-config.yaml", feature, feature) + if _, err := os.Stat(configPath); err == nil { + v := viper.New() + v.SetConfigFile(configPath) + v.SetConfigType("yaml") + + if readErr := v.ReadInConfig(); readErr == nil { + var featureCfg config.Config + if unmarshalErr := v.Unmarshal(&featureCfg); unmarshalErr == nil { + // Set default values if not configured + if featureCfg.Auth.JWTSecret == "" { + featureCfg.Auth.JWTSecret = "default-secret-key-please-change-in-production" + } + if featureCfg.Auth.AdminMasterPassword == "" { + featureCfg.Auth.AdminMasterPassword = "admin123" + } + cfg = &featureCfg + } + } + } + } + + // Fallback to default config if feature-specific not available + if cfg == nil { + cfg = createTestConfig(s.port) + } + dsn := fmt.Sprintf( "host=%s port=%d user=%s password=%s dbname=%s sslmode=%s", cfg.Database.Host, @@ -162,10 +233,10 @@ func (s *Server) initDBConnection() error { cfg.Database.SSLMode, ) - var err error - s.db, err = sql.Open("postgres", dsn) - if err != nil { - return fmt.Errorf("failed to open database connection: %w", err) + var dbErr error + s.db, dbErr = sql.Open("postgres", dsn) + if dbErr != nil { + return fmt.Errorf("failed to open database connection: %w", dbErr) } // Test the connection @@ -329,31 +400,48 @@ func (s *Server) GetBaseURL() string { } func createTestConfig(port int) *config.Config { - // Check if there's a test config file (used by config hot reloading tests) - // If it exists, use it. Otherwise, use default config. - testConfigPath := "test-config.yaml" - if _, err := os.Stat(testConfigPath); err == nil { - // Test config file exists, use it - v := viper.New() - v.SetConfigFile(testConfigPath) - v.SetConfigType("yaml") + // Check for feature-specific config file first + // This supports the new modular BDD test structure + feature := os.Getenv("FEATURE") + var configPaths []string - // Read the test config file - if err := v.ReadInConfig(); err == nil { - var cfg config.Config - if err := v.Unmarshal(&cfg); err == nil { - // Override server port for testing - cfg.Server.Port = port + if feature != "" { + // Feature-specific config takes precedence + configPaths = []string{ + fmt.Sprintf("features/%s/%s-test-config.yaml", feature, feature), + "test-config.yaml", // Fallback to legacy config + } + } else { + // When running all features, use legacy config + configPaths = []string{"test-config.yaml"} + } - // Set default auth values if not configured - if cfg.Auth.JWTSecret == "" { - cfg.Auth.JWTSecret = "default-secret-key-please-change-in-production" + // Try each config path in order + for _, configPath := range configPaths { + if _, err := os.Stat(configPath); err == nil { + // Config file exists, use it + v := viper.New() + v.SetConfigFile(configPath) + v.SetConfigType("yaml") + + // Read the config file + if err := v.ReadInConfig(); err == nil { + var cfg config.Config + if err := v.Unmarshal(&cfg); err == nil { + // Override server port for testing + cfg.Server.Port = port + + // Set default auth values if not configured + if cfg.Auth.JWTSecret == "" { + cfg.Auth.JWTSecret = "default-secret-key-please-change-in-production" + } + if cfg.Auth.AdminMasterPassword == "" { + cfg.Auth.AdminMasterPassword = "admin123" + } + + log.Debug().Str("config", configPath).Msg("Using test config file") + return &cfg } - if cfg.Auth.AdminMasterPassword == "" { - cfg.Auth.AdminMasterPassword = "admin123" - } - - return &cfg } } } diff --git a/scripts/test-by-tag.sh b/scripts/test-by-tag.sh new file mode 100755 index 0000000..c5719f1 --- /dev/null +++ b/scripts/test-by-tag.sh @@ -0,0 +1,64 @@ +#!/bin/bash + +# Tag-Based Test Runner Script +# Runs BDD tests with specific tags + +set -e + +# Check if tag is provided +if [ $# -eq 0 ]; then + echo "❌ Usage: $0 [feature]" + echo "Examples:" + echo " $0 @smoke # Run all smoke tests" + echo " $0 @critical auth # Run critical auth tests" + echo " $0 @v2 greet # Run v2 greet tests" + exit 1 +fi + +TAG=$1 +FEATURE="" + +# Check if feature is also provided +if [ $# -ge 2 ]; then + FEATURE=$2 +fi + +SCRIPTS_DIR=$(dirname `realpath ${BASH_SOURCE[0]}`) +cd $SCRIPTS_DIR/.. + +echo "🧪 Running tests with tag: $TAG" + +if [ -n "$FEATURE" ]; then + echo "📁 Feature: $FEATURE" + + # Set feature-specific environment variables + DATABASE="dance_lessons_coach_${FEATURE}_test" + CONFIG="features/${FEATURE}/${FEATURE}-test-config.yaml" + + export DLC_DATABASE_HOST="localhost" + export DLC_DATABASE_PORT="5432" + export DLC_DATABASE_USER="postgres" + export DLC_DATABASE_PASSWORD="postgres" + export DLC_DATABASE_NAME="${DATABASE}" + export DLC_DATABASE_SSL_MODE="disable" + export DLC_CONFIG_FILE="${CONFIG}" + + # Run feature-specific tests with tag filtering + echo "🚀 Running tagged tests for ${FEATURE} feature..." + cd "features/${FEATURE}" + FEATURE=${FEATURE} go test -v -tags="$TAG" . +else + echo "🚀 Running tagged tests for all features..." + + # Run all tests with tag filtering + # Note: Godog tag filtering is done through the godog command line + # For Go test integration, we need to use a different approach + echo "⚠️ Tag filtering for all features requires godog command directly" + echo "📝 Running: godog --tags=$TAG features/" + + # This would require setting up the test server manually + # For now, we'll show how it would work + echo "⏳ This functionality would require additional implementation" + echo "💡 Consider using: godog --tags=$TAG features/" + echo " after starting the test server manually" +fi diff --git a/scripts/test-feature.sh b/scripts/test-feature.sh new file mode 100755 index 0000000..33c07fd --- /dev/null +++ b/scripts/test-feature.sh @@ -0,0 +1,168 @@ +#!/bin/bash + +# Feature-Specific Test Runner Script +# Runs BDD tests for a specific feature with proper isolation + +set -e + +# Check if feature name is provided +if [ $# -eq 0 ]; then + echo "❌ Usage: $0 " + echo "Available features: auth, config, greet, health, jwt" + exit 1 +fi + +FEATURE=$1 +SCRIPTS_DIR=$(dirname `realpath ${BASH_SOURCE[0]}`) +cd $SCRIPTS_DIR/.. + +# Validate feature name +case $FEATURE in + auth|config|greet|health|jwt) + echo "🧪 Setting up ${FEATURE} feature tests..." + ;; + *) + echo "❌ Invalid feature: $FEATURE" + echo "Available features: auth, config, greet, health, jwt" + exit 1 + ;; +esac + +# Feature-specific configuration +DATABASE="dance_lessons_coach_${FEATURE}_test" +CONFIG="features/${FEATURE}/${FEATURE}-test-config.yaml" +PORT=$(grep "port:" "$CONFIG" | awk '{print $2}') + +# Setup function +setup_feature_environment() { + echo "🧪 Setting up ${FEATURE} feature tests..." + + # Check if we're in CI environment + if [ -n "$GITHUB_ACTIONS" ] || [ -n "$GITEA_ACTIONS" ]; then + # CI environment - PostgreSQL is already running as a service + echo "🏗️ CI environment detected" + + # Create database if it doesn't exist + if ! psql -h postgres -p 5432 -U postgres -lqt | cut -d \| -f 1 | grep -qw "${DATABASE}"; then + echo "📦 Creating ${FEATURE} test database..." + createdb -h postgres -p 5432 -U postgres "${DATABASE}" + echo "✅ ${FEATURE} test database created successfully!" + else + echo "✅ ${FEATURE} test database already exists" + fi + else + # Local environment - use docker compose + echo "💻 Local environment detected" + + # Check if PostgreSQL container is running, start it if not + if ! docker ps --format '{{.Names}}' | grep -q "^dance-lessons-coach-postgres$"; then + echo "🐋 Starting PostgreSQL container..." + docker compose up -d postgres + + # Wait for PostgreSQL to be ready + echo "⏳ Waiting for PostgreSQL to be ready..." + max_attempts=30 + attempt=0 + while [ $attempt -lt $max_attempts ]; do + if docker exec dance-lessons-coach-postgres pg_isready -U postgres 2>/dev/null; then + echo "✅ PostgreSQL is ready!" + break + fi + attempt=$((attempt + 1)) + sleep 1 + done + + if [ $attempt -eq $max_attempts ]; then + echo "❌ PostgreSQL failed to start" + exit 1 + fi + else + echo "✅ PostgreSQL container is already running" + fi + + # Create feature-specific database + if docker exec dance-lessons-coach-postgres psql -U postgres -lqt | cut -d \| -f 1 | grep -qw "${DATABASE}"; then + echo "✅ ${FEATURE} test database already exists" + else + echo "📦 Creating ${FEATURE} test database..." + if docker exec dance-lessons-coach-postgres createdb -U postgres "${DATABASE}"; then + echo "✅ ${FEATURE} test database created successfully!" + else + echo "❌ Failed to create ${FEATURE} test database" + exit 1 + fi + fi + fi +} + +# Run tests function +run_feature_tests() { + echo "🚀 Running ${FEATURE} feature tests..." + + # Set feature-specific environment variables + export DLC_DATABASE_HOST="localhost" + export DLC_DATABASE_PORT="5432" + export DLC_DATABASE_USER="postgres" + export DLC_DATABASE_PASSWORD="postgres" + export DLC_DATABASE_NAME="${DATABASE}" + export DLC_DATABASE_SSL_MODE="disable" + export DLC_CONFIG_FILE="${CONFIG}" + + # Run tests with proper coverage measurement + set +e + test_output=$(go test ./features/${FEATURE}/... -v -cover -coverpkg=./... -coverprofile=coverage-${FEATURE}.out 2>&1) + test_exit_code=$? + set -e + + echo "$test_output" + + # Check for undefined steps + if echo "$test_output" | grep -q "undefined"; then + echo "❌ FAILED: Found undefined steps in ${FEATURE} tests" + exit 1 + fi + + # Check for pending steps + if echo "$test_output" | grep -q "pending"; then + echo "❌ FAILED: Found pending steps in ${FEATURE} tests" + exit 1 + fi + + # Check for skipped steps + if echo "$test_output" | grep -q "skipped"; then + echo "❌ FAILED: Found skipped steps in ${FEATURE} tests" + exit 1 + fi + + # Check if tests passed + if [ $test_exit_code -eq 0 ]; then + echo "✅ All ${FEATURE} feature tests passed successfully!" + return 0 + else + echo "❌ ${FEATURE} feature tests failed" + return 1 + fi +} + +# Cleanup function +cleanup_feature_environment() { + echo "🧹 Cleaning up ${FEATURE} feature tests..." + + # Check if we're in CI environment + if [ -n "$GITHUB_ACTIONS" ] || [ -n "$GITEA_ACTIONS" ]; then + # CI environment - drop database + echo "🗑️ Dropping ${FEATURE} test database..." + dropdb -h postgres -p 5432 -U postgres "${DATABASE}" 2>/dev/null || true + echo "✅ ${FEATURE} test database cleaned up" + else + # Local environment - drop database + echo "🗑️ Dropping ${FEATURE} test database..." + docker exec dance-lessons-coach-postgres dropdb -U postgres "${DATABASE}" 2>/dev/null || true + echo "✅ ${FEATURE} test database cleaned up" + fi +} + +# Main execution +setup_feature_environment +run_feature_tests +cleanup_feature_environment diff --git a/scripts/validate-isolation.sh b/scripts/validate-isolation.sh new file mode 100755 index 0000000..299d134 --- /dev/null +++ b/scripts/validate-isolation.sh @@ -0,0 +1,110 @@ +#!/bin/bash + +# Isolation Validation Script +# Validates that feature isolation is working correctly + +set -e + +echo "🔍 Validating BDD test isolation..." + +# Check feature directories exist +echo "📁 Checking feature directory structure..." +for feature in auth config greet health jwt; do + if [ ! -d "features/${feature}" ]; then + echo "❌ Missing features/${feature} directory" + exit 1 + fi + + # Check for feature files + if [ -z "$(find features/${feature} -name "*.feature" -type f)" ]; then + echo "❌ No feature files found in features/${feature}" + exit 1 + fi + + # Check for config files + if [ ! -f "features/${feature}/${feature}-test-config.yaml" ]; then + echo "❌ Missing config file for ${feature} feature" + exit 1 + fi + + echo "✅ ${feature} feature structure validated" +done + +# Check for unique ports +echo "🔌 Checking for unique port assignments..." +port_auth=$(grep "port:" "features/auth/auth-test-config.yaml" | awk '{print $2}') +port_config=$(grep "port:" "features/config/config-test-config.yaml" | awk '{print $2}') +port_greet=$(grep "port:" "features/greet/greet-test-config.yaml" | awk '{print $2}') +port_health=$(grep "port:" "features/health/health-test-config.yaml" | awk '{print $2}') +port_jwt=$(grep "port:" "features/jwt/jwt-test-config.yaml" | awk '{print $2}') + +# Check for port conflicts +if [ "$port_auth" = "$port_config" ] || [ "$port_auth" = "$port_greet" ] || [ "$port_auth" = "$port_health" ] || [ "$port_auth" = "$port_jwt" ]; then + echo "❌ Port conflict detected with auth port $port_auth" + exit 1 +fi + +if [ "$port_config" = "$port_greet" ] || [ "$port_config" = "$port_health" ] || [ "$port_config" = "$port_jwt" ]; then + echo "❌ Port conflict detected with config port $port_config" + exit 1 +fi + +if [ "$port_greet" = "$port_health" ] || [ "$port_greet" = "$port_jwt" ]; then + echo "❌ Port conflict detected with greet port $port_greet" + exit 1 +fi + +if [ "$port_health" = "$port_jwt" ]; then + echo "❌ Port conflict detected with health port $port_health" + exit 1 +fi + +echo "✅ All features have unique ports" + +# Check for unique database names +echo "🗃️ Checking for unique database names..." +db_auth="dance_lessons_coach_auth_test" +db_config="dance_lessons_coach_config_test" +db_greet="dance_lessons_coach_greet_test" +db_health="dance_lessons_coach_health_test" +db_jwt="dance_lessons_coach_jwt_test" + +# Check for database name conflicts +if [ "$db_auth" = "$db_config" ] || [ "$db_auth" = "$db_greet" ] || [ "$db_auth" = "$db_health" ] || [ "$db_auth" = "$db_jwt" ]; then + echo "❌ Database conflict detected with auth database" + exit 1 +fi + +if [ "$db_config" = "$db_greet" ] || [ "$db_config" = "$db_health" ] || [ "$db_config" = "$db_jwt" ]; then + echo "❌ Database conflict detected with config database" + exit 1 +fi + +if [ "$db_greet" = "$db_health" ] || [ "$db_greet" = "$db_jwt" ]; then + echo "❌ Database conflict detected with greet database" + exit 1 +fi + +if [ "$db_health" = "$db_jwt" ]; then + echo "❌ Database conflict detected with health database" + exit 1 +fi + +echo "✅ All features have unique database names" + +# Test that each feature can be run independently +echo "🧪 Testing feature independence..." +for feature in auth config greet health jwt; do + echo "Testing ${feature} feature..." + + # Try to run the feature test script with setup only + if ! bash scripts/test-feature.sh $feature 2>&1 | grep -q "Setting up ${feature} feature tests"; then + echo "❌ Failed to setup ${feature} feature tests" + exit 1 + fi + + echo "✅ ${feature} feature can be set up independently" +done + +echo "✅ All isolation validations passed!" +echo "🎉 BDD test isolation is working correctly"