- Add pkg/bdd/steps/scenario_state.go with thread-safe per-scenario state manager - Update auth_steps.go, jwt_retention_steps.go to use per-scenario state accessors - Add LastSecret and LastError fields to ScenarioState for JWT retention testing - Update steps.go with SetScenarioKeyForAllSteps function - Update suite.go to generate scenario keys and clear state properly - Mark config hot-reload scenarios as @flaky (timing-sensitive) - Fix validate-test-suite.sh: add -p 1 flag for sequential execution, filter JSON logs, add --count flag - Add CONFIG_SCHEMA.md documenting configuration architecture - Split greet tests into v1/v2 sub-tests with explicit v2 enable/disable Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
252 lines
7.7 KiB
Markdown
252 lines
7.7 KiB
Markdown
# BDD Steps Organization
|
|
|
|
This folder contains the step definitions for the BDD tests, organized by domain for better maintainability and scalability.
|
|
|
|
## Structure
|
|
|
|
```
|
|
pkg/bdd/steps/
|
|
├── steps.go # Main registration file that ties everything together
|
|
├── scenario_state.go # Per-scenario state isolation manager
|
|
├── common_steps.go # Shared steps used across multiple domains
|
|
├── auth_steps.go # Authentication and user management steps
|
|
├── config_steps.go # Configuration and hot-reloading steps
|
|
├── greet_steps.go # Greet-related steps (v1 and v2 API)
|
|
├── health_steps.go # Health check and server status steps
|
|
├── jwt_retention_steps.go # JWT secret retention policy steps
|
|
└── README.md # This file
|
|
```
|
|
|
|
## Design Principles
|
|
|
|
1. **Domain Separation**: Steps are grouped by functional domain
|
|
2. **Single Responsibility**: Each file focuses on a specific area of functionality
|
|
3. **Reusability**: Common steps are shared via `common_steps.go`
|
|
4. **Scalability**: Easy to add new domains as the application grows
|
|
5. **State Isolation**: Use per-scenario state to prevent pollution between test scenarios
|
|
|
|
## Adding New Steps
|
|
|
|
1. **For new domains**: Create a new `*_steps.go` file following the existing pattern
|
|
2. **For existing domains**: Add to the appropriate domain file
|
|
3. **For shared functionality**: Add to `common_steps.go`
|
|
4. **Register all steps**: Update `steps.go` to include the new steps
|
|
|
|
## Step Naming Convention
|
|
|
|
- Use descriptive, action-oriented names
|
|
- Follow the pattern: `i[Action][Object]` or `the[Object][State]`
|
|
- Example: `iRequestAGreetingFor`, `theAuthenticationShouldBeSuccessful`
|
|
- Use present tense for actions: "I authenticate", "the server reloads"
|
|
|
|
## State Isolation Pattern
|
|
|
|
**Problem:** Step definition structs (AuthSteps, GreetSteps, etc.) maintain state in their fields (e.g., `lastToken`, `lastUserID`). This state persists across all scenarios in a test process, causing pollution even with database schema isolation.
|
|
|
|
**Solution:** Use the `ScenarioState` manager for per-scenario state isolation.
|
|
|
|
### How It Works
|
|
|
|
The `scenario_state.go` provides a thread-safe mechanism to store and retrieve state that is isolated per scenario:
|
|
|
|
```go
|
|
// Get scenario-specific state
|
|
state := steps.GetScenarioState(scenarioName)
|
|
|
|
// Store scenario-specific data
|
|
state.LastToken = token
|
|
state.LastUserID = userID
|
|
|
|
// Retrieve scenario-specific data
|
|
token := state.LastToken
|
|
```
|
|
|
|
### Usage in Step Definitions
|
|
|
|
Instead of storing state in struct fields:
|
|
|
|
```go
|
|
// ❌ NOT RECOMMENDED - state shared across all scenarios
|
|
type AuthSteps struct {
|
|
client *testserver.Client
|
|
lastToken string // Shared across all scenarios!
|
|
lastUserID uint // Shared across all scenarios!
|
|
}
|
|
|
|
func (s *AuthSteps) iShouldReceiveAValidJWTToken() error {
|
|
s.lastToken = extractedToken // Pollutes other scenarios
|
|
return nil
|
|
}
|
|
```
|
|
|
|
Use per-scenario state:
|
|
|
|
```go
|
|
// ✅ RECOMMENDED - state isolated per scenario
|
|
type AuthSteps struct {
|
|
client *testserver.Client
|
|
scenarioName string // Track current scenario for state isolation
|
|
}
|
|
|
|
func (s *AuthSteps) iShouldReceiveAValidJWTToken() error {
|
|
state := steps.GetScenarioState(s.scenarioName)
|
|
state.LastToken = extractedToken // Isolated to this scenario
|
|
return nil
|
|
}
|
|
```
|
|
|
|
### Integration with Suite Hooks
|
|
|
|
Clear state in AfterScenario to prevent memory growth:
|
|
|
|
```go
|
|
sc.AfterScenario(func(s *godog.Scenario, err error) {
|
|
scenarioKey := s.Name
|
|
if s.Uri != "" {
|
|
scenarioKey = fmt.Sprintf("%s:%s", s.Uri, s.Name)
|
|
}
|
|
steps.ClearScenarioState(scenarioKey)
|
|
})
|
|
```
|
|
|
|
### ScenarioState Structure
|
|
|
|
The `ScenarioState` struct contains common fields needed across step definitions:
|
|
|
|
```go
|
|
type ScenarioState struct {
|
|
LastToken string
|
|
FirstToken string
|
|
LastUserID uint
|
|
// Add more fields as needed for other step types
|
|
}
|
|
```
|
|
|
|
If you need additional scenario-scoped fields, add them to the `ScenarioState` struct.
|
|
|
|
## Testing the Steps
|
|
|
|
Run BDD tests with:
|
|
```bash
|
|
# Run all features
|
|
go test ./features/... -v
|
|
|
|
# Run specific feature
|
|
go test ./features/auth -v
|
|
|
|
# Run with state tracing enabled
|
|
BDD_TRACE_STATE=1 go test ./features/auth -v
|
|
|
|
# Validate full test suite
|
|
./scripts/validate-test-suite.sh 1
|
|
```
|
|
|
|
## State Cleanup Strategy
|
|
|
|
| Cleanup Level | When | What | Implementation |
|
|
|---------------|------|------|----------------|
|
|
| Per-Scenario | After each scenario | Step struct fields | `ClearScenarioState()` |
|
|
| Per-Scenario | After each scenario | Database state | `CleanupDatabase()` (if no schema isolation) |
|
|
| Per-Scenario | After each scenario | Schema | `DROP SCHEMA` (if schema isolation enabled) |
|
|
| Per-Process | After each feature test | Server-level state | `ResetJWTSecrets()` |
|
|
| Per-Suite | After all scenarios | All state | Server restart |
|
|
|
|
## Best Practices
|
|
|
|
### 1. Use Per-Scenario State for Shared Data
|
|
|
|
Any data that:
|
|
- Is modified during scenario execution
|
|
- Affects subsequent steps in the same scenario
|
|
- Should NOT affect other scenarios
|
|
|
|
**Use:** `GetScenarioState(scenarioName).Field`
|
|
|
|
### 2. Keep Step Definitions Stateless Where Possible
|
|
|
|
If a step doesn't need to store intermediate state, don't store it:
|
|
```go
|
|
// ✅ Good - stateless
|
|
func (s *GreetSteps) iRequestAGreetingFor(name string) error {
|
|
return s.client.Request("GET", fmt.Sprintf("/api/v1/greet/%s", name), nil)
|
|
}
|
|
|
|
// ❌ Avoid - unnecessary state
|
|
func (s *GreetSteps) iRequestAGreetingFor(name string) error {
|
|
s.lastGreetedName = name // Unnecessary unless used later
|
|
return s.client.Request("GET", fmt.Sprintf("/api/v1/greet/%s", name), nil)
|
|
}
|
|
```
|
|
|
|
### 3. Prefix Config Files Per-Scenario
|
|
|
|
If your scenario modifies config files, use scenario-specific paths:
|
|
```go
|
|
configPath := fmt.Sprintf("features/%s/%s-scenario-%s.yaml",
|
|
feature, feature, scenarioKey)
|
|
```
|
|
|
|
### 4. Document Dependencies
|
|
|
|
If a step depends on state set by another step, document it:
|
|
|
|
```go
|
|
// Step: The user should have a valid JWT token
|
|
// Requires: iAuthenticateWithUsernameAndPassword to have been called first
|
|
func (s *AuthSteps) theUserShouldHaveAValidJWTToken() error {
|
|
state := steps.GetScenarioState(s.scenarioName)
|
|
if state.LastToken == "" {
|
|
return fmt.Errorf("no token found - did you authenticate first?")
|
|
}
|
|
// Verify token is valid...
|
|
}
|
|
```
|
|
|
|
## Future Domains
|
|
|
|
As the application grows, consider adding:
|
|
- `payment_steps.go` - Payment processing steps
|
|
- `notification_steps.go` - Notification and email steps
|
|
- `admin_steps.go` - Admin-specific functionality steps
|
|
- `api_steps.go` - General API interaction patterns
|
|
- `user_steps.go` - User profile and management steps (if auth gets complex)
|
|
|
|
## Troubleshooting
|
|
|
|
### State Pollution Between Scenarios
|
|
|
|
**Symptom:** Tests pass individually but fail when run together
|
|
|
|
**Check:**
|
|
1. Are you using struct fields to store state? → Use `ScenarioState` instead
|
|
2. Are database tables being cleaned up? → Verify `CleanupDatabase()` or schema isolation
|
|
3. Are JWT secrets being reset? → Verify `ResetJWTSecrets()` is called
|
|
|
|
**Debug:** Enable state tracing:
|
|
```bash
|
|
BDD_TRACE_STATE=1 go test ./features/auth -v
|
|
```
|
|
|
|
### Timeout or Delay Issues
|
|
|
|
**Symptom:** Config reloading tests fail intermittently
|
|
|
|
**Cause:** Server monitors config files every 1 second
|
|
|
|
**Fix:** Add delays >1100ms after config file changes:
|
|
```go
|
|
time.Sleep(1100 * time.Millisecond) // Wait for monitoring cycle
|
|
```
|
|
|
|
### Missing Step Definitions
|
|
|
|
**Symptom:** `undefined step` error
|
|
|
|
**Check:**
|
|
1. Step is defined in the appropriate `*_steps.go` file
|
|
2. Step is registered in `steps.go`
|
|
3. Step regex matches the feature file text exactly
|
|
4. No typos in the step name
|
|
|
|
**Tip:** Run with `-v` to see which step is undefined
|