dance-lessons-coach/pkg/bdd/testserver/CONFIG_SCHEMA.md

# BDD Test Configuration Schema

## Overview

This document describes the configuration architecture for BDD tests in the dance-lessons-coach project.
It establishes a clear hierarchy and flow of configuration parameters to ensure predictable, maintainable,
and isolated test execution.

## Configuration Sources (Priority Order)

### 1. Explicit Parameters (Highest Priority)
Passed directly between components with no hidden behavior:
- `FEATURE`: Which feature is being tested (`greet`, `config`, `auth`, `health`, `jwt`)
- `GODOG_TAGS`: Scenario tag filters (e.g., `@v2`, `~@flaky`, `~@todo`)
- `Config` struct: Passed explicitly to server initialization

### 2. Feature-Specific Configuration Files
Loaded from filesystem when testing specific features:
- Path: `features/{FEATURE}/{FEATURE}-test-config.yaml`
- Used by: Config hot-reload tests only
- Monitored by: `testserver.monitorConfigFile()`
- Example: `features/config/config-test-config.yaml`

### 3. Environment Variables (External Control Only)
Set by test scripts and CI/CD, **NOT read deep in implementation code**:

| Variable | Purpose | Default | Set By |
|----------|---------|---------|-------|
| `DLC_API_V2_ENABLED` | Enable v2 API globally | `false` | Test scripts |
| `BDD_SCHEMA_ISOLATION` | Enable per-scenario database schema isolation | `false` | Test scripts, validate-test-suite.sh |
| `BDD_ENABLE_CLEANUP_LOGS` | Enable detailed cleanup logging | `false` | Test scripts |
| `BDD_TRACE_STATE` | Enable state tracing | `false` | Test scripts |
| `FIXED_TEST_PORT` | Use fixed port instead of random | `false` | Test scripts |
| `FEATURE` | Current feature under test | `""` | testsetup.CreateTestSuite |
| `GODOG_TAGS` | Tag filter for scenario selection | `"~@flaky && ~@todo && ~@skip"` | CreateTestSuite |

### 4. Hardcoded Defaults (Fallback)
Used when no other source provides a value:
- Port: Random in range 10000-19999 (or 9191 if FIXED_TEST_PORT=true)
- JWT Secret: `test-secret-key-for-bdd-tests`
- Database: localhost:5432, postgres/postgres, dance_lessons_coach
- Logging Level: debug
- v2_enabled: false

## Configuration Layers (Mermaid Diagram)

```mermaid
flowchart TB
    subgraph TestExecutionControl["Test Execution Control
    (Shell/Script Layer)"]
        A1[Environment Variables]
        A2[DLC_API_V2_ENABLED]
        A3[BDD_SCHEMA_ISOLATION]
        A4[BDD_ENABLE_CLEANUP_LOGS]
        A5[FEATURE]
        A6[GODOG_TAGS]
    end

    subgraph TestSuiteSetup["Test Suite Setup
    (pkg/bdd/testsetup)"]
        B1[CreateTestSuite]
        B2[Set FEATURE]
        B3[Set GODOG_TAGS]
        B4[Configure godog.Options]
    end

    subgraph ServerSetup["Server Setup
    (pkg/bdd/suite)"]
        C1[InitializeTestSuite]
        C2[Create sharedServer]
        C3[InitializeScenario]
    end

    subgraph ServerConfiguration["Server Configuration
    (pkg/bdd/testserver)"]
        D1[Server.Start]
        D2[shouldEnableV2]
        D3[createTestConfig]
        D4[monitorConfigFile]
        D5[ReloadConfig]
        D6[loadConfigFromFile]
    end

    subgraph ScenarioExecution["Scenario Execution
    (pkg/bdd/steps)"]
        E1[BeforeScenario]
        E2[SetScenarioKey]
        E3[Execute Steps]
        E4[AfterScenario]
        E5[ClearScenarioState]
    end

    A1 --> B1
    A2 --> D2
    A3 --> D1
    A4 --> D1
    A5 --> B2
    A5 --> D2
    A6 --> B3
    A6 --> D2

    B1 --> C1
    B2 --> C1
    B3 --> C1
    B4 --> C1

    C1 --> D1
    C2 --> D1
    C3 --> E1

    D1 --> D4
    D2 --> D3
    D3 --> D1
    D4 --> D5
    D5 --> D1
    D5 --> D6
    D6 --> D3

    D1 --> E1
    E1 --> E2
    E2 --> E3
    E3 --> E4
    E4 --> E5

    classDef external fill:#09f,stroke:#333
    classDef setup fill:#08f,stroke:#333
    classDef server fill:#090,stroke:#333
    classDef scenario fill:#000,stroke:#333

    class A1,A2,A3,A4,A5,A6 external
    class B1,B2,B3,B4 setup
    class C1,C2,C3 setup
    class D1,D2,D3,D4,D5,D6 server
    class E1,E2,E3,E4,E5 scenario
```

## Configuration Flow (Mermaid Sequence Diagram)

```mermaid
sequenceDiagram
    participant Script as Test Script
    participant TestSetup as testsetup
    participant Suite as suite.go
    participant Server as testserver
    participant ConfigFile as Config File
    participant Steps as Step Definitions

    Script->>Script: Set env vars (BDD_*, DLC_*)
    Script->>TestSetup: Run go test ./features/{feature}
    
    TestSetup->>TestSetup: Read FEATURE from env
    TestSetup->>TestSetup: Read GODOG_TAGS from env
    TestSetup->>Suite: CreateTestSuite(FEATURE, tags)
    
    Suite->>Server: InitializeTestSuite -> NewServer()
    Server->>Server: shouldEnableV2() checks FEATURE+GODOG_TAGS
    Server->>Server: createTestConfig(port, v2Enabled)
    Server->>Server: Start()
    Server->>Server: Start monitorConfigFile() goroutine
    
    Suite->>Suite: InitializeScenario
    Suite->>Steps: Create step context
    
    loop Each Scenario
        Suite->>Server: BeforeScenario: SetupSchemaIsolation
        Suite->>Steps: SetScenarioKeyForAllSteps
        Steps->>Steps: Clear scenario state
        
        Steps->>Server: Execute step requests
        
        alt Config Feature + File Modified
            ConfigFile->>Server: File modification detected
            Server->>Server: ReloadConfig()
            Server->>ConfigFile: loadConfigFromFile()
            Server->>Server: Restart with new config
        end
        
        Suite->>Server: AfterScenario: Cleanup
        Suite->>Steps: ClearScenarioState
    end
```

## Use Cases

### UC-1: Default Test Run (No v2, No Config File)
```
Input:     go test ./features/greet
FEATURE:   greet
GODOG_TAGS: ~@flaky && ~@todo && ~@skip
Config Source: createTestConfig(port)
v2_enabled: false
Result: v1 scenarios pass, v2 scenarios skipped by tag filter
```

### UC-2: v2 API Tests (Split Test Suite)
```
Input:     go test ./features/greet (with GODOG_TAGS="@v2" in v2 subtest)
FEATURE:   greet
GODOG_TAGS: @v2 && ~@skip
Config Source: createTestConfig(port) with v2 check
v2_enabled: true (because FEATURE=greet AND tags contain @v2)
Result: v2 scenarios execute with v2 API available

Flow:
1. TestGreetBDD runs v1 subtest with tags="~@v2"
2. TestGreetBDD runs v2 subtest with tags="@v2"
3. Each subtest starts its own server
4. Server in v2 subtest has v2_enabled=true
5. v2 scenarios pass
```

### UC-3: Config Hot Reload Tests
```
Input:     go test ./features/config
FEATURE:   config
GODOG_TAGS: ~@flaky && ~@todo && ~@skip
Config File: features/config/config-test-config.yaml
Config Monitor: Watches config file for changes

When config file is modified:
1. monitorConfigFile() detects file change via mod time
2. Calls ReloadConfig()
3. ReloadConfig() for FEATURE=config: loads from config file
4. Server restarts with new config
5. Subsequent scenarios see new configuration

Note: This is the ONLY feature that uses config file hot-reload.
      All other features use hardcoded/test defaults.
```

### UC-4: Config Hot Reload with v2 Enable
```
Scenario: Hot reloading feature flags
Steps:
1. Server starts with default config (v2_enabled: false)
2. Test sets v2_enabled: true in config file
3. Config monitor detects change
4. ReloadConfig() called
5. Server loads from config file (NOT createTestConfig)
6. Server restarts with v2_enabled: true
7. Test verifies v2 API works

Current Bug: ReloadConfig() calls createTestConfig() which:
- Reads FEATURE=config
- Reads GODOG_TAGS (doesn't contain @v2)
- Sets v2_enabled: false
- Overrides the config file setting!

Fix: ReloadConfig() must load from file for config feature.
```

## Implementation Details

### Config Creation Flow

```go
// pkg/bdd/testserver/server.go

func NewServer() *Server {
    port := getRandomPort() // 10000-19999
    return &Server{port: port}
}

func (s *Server) Start() error {
    cfg := createTestConfig(s.port)
    // ... start server with cfg
    go s.monitorConfigFile()
}

// CURRENT - BAD
func createTestConfig(port int) *config.Config {
    feature := os.Getenv("FEATURE")
    tags := os.Getenv("GODOG_TAGS")
    
    enableV2 := false
    if feature == "greet" && strings.Contains(tags, "@v2") {
        enableV2 = true
    }
    // ...
    return &config.Config{
        API: config.APIConfig{V2Enabled: enableV2},
        // ...
    }
}

// PROPOSED - GOOD
func createTestConfig(port int, opts ConfigOptions) *config.Config {
    defaults := &config.Config{
        Server: config.ServerConfig{Host: "0.0.0.0", Port: port},
        // ... all hardcoded defaults
    }
    
    // Apply explicit options (passed from caller)
    if opts.V2Enabled {
        defaults.API.V2Enabled = true
    }
    
    return defaults
}

// ConfigOptions passed from testsuite
type ConfigOptions struct {
    V2Enabled      bool
    UseConfigFile  bool
    ConfigFilePath string
}
```

### Reload Flow Fix

```go
// pkg/bdd/testserver/server.go

func (s *Server) ReloadConfig() error {
    feature := os.Getenv("FEATURE")
    
    if feature == "config" && s.configFilePath != "" {
        // For config tests: load from monitored file
        cfg, err := loadConfigFromFile(s.configFilePath)
        if err != nil {
            return err
        }
        return s.applyConfig(cfg)
    }
    
    // For all other features: use defaults
    // (hot reload not supported for non-config features)
    cfg := createDefaultConfig(s.port)
    return s.applyConfig(cfg)
}

func loadConfigFromFile(path string) (*config.Config, error) {
    v := viper.New()
    v.SetConfigFile(path)
    v.SetConfigType("yaml")
    
    if err := v.ReadInConfig(); err != nil {
        return nil, err
    }
    
    var cfg config.Config
    if err := v.Unmarshal(&cfg); err != nil {
        return nil, err
    }
    
    // Apply hardcoded values that should NOT come from file
    // (database connection for BDD tests, etc.)
    cfg.Database.Host = getDatabaseHost()
    cfg.Database.Port = getDatabasePort()
    cfg.Database.User = "postgres"
    cfg.Database.Password = "postgres"
    cfg.Database.Name = "dance_lessons_coach"
    
    return &cfg, nil
}
```

## Configuration File Format

### Config Test File (features/config/config-test-config.yaml)
```yaml
server:
  host: "127.0.0.1"
  port: 9191

logging:
  level: "info"
  json: false

api:
  v2_enabled: false  # Will be toggled by tests

telemetry:
  enabled: true
  sampler:
    type: "parentbased_always_on"
    ratio: 1.0

auth:
  jwt:
    ttl: 1h

database:
  # These are OVERRIDDEN by BDD test infrastructure
  host: "localhost"
  port: 5432
  user: "postgres"
  password: "postgres"
  name: "dance_lessons_coach_bdd_test"
  ssl_mode: "disable"
```

## State Isolation

### Per-Scenario State
- Managed by: `pkg/bdd/steps/scenario_state.go`
- Key: SHA256 hash of scenario URI + name
- State includes: LastToken, FirstToken, LastUserID, LastSecret, LastError
- Cleared: At start of each scenario in BeforeScenario hook

### Database Schema Isolation
- Enabled by: `BDD_SCHEMA_ISOLATION=true`
- Mechanism: Creates unique schema per scenario
- Schema name: `test_{sha256(scenarioKey)[:8]}`
- Search path: Set via `SET search_path TO ...`
- Cleanup: Schema dropped after scenario

### Server-Level State Reset
- JWT secrets: Reset after every scenario via `ResetJWTSecrets()`
- Database: Cleaned up after every scenario
- Auth state: Per-scenario via state manager

## Package Responsibilities

### pkg/bdd/testserver
- **Purpose**: Test HTTP server management
- **Responsibilities**:
  - Server lifecycle (Start, Stop)
  - Configuration loading and reloading
  - Database cleanup
  - Schema isolation
  - JWT secret management
  - Config file monitoring (config feature only)

### pkg/bdd/testsetup  
- **Purpose**: Godog test suite setup
- **Responsibilities**:
  - Feature test file discovery
  - Test suite configuration
  - Tag filtering
  - godog options setup

### pkg/bdd/suite
- **Purpose**: Test suite initialization hooks
- **Responsibilities**:
  - BeforeSuite/AfterSuite hooks
  - BeforeScenario/AfterScenario hooks
  - Step context creation
  - State isolation setup

### pkg/bdd/steps
- **Purpose**: Step definitions
- **Responsibilities**:
  - All Gherkin step implementations
  - Per-scenario state management
  - Per-feature step organization

## Migration Plan

### Phase 1: Fix Config Reload (Urgent)
1. Create `loadConfigFromFile()` function
2. Modify `ReloadConfig()` to use file for config feature
3. Add tests to verify config hot-reload works

### Phase 2: Clean Up Config Creation
1. Create `ConfigOptions` struct
2. Modify `createTestConfig()` to accept options
3. Update callers to pass explicit options
4. Remove env var reading from deep in config creation

### Phase 3: Document and Validate
1. Write comprehensive documentation (this file)
2. Add validation tests for all use cases
3. Create troubleshooting guide

### Phase 4: Consider Package Merge (Optional)
1. Evaluate merging testserver + testsetup
2. Design new `pkg/bdd/testing` package structure
3. Migrate code incrementally

## Rules for Adding New Configuration

1. **Prefer explicit parameters** over environment variables
2. **Read env vars at ONE layer only** (typically test entry point)
3. **Document all config sources** in this file
4. **Test config combinations** to prevent override bugs
5. **Never read env vars in hot paths** (scenario steps, server handlers)

## Troubleshooting

### Symptom: Config file changes not applied
- Check: Is FEATURE=config?
- Check: Does config file exist at `features/config/config-test-config.yaml`?
- Check: Does monitorConfigFile() detect the change?
- Fix: ReloadConfig() must load from file, not createTestConfig()

### Symptom: v2 tests fail with 404
- Check: Is FEATURE=greet?
- Check: Does GODOG_TAGS contain @v2?
- Check: Does createTestConfig() see the tags?
- Fix: Ensure tags are set before server creation

### Symptom: State pollution between scenarios
- Check: Is schema isolation enabled?
- Check: Are step definitions using per-scenario state?
- Fix: Use ScenarioState for all mutable state

## References

- [Godog Documentation](https://github.com/cucumber/godog)
- [pkg/config/config.go](../config/config.go) - Config struct definitions
- [pkg/bdd/testsetup/testsetup.go](../testsetup/testsetup.go) - Test suite creation
- [pkg/bdd/suite.go](../suite.go) - Test hooks
- [ADR-0008: BDD Testing](../adr/0008-bdd-testing.md)