dance-lessons-coach/.vibe/skills/bdd-testing/references/TEST_SERVER.md

# Test Server Implementation Guide

Complete guide to implementing the hybrid in-process test server for BDD testing.

## Architecture Overview

### Hybrid In-Process Testing

```mermaid
graph TD
    A[BDD Tests] -->|HTTP Requests| B[Test Server]
    B -->|Uses Real Code| C[Actual Server Implementation]
    C -->|Same Process| A
```

**Key Benefits:**
- No external process management
- Real server behavior
- Fast execution
- Reliable startup/shutdown

## Implementation

### Server Structure

```go
// pkg/bdd/testserver/server.go
type Server struct {
    httpServer  *http.Server
    port       int
    baseURL    string
}
```

### Server Construction

```go
func NewServer() *Server {
    return &Server{
        port: 9191,  // Fixed port for consistency
    }
}
```

### Server Startup

```go
func (s *Server) Start() error {
    s.baseURL = fmt.Sprintf("http://localhost:%d", s.port)
    
    // Create real server instance
    cfg := createTestConfig(s.port)
    realServer := server.NewServer(cfg, context.Background())
    
    // Configure HTTP server
    s.httpServer = &http.Server{
        Addr:    fmt.Sprintf(":%d", s.port),
        Handler: realServer.Router(),  // Use real router!
    }
    
    // Start server in goroutine
    go func() {
        if err := s.httpServer.ListenAndServe(); err != nil {
            if err != http.ErrServerClosed {
                log.Error().Err(err).Msg("Test server failed")
            }
        }
    }()
    
    // Wait for server to be ready
    return s.waitForServerReady()
}
```

### Readiness Verification

```go
func (s *Server) waitForServerReady() error {
    maxAttempts := 30
    
    for attempt := 0; attempt < maxAttempts; attempt++ {
        resp, err := http.Get(fmt.Sprintf("%s/api/ready", s.baseURL))
        
        if err == nil && resp.StatusCode == http.StatusOK {
            resp.Body.Close()
            return nil
        }
        
        if resp != nil {
            resp.Body.Close()
        }
        
        time.Sleep(100 * time.Millisecond)
    }
    
    return fmt.Errorf("server did not become ready after %d attempts", maxAttempts)
}
```

### Graceful Shutdown

```go
func (s *Server) Stop() error {
    if s.httpServer == nil {
        return nil
    }
    
    // Graceful shutdown with timeout
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()
    
    return s.httpServer.Shutdown(ctx)
}
```

## Configuration

### Test Configuration Factory

```go
func createTestConfig(port int) *config.Config {
    return &config.Config{
        Server: config.ServerConfig{
            Host: "localhost",
            Port: port,
        },
        Shutdown: config.ShutdownConfig{
            Timeout: 5 * time.Second,
        },
        Logging: config.LoggingConfig{
            JSON:   false,
            Level:  "trace",
        },
        Telemetry: config.TelemetryConfig{
            Enabled: false,  // Disable telemetry in tests
        },
    }
}
```

## Client Implementation

### HTTP Client

```go
// pkg/bdd/testserver/client.go
type Client struct {
    server   *Server
    lastResp *http.Response
    lastBody []byte
}

func NewClient(server *Server) *Client {
    return &Client{
        server: server,
    }
}
```

### Request Method

```go
func (c *Client) Request(method, path string, body []byte) error {
    url := c.server.GetBaseURL() + path
    
    req, err := http.NewRequest(method, url, nil)
    if err != nil {
        return fmt.Errorf("failed to create request: %w", err)
    }
    
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return fmt.Errorf("request failed: %w", err)
    }
    defer resp.Body.Close()
    
    c.lastResp = resp
    c.lastBody, err = io.ReadAll(resp.Body)
    
    return err
}
```

### Response Validation

```go
func (c *Client) ExpectResponseBody(expected string) error {
    if c.lastResp == nil {
        return fmt.Errorf("no response received")
    }
    
    actual := string(c.lastBody)
    actual = strings.TrimSuffix(actual, "\n")  // Critical: trim newline!
    
    if actual != expected {
        return fmt.Errorf("expected response body %q, got %q", expected, actual)
    }
    
    return nil
}

func (c *Client) ExpectStatusCode(expected int) error {
    if c.lastResp == nil {
        return fmt.Errorf("no response received")
    }
    
    if c.lastResp.StatusCode != expected {
        return fmt.Errorf("expected status %d, got %d", 
            expected, c.lastResp.StatusCode)
    }
    
    return nil
}
```

## Test Suite Integration

### Shared Server Pattern

```go
// pkg/bdd/suite.go
var sharedServer *testserver.Server

func InitializeTestSuite(ctx *godog.TestSuiteContext) {
    ctx.BeforeSuite(func() {
        sharedServer = testserver.NewServer()
        if err := sharedServer.Start(); err != nil {
            panic(err)
        }
    })
    
    ctx.AfterSuite(func() {
        if sharedServer != nil {
            sharedServer.Stop()
        }
    })
}

func InitializeScenario(ctx *godog.ScenarioContext) {
    client := testserver.NewClient(sharedServer)
    steps.InitializeAllSteps(ctx, client)
}
```

### Dedicated Server Pattern (for shutdown tests)

```go
func InitializeShutdownTestSuite(ctx *godog.TestSuiteContext) {
    // No shared server for shutdown tests
}

func InitializeShutdownScenario(ctx *godog.ScenarioContext) {
    server := testserver.NewServer()
    client := testserver.NewClient(server)
    
    ctx.BeforeScenario(func(*godog.Scenario) {
        if err := server.Start(); err != nil {
            panic(err)
        }
    })
    
    ctx.AfterScenario(func(*godog.Scenario, error) {
        server.Stop()
    })
    
    shutdown_steps.InitializeShutdownSteps(ctx, client, server)
}
```

## Debugging Techniques

### Server Health Checks

```bash
# Check if server is running
curl http://localhost:9191/api/ready

# Check health endpoint
curl http://localhost:9191/api/health

# Test greet endpoint
curl http://localhost:9191/api/v1/greet/John
```

### Common Server Issues

| Issue | Cause | Solution |
|-------|-------|----------|
| Connection refused | Server not started | Check BeforeSuite hook |
| Port already in use | Previous test crashed | Kill process on port 9191 |
| Server not ready | Startup timeout | Increase maxAttempts in waitForServerReady |
| Wrong responses | Configuration issue | Verify createTestConfig values |

### Debugging Server Startup

```go
// Add debug logging to waitForServerReady
func (s *Server) waitForServerReady() error {
    for attempt := 0; attempt < 30; attempt++ {
        log.Debug().Int("attempt", attempt+1).Msg("Checking server readiness")
        
        resp, err := http.Get(s.baseURL + "/api/ready")
        
        if err != nil {
            log.Debug().Err(err).Msg("Server not ready yet")
        } else {
            log.Debug().Int("status", resp.StatusCode).Msg("Server responded")
            resp.Body.Close()
            if resp.StatusCode == http.StatusOK {
                log.Info().Msg("Server is ready")
                return nil
            }
        }
        
        time.Sleep(100 * time.Millisecond)
    }
    
    return fmt.Errorf("server never became ready")
}
```

## Performance Optimization

### Connection Reuse

```go
// Create reusable HTTP client
var testClient = &http.Client{
    Timeout: 30 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        10,
        IdleConnTimeout:     90 * time.Second,
        DisableKeepAlives:   false,
        DisableCompression:  true,
    },
}

// Use in client requests
resp, err := testClient.Do(req)
```

### Parallel Test Execution

```go
// pkg/bdd/bdd_test.go
func TestBDD(t *testing.T) {
    suite := godog.TestSuite{
        Name:                 "dance-lessons-coach BDD Tests",
        TestSuiteInitializer: bdd.InitializeTestSuite,
        ScenarioInitializer:   bdd.InitializeScenario,
        Options: &godog.Options{
            Format:   "progress",
            Paths:    []string{"."},
            TestingT:      t,
			Strict: true,
			Randomize:     -1,
			StopOnFailure: true,
            // Enable parallel execution
            Concurrency: 4,  // Number of parallel scenarios
        },
    }
    
    if suite.Run() != 0 {
        t.Fatal("non-zero status returned, failed to run BDD tests")
    }
}
```

## Advanced Patterns

### Dynamic Port Allocation

**Not recommended** for our use case, but possible:

```go
func findFreePort() (int, error) {
    addr, err := net.ResolveTCPAddr("tcp", "localhost:0")
    if err != nil {
        return 0, err
    }
    
    l, err := net.ListenTCP("tcp", addr)
    if err != nil {
        return 0, err
    }
    defer l.Close()
    
    return l.Addr().(*net.TCPAddr).Port, nil
}
```

### Multiple Server Instances

```go
// For testing different configurations
type ServerConfig struct {
    Port    int
    Timeout time.Duration
    Logging bool
}

func NewServerWithConfig(config ServerConfig) *Server {
    return &Server{
        port: config.Port,
        // ...
    }
}
```

### Custom Middleware

```go
// Add test-specific middleware
func (s *Server) Start() error {
    // ... existing setup ...
    
    // Add test middleware
    handler := s.httpServer.Handler
    s.httpServer.Handler = addTestMiddleware(handler)
    
    // ... rest of startup ...
}

func addTestMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Add test headers, logging, etc.
        w.Header().Set("X-Test-Server", "true")
        next.ServeHTTP(w, r)
    })
}
```

## Security Considerations

### Test Server Isolation

```go
// Bind to localhost only
func (s *Server) Start() error {
    s.httpServer.Addr = "localhost:9191"  // localhost only!
    // ...
}
```

### Sensitive Data Handling

```go
// Scrub sensitive data from test responses
func scrubSensitiveData(body []byte) []byte {
    // Remove API keys, tokens, etc.
    return bytes.ReplaceAll(body, []byte("api-key-"), []byte("REDACTED-"))
}
```

### Resource Cleanup

```go
// Ensure proper cleanup in AfterSuite
func InitializeTestSuite(ctx *godog.TestSuiteContext) {
    ctx.AfterSuite(func() {
        if sharedServer != nil {
            // Give server time to shutdown gracefully
            ctx := context.Background()
            if err := sharedServer.Stop(); err != nil {
                log.Error().Err(err).Msg("Failed to stop test server")
            }
            
            // Verify server is actually stopped
            for i := 0; i < 5; i++ {
                _, err := http.Get("http://localhost:9191/api/health")
                if err != nil {
                    break  // Server stopped
                }
                time.Sleep(100 * time.Millisecond)
            }
        }
    })
}
```

## Best Practices Summary

### ✅ DO

1. **Use fixed port** (9191) for consistency
2. **Verify server readiness** before running tests
3. **Use real server code** for realistic testing
4. **Implement graceful shutdown** with timeouts
5. **Reuse HTTP connections** for better performance
6. **Clean up resources** in AfterSuite hooks
7. **Bind to localhost** for security
8. **Add debug logging** for troubleshooting

### ❌ DON'T

1. **Don't use external processes** (complex management)
2. **Don't mock server responses** (defeats black box testing)
3. **Don't share state between scenarios** (use fresh clients)
4. **Don't ignore shutdown errors** (resource leaks)
5. **Don't use dynamic ports** (harder to debug)
6. **Don't expose test server externally** (security risk)
7. **Don't forget to clean up** (port conflicts)

## Troubleshooting Checklist

1. **Server not starting?**
   - [ ] Check port 9191 is available
   - [ ] Verify BeforeSuite hook runs
   - [ ] Check server logs for errors
   - [ ] Test readiness endpoint manually

2. **Tests timing out?**
   - [ ] Increase waitForServerReady attempts
   - [ ] Check server startup logs
   - [ ] Verify database connections (if any)
   - [ ] Test with simpler scenarios first

3. **Connection refused?**
   - [ ] Verify server is running (`curl localhost:9191`)
   - [ ] Check for port conflicts
   - [ ] Restart test suite
   - [ ] Kill any zombie processes

4. **Wrong responses?**
   - [ ] Verify test configuration
   - [ ] Check real server implementation
   - [ ] Test endpoints manually
   - [ ] Compare with production behavior

## Performance Benchmarks

### Our Implementation Results

| Metric | Value |
|--------|-------|
| Server startup time | ~100-200ms |
| Test execution time | ~50-100ms per scenario |
| Memory usage | ~50-100MB |
| Concurrent scenarios | 4-8 parallel |
| Total test suite | ~1-2 seconds |

### Optimization Opportunities

1. **Connection pooling**: Reuse HTTP connections
2. **Parallel execution**: Run scenarios concurrently
3. **Lazy initialization**: Start server only when needed
4. **Caching**: Cache configuration and setup
5. **Minimal logging**: Reduce log overhead in tests

## Integration with Existing Code

### Using Real Server Components

```go
// pkg/bdd/testserver/server.go
func (s *Server) Start() error {
    // Use REAL server from pkg/server
    cfg := createTestConfig(s.port)
    realServer := server.NewServer(cfg, context.Background())
    
    // Use real router with all real handlers
    s.httpServer.Handler = realServer.Router()
    
    return s.waitForServerReady()
}
```

### Benefits of Real Server Integration

1. **Realistic testing**: Tests actual server behavior
2. **No mocking needed**: Uses real handlers and middleware
3. **Catches real bugs**: Finds issues that would occur in production
4. **Easy maintenance**: Changes to server automatically reflected in tests
5. **Consistent behavior**: Tests match production exactly

## Future Enhancements

### Potential Improvements

1. **Automatic port detection**: Find free port if 9191 is taken
2. **Health monitoring**: Continuous server health checks
3. **Performance metrics**: Track test execution times
4. **Test coverage**: Integration with coverage tools
5. **Docker support**: Run tests in containers
6. **Configuration options**: Make port, timeouts configurable

### Not Recommended

1. **Dynamic port allocation**: Makes debugging harder
2. **External process management**: Too complex and unreliable
3. **Mock servers**: Defeats black box testing purpose
4. **Global state sharing**: Causes test interference

## Conclusion

The hybrid in-process test server pattern provides the perfect balance of:
- **Reliability**: No external process management issues
- **Realism**: Uses actual server code and behavior
- **Performance**: Fast startup and execution
- **Debuggability**: Fixed port and clear architecture
- **Maintainability**: Simple implementation and integration

This approach has proven successful in our BDD implementation and is recommended for all API testing scenarios.