diff --git a/adr/0025-bdd-scenario-isolation-strategies.md b/adr/0025-bdd-scenario-isolation-strategies.md index d1dfadb..2ca750c 100644 --- a/adr/0025-bdd-scenario-isolation-strategies.md +++ b/adr/0025-bdd-scenario-isolation-strategies.md @@ -174,7 +174,7 @@ AfterScenario: DELETE FROM all_tables; ## Decision Outcome -**Chosen option: Schema-per-Scenario (Option 3)** +**Chosen option: Schema-per-Scenario + In-Memory State Reset (Option 3 Enhanced)** We will implement schema-per-scenario because it: @@ -184,27 +184,88 @@ We will implement schema-per-scenario because it: 4. **Works with scenario transactions** - Scenarios can commit freely 5. Is **fast** - Schema operations are cheap +**However, we discovered a critical limitation:** PostgreSQL schemas only isolate **database tables**. In-memory state (application-level caches, user stores, JWT secret managers) **persists across scenarios** because they're stored in the shared `sharedServer` Go instance. Schema isolation does NOT solve this. + +### Enhanced Strategy: Multi-Layer Isolation + +To achieve **complete scenario isolation**, we need a **2-layer approach:** + +| Layer | Component | Strategy | Status | +|-------|-----------|----------|--------| +| DB | PostgreSQL tables | Schema-per-scenario | ✅ Implemented | +| Memory | User store | Reset/clear between scenarios | ⚠️ TODO | +| Memory | JWT secrets | Reset to initial state | ✅ Implemented | +| Memory | Auth cache | Reset/clear between scenarios | ⚠️ TODO | +| Cache | Redis/Memcached | Key prefix with schema hash | ⚠️ TODO | + +### Key Insight: Cache and In-Memory Store Isolation + +**For caches (Redis, Memcached, in-process):** +- Use **schema hash as key prefix/suffix**: `cache_key_{schema_hash}` or `{schema_hash}_cache_key` +- This ensures each scenario gets isolated cache namespace +- Works even with external cache services +- Consistent with schema isolation philosophy + +**For in-memory stores (user repository, etc.):** +- Add `Reset()` methods that clear all state +- Call in `AfterScenario` alongside schema teardown +- Or use schema-prefix approach for shared stores + +### Alternative Approach: Background Explicit State Setup + +**Considered but rejected:** Adding explicit "Given no user X exists" steps or heavy Background sections. + +**Pros:** More readable, explicit about state +**Cons:** +- Error-prone (must remember for every entity) +- Verbose (many Given steps) +- Doesn't scale with many entities +- Still has race conditions with concurrent scenarios + +**Verdict:** Automated cleanup (schema drop + memory reset) is more reliable than manual Background setup. + ### Implementation Plan -**Phase 1: Foundation** +**Phase 1: Foundation (✅ Complete)** - Add scenario-aware schema management to test server - Implement schema creation/drop in BeforeScenario/AfterScenario hooks - Handle `search_path` configuration for each scenario's database connection -**Phase 2: Connection Pooling** +**Phase 2: In-Memory State Reset (🟡 TODO)** +- Add `ResetUsers()` method to clear in-memory user store +- Add `ResetCache()` method for auth/rateLimiting caches +- Call these in AfterScenario alongside JWT secret reset +- **Cache key strategy**: `key_{schema_hash}` for all cache operations + +**Phase 3: Connection Pooling** - Configure connection pool to respect per-scenario `search_path` - Each scenario gets isolated connections -**Phase 3: In-Memory State** -- Extend cleanup to handle JWT secrets (already implemented in suite.go) -- Add config reset capability - **Phase 4: Validation** -- Run full test suite to identify ORM/schema issues +- Run full test suite to verify complete isolation - Fix any hardcoded `public` schema references ### Schema Naming Convention +``` +Schema name: test_{sha256(feature:scenario)[:8]} +Cache key prefix: {sha256(feature:scenario)[:8]}_ +``` + +Example: +- Feature: `auth`, Scenario: `Successful user authentication` +- Hash: `sha256("auth:Successful user authentication")[:8]` = `a3f7b2c1` +- Schema: `test_a3f7b2c1` +- Cache key: `a3f7b2c1_user:newuser` instead of just `user:newuser` + +Benefits: +- Unique per scenario +- Consistent across test runs (same scenario = same hash) +- Short (8 chars) - efficient for cache keys +- Identifiable for debugging + +### Schema Naming Convention + ``` Schema name: test_{sha256(feature + scenario)[:8]} ```