pgsql-jellyfin/PHASE3_STRATEGY.md

# 🔥 Phase 3 Strategy: BaseItemRepository Async Conversion

## ⚠️ CRITICAL: This is the Most Complex Repository

**Status**: Planning Phase  
**Complexity**: ⭐⭐⭐⭐⭐ Very High  
**Risk**: 🔴 HIGH  
**Impact**: 🔴 CRITICAL  

---

## 📊 Scope Analysis

### By the Numbers
- **Sync Operations**: 110
- **Public Methods**: 40+
- **Files to Modify**: 50-100
- **API Endpoints Affected**: 100+
- **Estimated Effort**: 4-6 weeks (full-time)

### Why This is Different

**BaseItemRepository is THE CORE of Jellyfin:**
- Used by nearly every feature
- Central to all library operations
- Used by all media scanning services
- Core of search and filtering
- Handles all item CRUD operations

---

## 🎯 Recommended Approach: Sub-Phases

**DO NOT** attempt to convert all at once. Break into 5 manageable sub-phases:

### **Sub-Phase 3a: Query Operations** (Week 1-2)
**Focus**: Read-only query methods  
**Risk**: 🟡 MEDIUM  
**Operations**: ~30

#### Methods to Convert:
- `GetItems(InternalItemsQuery query)`
- `GetItemList(InternalItemsQuery query)`
- `GetItemIdsList(InternalItemsQuery query)`
- Filter methods
- Sort methods

#### Why First:
- Most commonly used
- Read-only (safer)
- High impact on performance
- Good testing ground

---

### **Sub-Phase 3b: Item Retrieval** (Week 3-4)
**Focus**: Single item retrieval methods  
**Risk**: 🟡 MEDIUM  
**Operations**: ~25

#### Methods to Convert:
- `RetrieveItem(Guid id)`
- `GetCount(InternalItemsQuery query)`
- `GetItemById(Guid id)`
- Existence checks

#### Why Second:
- Still mostly read-only
- Less complex than writes
- Tests full retrieval path

---

### **Sub-Phase 3c: Write Operations** (Week 5)
**Focus**: Item creation and updates  
**Risk**: 🔴 HIGH  
**Operations**: ~20

#### Methods to Convert:
- `SaveItems(IEnumerable<BaseItem> items, CancellationToken cancellationToken)`
- `UpdateInheritedValues(CancellationToken cancellationToken)`
- `UpdateImages(BaseItem item)`

#### Why Third:
- More complex logic
- Higher risk of data issues
- Needs extensive testing
- Build on query experience

---

### **Sub-Phase 3d: Delete Operations** (Week 6)
**Focus**: Item deletion  
**Risk**: 🔴 HIGH  
**Operations**: ~15

#### Methods to Convert:
- `DeleteItem(Guid id, CancellationToken cancellationToken)`
- Cascade deletes
- Cleanup operations

#### Why Fourth:
- Highest risk (data loss potential)
- Complex cascading logic
- Needs PeopleRepository (done ✅)
- Thorough testing critical

---

### **Sub-Phase 3e: Aggregations & Statistics** (Week 7)
**Focus**: Aggregation methods  
**Risk**: 🟡 MEDIUM  
**Operations**: ~20

#### Methods to Convert:
- Artist aggregations
- Album aggregations
- Statistics queries
- Genre aggregations

#### Why Last:
- Less critical
- Complex queries
- Lower risk
- Good finale

---

## 📋 Per Sub-Phase Checklist

### Preparation (1 day before starting sub-phase)
- [ ] Identify all methods in sub-phase
- [ ] Map all consumers of those methods
- [ ] Create detailed conversion plan
- [ ] Set up feature flag (if needed)
- [ ] Create test branch

### Implementation (3-5 days)
- [ ] Update interface with async methods
- [ ] Convert repository implementation
- [ ] Keep sync wrappers
- [ ] Add cancellation token support
- [ ] Proper error handling

### Consumer Updates (2-3 days)
- [ ] Update service layer
- [ ] Update API controllers
- [ ] Update background services
- [ ] Update scheduled tasks

### Testing (2-3 days)
- [ ] Unit tests
- [ ] Integration tests
- [ ] Performance testing
- [ ] Load testing
- [ ] Regression testing

### Review & Merge (1 day)
- [ ] Code review
- [ ] Address feedback
- [ ] Merge to main
- [ ] Monitor production (if deployed)

---

## 🔍 Detailed Method Analysis

### Sub-Phase 3a Example: GetItems()

**Current Signature**:
```csharp
IReadOnlyList<BaseItem> GetItems(InternalItemsQuery query)
```

**Target Signature**:
```csharp
// Sync wrapper (backward compat)
IReadOnlyList<BaseItem> GetItems(InternalItemsQuery query)
{
    return GetItemsAsync(query, CancellationToken.None)
        .GetAwaiter()
        .GetResult();
}

// New async version
async Task<IReadOnlyList<BaseItem>> GetItemsAsync(
    InternalItemsQuery query, 
    CancellationToken cancellationToken = default)
{
    await using var context = _dbProvider.CreateDbContext();
    var dbQuery = TranslateQuery(context.Items.AsNoTracking(), query);
    
    var results = await dbQuery
        .ToListAsync(cancellationToken)
        .ConfigureAwait(false);
        
    return results.Select(Map).ToList();
}
```

**Consumers to Update** (examples):
- `LibraryManager.GetItems()`
- `BaseItem.GetChildren()`
- API Controllers (`ItemsController`, `FilterController`, etc.)
- Scheduled tasks
- Background services

**Estimated Impact**:
- 20-30 consumers
- 10-15 files to modify

---

## ⚠️ Critical Considerations

### 1. Performance Impact

**Concern**: BaseItemRepository is performance-critical  
**Mitigation**:
- Extensive benchmarking before and after
- Monitor query performance
- Watch for N+1 queries
- Consider caching strategy

### 2. Data Consistency

**Concern**: Complex transaction logic  
**Mitigation**:
- Transaction scope carefully managed
- Rollback on errors
- Comprehensive error handling
- Extensive testing

### 3. Breaking Changes

**Concern**: Used everywhere  
**Mitigation**:
- Keep ALL sync methods
- Gradual consumer migration
- Feature flags for rollback
- Canary deployment strategy

### 4. Testing Coverage

**Concern**: Not all scenarios covered  
**Mitigation**:
- Add missing tests BEFORE conversion
- Integration tests for all methods
- Load testing under realistic conditions
- Beta testing with select users

---

## 🧪 Testing Strategy

### Per Sub-Phase Testing

**Unit Tests**:
- All repository methods
- Edge cases
- Error conditions
- Cancellation scenarios

**Integration Tests**:
- Repository → Service → API
- Full request/response cycle
- Multiple concurrent requests
- Transaction rollback scenarios

**Performance Tests**:
- Benchmark each method
- Compare sync vs async performance
- Load testing (100+ concurrent users)
- Monitor connection pool usage

**Regression Tests**:
- Ensure no behavior changes
- Verify all existing features work
- Check edge cases still handled

---

## 📊 Risk Mitigation

### High-Risk Areas

1. **GetItems() - Most Used Method**
   - Test extensively
   - Monitor performance
   - Gradual rollout

2. **SaveItems() - Data Modification**
   - Extra validation
   - Transaction logging
   - Rollback testing

3. **DeleteItem() - Data Loss Risk**
   - Backup strategy
   - Soft delete consideration
   - Audit logging

### Rollback Plan

**If issues detected**:
1. Feature flag to disable async
2. Revert to sync wrappers
3. Investigate issues
4. Fix and re-deploy

**Monitoring**:
- Error rates
- Response times
- Connection pool usage
- Database performance

---

## 🎯 Success Criteria Per Sub-Phase

### Technical
- [ ] All methods in sub-phase converted
- [ ] Build successful
- [ ] All tests passing
- [ ] Performance within 5% of baseline
- [ ] No memory leaks

### Functional
- [ ] All features working
- [ ] No user-reported issues
- [ ] API responses correct
- [ ] Error handling proper

### Non-Functional
- [ ] Documentation updated
- [ ] Code reviewed
- [ ] Monitoring in place
- [ ] Rollback tested

---

## 📅 Detailed Timeline

### Week 0: Preparation (Before Starting)
- [ ] Complete Phase 1 & 2 presentation
- [ ] Get stakeholder approval
- [ ] Allocate dedicated resources
- [ ] Set up monitoring
- [ ] Create performance baselines

### Week 1-2: Sub-Phase 3a (Query Operations)
- Day 1-2: Interface & implementation
- Day 3-5: Consumer updates
- Day 6-8: Testing
- Day 9-10: Review & merge

### Week 3-4: Sub-Phase 3b (Item Retrieval)
- Day 1-2: Interface & implementation
- Day 3-5: Consumer updates
- Day 6-8: Testing
- Day 9-10: Review & merge

### Week 5: Sub-Phase 3c (Write Operations)
- Day 1-2: Interface & implementation
- Day 3: Consumer updates
- Day 4-5: Extensive testing

### Week 6: Sub-Phase 3d (Delete Operations)
- Day 1-2: Interface & implementation
- Day 3: Consumer updates
- Day 4-5: Extensive testing

### Week 7: Sub-Phase 3e (Aggregations)
- Day 1-2: Interface & implementation
- Day 3: Consumer updates
- Day 4-5: Testing

### Week 8: Final Integration & Deployment
- Full regression testing
- Performance validation
- Production deployment
- Monitoring

---

## 🔧 Code Example: Complex Method

### SaveItems() - Before/After

**BEFORE (Synchronous)**:
```csharp
public void SaveItems(IEnumerable<BaseItem> items, CancellationToken cancellationToken)
{
    using var context = _dbProvider.CreateDbContext();
    using var transaction = context.Database.BeginTransaction();
    
    foreach (var item in items)
    {
        // Validate item
        ValidateItem(item);
        
        // Save to database
        var entity = Map(item);
        context.Items.Add(entity);
        
        // Update people
        _peopleRepository.UpdatePeople(item.Id, item.People);
        
        // Update images
        UpdateImages(context, item);
    }
    
    context.SaveChanges();
    transaction.Commit();
}
```

**AFTER (Asynchronous)**:
```csharp
// Sync wrapper for backward compatibility
public void SaveItems(IEnumerable<BaseItem> items, CancellationToken cancellationToken)
{
    SaveItemsAsync(items, cancellationToken)
        .GetAwaiter()
        .GetResult();
}

// Async implementation
public async Task SaveItemsAsync(
    IEnumerable<BaseItem> items, 
    CancellationToken cancellationToken = default)
{
    await using var context = _dbProvider.CreateDbContext();
    await using var transaction = await context.Database
        .BeginTransactionAsync(cancellationToken)
        .ConfigureAwait(false);
    
    foreach (var item in items)
    {
        cancellationToken.ThrowIfCancellationRequested();
        
        // Validate item
        ValidateItem(item);
        
        // Save to database
        var entity = Map(item);
        await context.Items.AddAsync(entity, cancellationToken)
            .ConfigureAwait(false);
        
        // Update people (using async PeopleRepository)
        await _peopleRepository
            .UpdatePeopleAsync(item.Id, item.People, cancellationToken)
            .ConfigureAwait(false);
        
        // Update images
        await UpdateImagesAsync(context, item, cancellationToken)
            .ConfigureAwait(false);
    }
    
    await context.SaveChangesAsync(cancellationToken)
        .ConfigureAwait(false);
    await transaction.CommitAsync(cancellationToken)
        .ConfigureAwait(false);
}
```

---

## 💡 Recommendations

### DO ✅
1. **Break into sub-phases** - Don't do all at once
2. **Test extensively** - This is critical code
3. **Monitor performance** - Watch for regressions
4. **Keep sync wrappers** - Maintain backward compatibility
5. **Use feature flags** - Enable gradual rollout
6. **Document everything** - Future maintainers will thank you
7. **Get code reviews** - Multiple eyes on critical code

### DON'T ❌
1. **Rush it** - Take the time needed
2. **Skip tests** - You'll regret it
3. **Remove sync methods** - You'll break everything
4. **Deploy without monitoring** - You need visibility
5. **Ignore performance** - This is hot path code
6. **Work alone** - Get help and reviews

---

## 🎯 Recommended Next Steps

### Step 1: Present Phase 1 & 2 Results
**Audience**: Technical leadership, stakeholders  
**Materials**: Use `STAKEHOLDER_PRESENTATION.md`  
**Message**: 
- 5/6 repositories complete (83%)
- 34 operations converted
- Zero issues, all builds passing
- Ready for Phase 3 with proper planning

### Step 2: Get Approval for Phase 3
**Request**:
- Dedicated developer(s) for 8 weeks
- QA support throughout
- Stakeholder check-ins every 2 weeks
- Approval to use feature flags

### Step 3: Preparation Week
**Tasks**:
- Analyze BaseItemRepository in detail
- Create performance baselines
- Identify all 110 operations
- Map all consumers
- Set up monitoring

### Step 4: Begin Sub-Phase 3a
**Focus**: Query operations (lowest risk)  
**Timeline**: 2 weeks  
**Deliverable**: Async query methods working  

---

## 📚 Documentation Needed

For Phase 3, create:
- [ ] Detailed method inventory
- [ ] Consumer mapping document
- [ ] Performance baseline report
- [ ] Testing strategy document
- [ ] Deployment plan
- [ ] Rollback procedures
- [ ] Monitoring dashboard

---

## 🎉 Current Achievement

**You've completed 83% of the migration!**

- ✅ Phase 1: 4/4 repositories (100%)
- ✅ Phase 2: 1/1 repository (100%)
- ⏳ Phase 3: 0/1 repository (planning)

**That's 5 out of 6 repositories done!**

The foundation is solid. The pattern is proven. Phase 3 is well-planned. You're in excellent position to complete the final 17% of the work.

---

**Status**: Planning Complete  
**Recommendation**: Present results, get approval, then begin Phase 3  
**Confidence**: HIGH (with proper planning) ✅  
**Risk**: Manageable (with sub-phases) 🟡  

**Document Version**: 1.0  
**Date**: 2025-01-15  
**Next Action**: Stakeholder Presentation