wjones
86883cd5c6
- Refactor migrations and provider to use multiple PostgreSQL schemas, each matching a legacy SQLite database (activitylog, authentication, displaypreferences, library, users). - All tables, foreign keys, and indexes are now schema-qualified; Down migration drops tables by schema. - Provider ensures schemas exist before migrations; entities are mapped to correct schemas in OnModelCreating. - Add support for max-pool-size, min-pool-size, and multiplexing connection options; update logging accordingly. - VACUUM ANALYZE now runs per schema during scheduled optimization. - TruncateAllTablesAsync now truncates tables with schema qualification. - README updated with schema structure, new options, and multiplexing warnings. - CacheDecorator now calls async repository methods using .GetAwaiter().GetResult(), with documentation. - Lays groundwork for full async/await and multiplexing support in the database layer.
395 lines
11 KiB
Markdown
395 lines
11 KiB
Markdown
# Repository Async Conversion Priority List
|
|
|
|
Based on analysis of **1,189 synchronous database operations** found in the codebase.
|
|
|
|
## 📊 Priority Matrix
|
|
|
|
Priority determined by:
|
|
- **Complexity**: Lower is better (fewer operations, simpler logic)
|
|
- **Impact**: Higher is better (frequently used, performance-critical)
|
|
- **Dependencies**: Fewer dependencies = higher priority
|
|
|
|
## ✅ Phase 1: Simple Repositories (POC Complete + Easy Wins)
|
|
|
|
### 1. **KeyframeRepository** ✅ **COMPLETED - POC**
|
|
- **Sync Operations**: 3
|
|
- **Complexity**: ⭐ Very Low
|
|
- **Files Changed**: 3
|
|
- `IKeyframeRepository.cs`
|
|
- `KeyframeRepository.cs`
|
|
- `KeyframeManager.cs` + interface
|
|
- `CacheDecorator.cs` (sync wrapper needed)
|
|
- **Status**: ✅ **CONVERTED**
|
|
- **Build**: ✅ **PASSING**
|
|
- **Notes**: One method (`TryExtractKeyframes`) remains sync by interface design
|
|
|
|
### 2. **MediaAttachmentRepository** ✅ **COMPLETED - POC**
|
|
- **Sync Operations**: 5
|
|
- **Complexity**: ⭐⭐ Low
|
|
- **Estimated Effort**: 2-3 days
|
|
- **Files to Change**: ~5
|
|
- **API Impact**: Minimal (internal use)
|
|
- **Dependencies**: Low
|
|
- **Recommendation**: **Do Next**
|
|
|
|
### 3. **MediaStreamRepository** ✅ ✅ **COMPLETED - POC**
|
|
- **Sync Operations**: 5
|
|
- **Complexity**: ⭐⭐ Low
|
|
- **Estimated Effort**: 2-3 days
|
|
- **Files to Change**: ~6
|
|
- **API Impact**: Low
|
|
- **Dependencies**: Low
|
|
- **Recommendation**: Do in Phase 1
|
|
|
|
### 4. **ChapterRepository** ✅ **COMPLETED - POC**
|
|
- **Sync Operations**: 6
|
|
- **Complexity**: ⭐⭐ Low-Medium
|
|
- **Files Changed**: 6
|
|
- `IChapterRepository.cs`
|
|
- `ChapterRepository.cs`
|
|
- `IChapterManager.cs`
|
|
- `ChapterManager.cs`
|
|
- `ChapterImagesTask.cs`
|
|
- `DtoService.cs`
|
|
- **Status**: ✅ **CONVERTED**
|
|
- **Build**: ✅ **PASSING**
|
|
- **API Impact**: Medium (chapter API endpoints)
|
|
- **Dependencies**: Medium
|
|
|
|
## ⚡ Phase 2: Medium Complexity
|
|
|
|
### 5. **PeopleRepository** ✅ **COMPLETED - POC**
|
|
- **Sync Operations**: 15
|
|
- **Complexity**: ⭐⭐⭐ Medium
|
|
- **Estimated Effort**: 1 week
|
|
- **Files to Change**: ~15
|
|
- **API Impact**: Medium-High (people/actors endpoints)
|
|
- **Dependencies**: Medium (used by BaseItemRepository)
|
|
- **Recommendation**: Do in Phase 2
|
|
|
|
## 🔥 Phase 3: Complex (High Impact, High Risk)
|
|
|
|
### 6. **BaseItemRepository** ⚠️
|
|
- **Sync Operations**: 110
|
|
- **Complexity**: ⭐⭐⭐⭐⭐ Very High
|
|
- **Estimated Effort**: 4-6 weeks
|
|
- **Files to Change**: 50+
|
|
- **API Impact**: 🔴 **CRITICAL** (All item endpoints)
|
|
- **Dependencies**: 🔴 **VERY HIGH** (Core of Jellyfin)
|
|
- **Recommendation**: **DO LAST** - Needs careful planning
|
|
- **Special Notes**:
|
|
- Central to all library operations
|
|
- Used by nearly every API endpoint
|
|
- Requires extensive testing
|
|
- Consider breaking into smaller pieces
|
|
|
|
---
|
|
|
|
## 📅 Recommended Timeline
|
|
|
|
### **Sprint 1-2: Simple Repositories** ✅ **COMPLETE** (3-4 weeks)
|
|
- ✅ KeyframeRepository (DONE)
|
|
- ✅ MediaAttachmentRepository (DONE)
|
|
- ✅ MediaStreamRepository (DONE)
|
|
- ✅ ChapterRepository (DONE)
|
|
|
|
**Goal**: ✅ **ACHIEVED** - Gained experience, established patterns, validated approach
|
|
|
|
### **Sprint 3-4: Medium Complexity** ⏳ **NEXT** (4 weeks)
|
|
- Week 1-3: PeopleRepository conversion
|
|
- Week 4: Integration testing, bug fixes
|
|
|
|
**Goal**: Test patterns with more complex scenarios
|
|
|
|
### **Sprint 5-10: BaseItemRepository** (6 weeks)
|
|
- Week 1-2: Planning, breaking down into modules
|
|
- Week 3-5: Conversion in phases
|
|
- Week 6: Testing, performance validation
|
|
|
|
**Goal**: Complete core conversion with minimal disruption
|
|
|
|
---
|
|
|
|
## 🎯 Conversion Order Rationale
|
|
|
|
### Why This Order?
|
|
|
|
1. **KeyframeRepository First** (✅ Done)
|
|
- Smallest scope for POC
|
|
- Low risk - not heavily used
|
|
- Validates async patterns
|
|
- Builds team confidence
|
|
|
|
2. **Media*Repository Next**
|
|
- Still simple but more commonly used
|
|
- Tests pattern on slightly larger scope
|
|
- Related functionality (easier to reason about)
|
|
|
|
3. **ChapterRepository After**
|
|
- Medium complexity
|
|
- Has API endpoints (tests full stack)
|
|
- Good transition to Phase 2
|
|
|
|
4. **PeopleRepository Mid-Game**
|
|
- More complex queries
|
|
- Used by BaseItemRepository
|
|
- Must be done before BaseItemRepository
|
|
|
|
5. **BaseItemRepository Last**
|
|
- Most complex
|
|
- Highest risk
|
|
- By now, team has experience
|
|
- Patterns are well-established
|
|
|
|
---
|
|
|
|
## 📋 Detailed Conversion Steps
|
|
|
|
For each repository, follow this process:
|
|
|
|
### 1. **Preparation** (1 day)
|
|
- [ ] Read current implementation
|
|
- [ ] Identify all public methods
|
|
- [ ] Map all consumers (where it's used)
|
|
- [ ] List required interface changes
|
|
- [ ] Create feature branch
|
|
|
|
### 2. **Interface Updates** (0.5 days)
|
|
- [ ] Update interface with async methods
|
|
- [ ] Add CancellationToken parameters
|
|
- [ ] Update XML documentation
|
|
|
|
### 3. **Implementation** (1-3 days depending on complexity)
|
|
- [ ] Convert repository methods to async
|
|
- [ ] Update all database operations
|
|
- [ ] Add proper async disposal (`await using`)
|
|
- [ ] Add ConfigureAwait(false) where appropriate
|
|
|
|
### 4. **Consumer Updates** (1-2 days)
|
|
- [ ] Update service layer
|
|
- [ ] Update API controllers
|
|
- [ ] Update background services
|
|
|
|
### 5. **Testing** (1-2 days)
|
|
- [ ] Update unit tests
|
|
- [ ] Update integration tests
|
|
- [ ] Add cancellation tests
|
|
- [ ] Performance testing
|
|
|
|
### 6. **Review & Merge** (0.5 days)
|
|
- [ ] Code review
|
|
- [ ] Fix any issues
|
|
- [ ] Merge to main
|
|
|
|
---
|
|
|
|
## 🔍 Detailed Repository Analysis
|
|
|
|
### **MediaAttachmentRepository**
|
|
**Sync Operations Found**: 5
|
|
- `ToList()`: 3 instances
|
|
- `ToArray()`: 1 instance
|
|
- `SaveChanges()`: 1 instance
|
|
|
|
**Files That Use It**:
|
|
- `MediaSourceManager.cs`
|
|
- `EncodingHelper.cs`
|
|
- API controllers for media info
|
|
|
|
**Breaking Changes**:
|
|
- `GetAttachmentStreams(Guid itemId)` → `GetAttachmentStreamsAsync(...)`
|
|
- `SaveAttachments(...)` → `SaveAttachmentsAsync(...)`
|
|
|
|
**Estimated Changes**: 5-7 files
|
|
|
|
---
|
|
|
|
### **MediaStreamRepository**
|
|
**Sync Operations Found**: 5
|
|
- `ToList()`: 3 instances
|
|
- `SaveChanges()`: 2 instances
|
|
|
|
**Files That Use It**:
|
|
- `MediaSourceManager.cs`
|
|
- `MediaInfoManager.cs`
|
|
- Playback state management
|
|
|
|
**Breaking Changes**:
|
|
- `GetMediaStreams(Guid itemId)` → `GetMediaStreamsAsync(...)`
|
|
- `SaveMediaStreams(...)` → `SaveMediaStreamsAsync(...)`
|
|
|
|
**Estimated Changes**: 6-8 files
|
|
|
|
---
|
|
|
|
### **ChapterRepository**
|
|
**Sync Operations Found**: 6
|
|
- `ToList()`: 4 instances
|
|
- `ExecuteDelete()`: 1 instance
|
|
- `SaveChanges()`: 1 instance
|
|
|
|
**Files That Use It**:
|
|
- `ChaptersController.cs` (API)
|
|
- `ChapterManager.cs`
|
|
- `MediaSourceManager.cs`
|
|
|
|
**Breaking Changes**:
|
|
- `GetChapters(Guid itemId)` → `GetChaptersAsync(...)`
|
|
- `SaveChapters(...)` → `SaveChaptersAsync(...)`
|
|
|
|
**API Endpoints Affected**:
|
|
- `GET /Items/{id}/Chapters`
|
|
- `POST /Items/{id}/Chapters`
|
|
|
|
**Estimated Changes**: 8-10 files
|
|
|
|
---
|
|
|
|
### **PeopleRepository**
|
|
**Sync Operations Found**: 15
|
|
- `ToList()`: 8 instances
|
|
- `ToArray()`: 4 instances
|
|
- `FirstOrDefault()`: 2 instances
|
|
- `SaveChanges()`: 1 instance
|
|
|
|
**Files That Use It**:
|
|
- `PersonController.cs` (API)
|
|
- `BaseItemRepository.cs` (⚠️ circular dependency)
|
|
- Various service layers
|
|
|
|
**Breaking Changes**:
|
|
- `GetPeople(InternalPeopleQuery query)` → `GetPeopleAsync(...)`
|
|
- `GetPerson(string name)` → `GetPersonAsync(...)`
|
|
- All people-related operations
|
|
|
|
**API Endpoints Affected**:
|
|
- `GET /Persons`
|
|
- `GET /Persons/{name}`
|
|
- People aggregations
|
|
|
|
**Estimated Changes**: 15-20 files
|
|
|
|
**⚠️ Special Consideration**:
|
|
Used by BaseItemRepository, so must be done before Phase 3
|
|
|
|
---
|
|
|
|
### **BaseItemRepository** ⚠️
|
|
**Sync Operations Found**: 110
|
|
- `ToList()`: 45+ instances
|
|
- `ToArray()`: 35+ instances
|
|
- `FirstOrDefault()`: 15+ instances
|
|
- `ExecuteDelete()`: 8 instances
|
|
- `SaveChanges()`: 5 instances
|
|
- `BeginTransaction()`: 2 instances
|
|
|
|
**Files That Use It**:
|
|
- **Nearly every API controller**
|
|
- `LibraryManager.cs` (core)
|
|
- All media scanning services
|
|
- All playback services
|
|
- Search functionality
|
|
- Filtering/sorting
|
|
|
|
**Breaking Changes**:
|
|
- 🔴 **40+ public methods** need conversion
|
|
- All item CRUD operations
|
|
- All query operations
|
|
- All aggregations
|
|
|
|
**API Endpoints Affected**:
|
|
- 🔴 **100+ endpoints** directly or indirectly
|
|
- All item retrieval
|
|
- All filtering/search
|
|
- All library browsing
|
|
|
|
**Estimated Changes**: 50-100 files
|
|
|
|
**⚠️ Critical Notes**:
|
|
- Core of entire application
|
|
- Must be done in phases
|
|
- Requires feature flags for gradual rollout
|
|
- Extensive testing required
|
|
- Consider performance impact
|
|
- May need database connection pool adjustments
|
|
|
|
**Suggested Sub-Phases for BaseItemRepository**:
|
|
1. **Phase 3a**: Query-only operations (GetItems, filters)
|
|
2. **Phase 3b**: Item retrieval (RetrieveItem, GetItem)
|
|
3. **Phase 3c**: Write operations (SaveItems, UpdateItems)
|
|
4. **Phase 3d**: Delete operations (DeleteItem)
|
|
5. **Phase 3e**: Aggregations and statistics
|
|
|
|
---
|
|
|
|
## 🧪 Testing Strategy Per Phase
|
|
|
|
### Phase 1: Simple Repositories
|
|
- **Unit Tests**: All repository methods
|
|
- **Integration Tests**: Repository → Service layer
|
|
- **Performance Tests**: Basic benchmarks
|
|
|
|
### Phase 2: Medium Complexity
|
|
- **Unit Tests**: All repository methods + edge cases
|
|
- **Integration Tests**: Full stack (Repository → Service → API)
|
|
- **Performance Tests**: Detailed benchmarks
|
|
- **Load Tests**: Concurrent operations
|
|
|
|
### Phase 3: BaseItemRepository
|
|
- **Unit Tests**: Comprehensive coverage
|
|
- **Integration Tests**: End-to-end scenarios
|
|
- **Performance Tests**: Extensive benchmarking
|
|
- **Load Tests**: Production-like loads
|
|
- **Stress Tests**: Breaking point analysis
|
|
- **Regression Tests**: Ensure no behavioral changes
|
|
- **Canary Deployment**: Test with real users gradually
|
|
|
|
---
|
|
|
|
## 📊 Risk Assessment
|
|
|
|
| Repository | Complexity | API Impact | Test Coverage | Overall Risk |
|
|
|-----------|-----------|------------|---------------|--------------|
|
|
| KeyframeRepository | ⭐ | Low | Good | 🟢 **LOW** |
|
|
| MediaAttachmentRepository | ⭐⭐ | Low | Good | 🟢 **LOW** |
|
|
| MediaStreamRepository | ⭐⭐ | Low | Good | 🟢 **LOW** |
|
|
| ChapterRepository | ⭐⭐⭐ | Medium | Good | 🟡 **MEDIUM** |
|
|
| PeopleRepository | ⭐⭐⭐ | Medium | Medium | 🟡 **MEDIUM** |
|
|
| BaseItemRepository | ⭐⭐⭐⭐⭐ | 🔴 Critical | Needs More | 🔴 **HIGH** |
|
|
|
|
---
|
|
|
|
## ✅ Success Metrics
|
|
|
|
### Per Repository:
|
|
- [ ] All sync operations converted
|
|
- [ ] All tests passing
|
|
- [ ] No performance regression (<5% slower)
|
|
- [ ] Build successful
|
|
- [ ] Code review approved
|
|
|
|
### Overall Project:
|
|
- [ ] 100% async database operations
|
|
- [ ] PostgreSQL multiplexing enabled
|
|
- [ ] 20-40% reduction in connection pool usage
|
|
- [ ] No API breaking changes (async signatures OK)
|
|
- [ ] Documentation updated
|
|
|
|
---
|
|
|
|
## 🚀 Next Steps
|
|
|
|
1. ✅ **Phase 1 COMPLETE**: All simple repositories converted (4/4)
|
|
2. ✅ **Pattern Established**: Repeatable async conversion process
|
|
3. ✅ **Documentation Complete**: Comprehensive guides and presentations
|
|
4. **Present Results**: Show Phase 1 completion to stakeholders
|
|
5. **Begin Phase 2**: Start PeopleRepository conversion (~1 week)
|
|
6. **Phase 2 Review**: Evaluate progress, adjust approach
|
|
|
|
---
|
|
|
|
**Document Version**: 2.0
|
|
**Last Updated**: 2025-01-15
|
|
**Status**: Phase 1 Complete ✅ | Phase 2 Ready 🚀
|
|
**Next Action**: Present to Stakeholders, then Begin PeopleRepository
|