Files
pgsql-jellyfin/ASYNC_CONVERSION_PRIORITY.md
T
wjones 86883cd5c6 Refactor PostgreSQL provider: multi-schema & async prep
- Refactor migrations and provider to use multiple PostgreSQL schemas, each matching a legacy SQLite database (activitylog, authentication, displaypreferences, library, users).
- All tables, foreign keys, and indexes are now schema-qualified; Down migration drops tables by schema.
- Provider ensures schemas exist before migrations; entities are mapped to correct schemas in OnModelCreating.
- Add support for max-pool-size, min-pool-size, and multiplexing connection options; update logging accordingly.
- VACUUM ANALYZE now runs per schema during scheduled optimization.
- TruncateAllTablesAsync now truncates tables with schema qualification.
- README updated with schema structure, new options, and multiplexing warnings.
- CacheDecorator now calls async repository methods using .GetAwaiter().GetResult(), with documentation.
- Lays groundwork for full async/await and multiplexing support in the database layer.
2026-02-23 09:38:22 -05:00

11 KiB

Repository Async Conversion Priority List

Based on analysis of 1,189 synchronous database operations found in the codebase.

📊 Priority Matrix

Priority determined by:

  • Complexity: Lower is better (fewer operations, simpler logic)
  • Impact: Higher is better (frequently used, performance-critical)
  • Dependencies: Fewer dependencies = higher priority

Phase 1: Simple Repositories (POC Complete + Easy Wins)

1. KeyframeRepository COMPLETED - POC

  • Sync Operations: 3
  • Complexity: Very Low
  • Files Changed: 3
    • IKeyframeRepository.cs
    • KeyframeRepository.cs
    • KeyframeManager.cs + interface
    • CacheDecorator.cs (sync wrapper needed)
  • Status: CONVERTED
  • Build: PASSING
  • Notes: One method (TryExtractKeyframes) remains sync by interface design

2. MediaAttachmentRepository COMPLETED - POC

  • Sync Operations: 5
  • Complexity: Low
  • Estimated Effort: 2-3 days
  • Files to Change: ~5
  • API Impact: Minimal (internal use)
  • Dependencies: Low
  • Recommendation: Do Next

3. MediaStreamRepository COMPLETED - POC

  • Sync Operations: 5
  • Complexity: Low
  • Estimated Effort: 2-3 days
  • Files to Change: ~6
  • API Impact: Low
  • Dependencies: Low
  • Recommendation: Do in Phase 1

4. ChapterRepository COMPLETED - POC

  • Sync Operations: 6
  • Complexity: Low-Medium
  • Files Changed: 6
    • IChapterRepository.cs
    • ChapterRepository.cs
    • IChapterManager.cs
    • ChapterManager.cs
    • ChapterImagesTask.cs
    • DtoService.cs
  • Status: CONVERTED
  • Build: PASSING
  • API Impact: Medium (chapter API endpoints)
  • Dependencies: Medium

Phase 2: Medium Complexity

5. PeopleRepository COMPLETED - POC

  • Sync Operations: 15
  • Complexity: Medium
  • Estimated Effort: 1 week
  • Files to Change: ~15
  • API Impact: Medium-High (people/actors endpoints)
  • Dependencies: Medium (used by BaseItemRepository)
  • Recommendation: Do in Phase 2

🔥 Phase 3: Complex (High Impact, High Risk)

6. BaseItemRepository ⚠️

  • Sync Operations: 110
  • Complexity: Very High
  • Estimated Effort: 4-6 weeks
  • Files to Change: 50+
  • API Impact: 🔴 CRITICAL (All item endpoints)
  • Dependencies: 🔴 VERY HIGH (Core of Jellyfin)
  • Recommendation: DO LAST - Needs careful planning
  • Special Notes:
    • Central to all library operations
    • Used by nearly every API endpoint
    • Requires extensive testing
    • Consider breaking into smaller pieces

Sprint 1-2: Simple Repositories COMPLETE (3-4 weeks)

  • KeyframeRepository (DONE)
  • MediaAttachmentRepository (DONE)
  • MediaStreamRepository (DONE)
  • ChapterRepository (DONE)

Goal: ACHIEVED - Gained experience, established patterns, validated approach

Sprint 3-4: Medium Complexity NEXT (4 weeks)

  • Week 1-3: PeopleRepository conversion
  • Week 4: Integration testing, bug fixes

Goal: Test patterns with more complex scenarios

Sprint 5-10: BaseItemRepository (6 weeks)

  • Week 1-2: Planning, breaking down into modules
  • Week 3-5: Conversion in phases
  • Week 6: Testing, performance validation

Goal: Complete core conversion with minimal disruption


🎯 Conversion Order Rationale

Why This Order?

  1. KeyframeRepository First ( Done)

    • Smallest scope for POC
    • Low risk - not heavily used
    • Validates async patterns
    • Builds team confidence
  2. Media*Repository Next

    • Still simple but more commonly used
    • Tests pattern on slightly larger scope
    • Related functionality (easier to reason about)
  3. ChapterRepository After

    • Medium complexity
    • Has API endpoints (tests full stack)
    • Good transition to Phase 2
  4. PeopleRepository Mid-Game

    • More complex queries
    • Used by BaseItemRepository
    • Must be done before BaseItemRepository
  5. BaseItemRepository Last

    • Most complex
    • Highest risk
    • By now, team has experience
    • Patterns are well-established

📋 Detailed Conversion Steps

For each repository, follow this process:

1. Preparation (1 day)

  • Read current implementation
  • Identify all public methods
  • Map all consumers (where it's used)
  • List required interface changes
  • Create feature branch

2. Interface Updates (0.5 days)

  • Update interface with async methods
  • Add CancellationToken parameters
  • Update XML documentation

3. Implementation (1-3 days depending on complexity)

  • Convert repository methods to async
  • Update all database operations
  • Add proper async disposal (await using)
  • Add ConfigureAwait(false) where appropriate

4. Consumer Updates (1-2 days)

  • Update service layer
  • Update API controllers
  • Update background services

5. Testing (1-2 days)

  • Update unit tests
  • Update integration tests
  • Add cancellation tests
  • Performance testing

6. Review & Merge (0.5 days)

  • Code review
  • Fix any issues
  • Merge to main

🔍 Detailed Repository Analysis

MediaAttachmentRepository

Sync Operations Found: 5

  • ToList(): 3 instances
  • ToArray(): 1 instance
  • SaveChanges(): 1 instance

Files That Use It:

  • MediaSourceManager.cs
  • EncodingHelper.cs
  • API controllers for media info

Breaking Changes:

  • GetAttachmentStreams(Guid itemId)GetAttachmentStreamsAsync(...)
  • SaveAttachments(...)SaveAttachmentsAsync(...)

Estimated Changes: 5-7 files


MediaStreamRepository

Sync Operations Found: 5

  • ToList(): 3 instances
  • SaveChanges(): 2 instances

Files That Use It:

  • MediaSourceManager.cs
  • MediaInfoManager.cs
  • Playback state management

Breaking Changes:

  • GetMediaStreams(Guid itemId)GetMediaStreamsAsync(...)
  • SaveMediaStreams(...)SaveMediaStreamsAsync(...)

Estimated Changes: 6-8 files


ChapterRepository

Sync Operations Found: 6

  • ToList(): 4 instances
  • ExecuteDelete(): 1 instance
  • SaveChanges(): 1 instance

Files That Use It:

  • ChaptersController.cs (API)
  • ChapterManager.cs
  • MediaSourceManager.cs

Breaking Changes:

  • GetChapters(Guid itemId)GetChaptersAsync(...)
  • SaveChapters(...)SaveChaptersAsync(...)

API Endpoints Affected:

  • GET /Items/{id}/Chapters
  • POST /Items/{id}/Chapters

Estimated Changes: 8-10 files


PeopleRepository

Sync Operations Found: 15

  • ToList(): 8 instances
  • ToArray(): 4 instances
  • FirstOrDefault(): 2 instances
  • SaveChanges(): 1 instance

Files That Use It:

  • PersonController.cs (API)
  • BaseItemRepository.cs (⚠️ circular dependency)
  • Various service layers

Breaking Changes:

  • GetPeople(InternalPeopleQuery query)GetPeopleAsync(...)
  • GetPerson(string name)GetPersonAsync(...)
  • All people-related operations

API Endpoints Affected:

  • GET /Persons
  • GET /Persons/{name}
  • People aggregations

Estimated Changes: 15-20 files

⚠️ Special Consideration: Used by BaseItemRepository, so must be done before Phase 3


BaseItemRepository ⚠️

Sync Operations Found: 110

  • ToList(): 45+ instances
  • ToArray(): 35+ instances
  • FirstOrDefault(): 15+ instances
  • ExecuteDelete(): 8 instances
  • SaveChanges(): 5 instances
  • BeginTransaction(): 2 instances

Files That Use It:

  • Nearly every API controller
  • LibraryManager.cs (core)
  • All media scanning services
  • All playback services
  • Search functionality
  • Filtering/sorting

Breaking Changes:

  • 🔴 40+ public methods need conversion
  • All item CRUD operations
  • All query operations
  • All aggregations

API Endpoints Affected:

  • 🔴 100+ endpoints directly or indirectly
  • All item retrieval
  • All filtering/search
  • All library browsing

Estimated Changes: 50-100 files

⚠️ Critical Notes:

  • Core of entire application
  • Must be done in phases
  • Requires feature flags for gradual rollout
  • Extensive testing required
  • Consider performance impact
  • May need database connection pool adjustments

Suggested Sub-Phases for BaseItemRepository:

  1. Phase 3a: Query-only operations (GetItems, filters)
  2. Phase 3b: Item retrieval (RetrieveItem, GetItem)
  3. Phase 3c: Write operations (SaveItems, UpdateItems)
  4. Phase 3d: Delete operations (DeleteItem)
  5. Phase 3e: Aggregations and statistics

🧪 Testing Strategy Per Phase

Phase 1: Simple Repositories

  • Unit Tests: All repository methods
  • Integration Tests: Repository → Service layer
  • Performance Tests: Basic benchmarks

Phase 2: Medium Complexity

  • Unit Tests: All repository methods + edge cases
  • Integration Tests: Full stack (Repository → Service → API)
  • Performance Tests: Detailed benchmarks
  • Load Tests: Concurrent operations

Phase 3: BaseItemRepository

  • Unit Tests: Comprehensive coverage
  • Integration Tests: End-to-end scenarios
  • Performance Tests: Extensive benchmarking
  • Load Tests: Production-like loads
  • Stress Tests: Breaking point analysis
  • Regression Tests: Ensure no behavioral changes
  • Canary Deployment: Test with real users gradually

📊 Risk Assessment

Repository Complexity API Impact Test Coverage Overall Risk
KeyframeRepository Low Good 🟢 LOW
MediaAttachmentRepository Low Good 🟢 LOW
MediaStreamRepository Low Good 🟢 LOW
ChapterRepository Medium Good 🟡 MEDIUM
PeopleRepository Medium Medium 🟡 MEDIUM
BaseItemRepository 🔴 Critical Needs More 🔴 HIGH

Success Metrics

Per Repository:

  • All sync operations converted
  • All tests passing
  • No performance regression (<5% slower)
  • Build successful
  • Code review approved

Overall Project:

  • 100% async database operations
  • PostgreSQL multiplexing enabled
  • 20-40% reduction in connection pool usage
  • No API breaking changes (async signatures OK)
  • Documentation updated

🚀 Next Steps

  1. Phase 1 COMPLETE: All simple repositories converted (4/4)
  2. Pattern Established: Repeatable async conversion process
  3. Documentation Complete: Comprehensive guides and presentations
  4. Present Results: Show Phase 1 completion to stakeholders
  5. Begin Phase 2: Start PeopleRepository conversion (~1 week)
  6. Phase 2 Review: Evaluate progress, adjust approach

Document Version: 2.0
Last Updated: 2025-01-15
Status: Phase 1 Complete | Phase 2 Ready 🚀
Next Action: Present to Stakeholders, then Begin PeopleRepository