pgsql-jellyfin/docs/TV_SHOWS_QUERY_PERFORMANCE_ANALYSIS.md

TV Shows/Series Query Performance Analysis

Executive Summary

TV Shows queries are inherently slower than other item types due to:

1. Extra table joins - TvExtras table required for Series grouping/deduplication
2. Complex filtering logic - Series-specific handling in IsPlayed filter and tag inheritance
3. In-memory deduplication - All matching episodes must be loaded before grouping (N+1 risk)
4. Missing composite indexes - No index on (Type, IsFolder) for efficient Series filtering
5. SeriesPresentationUniqueKey deduplication - Requires loading all results before grouping

---

1. TV Shows Query Logic in BaseItemRepository

Location

BaseItemRepository.cs

Query Flow for TV Shows

Step 1: PrepareItemQuery (Line 911)

private IQueryable<BaseItemEntity> PrepareItemQuery(JellyfinDbContext context, InternalItemsQuery filter)
{
    IQueryable<BaseItemEntity> dbQuery = context.BaseItems.AsNoTracking();
    dbQuery = dbQuery.AsSingleQuery();
    return dbQuery;
}

• Creates a base query from BaseItems table
• AsSingleQuery() - Prevents EF Core query splitting (important for TV Shows to avoid N+1)

Step 2: TranslateQuery (Line 2577)

• Applies filtering including:
  • Type filtering: e.Type != "Series" (for excluding Series)
  • IsSeries flag: e.IsSeries == filter.IsSeries
  • IsPlayed special logic (line 2995-3000)

Step 3: ApplyOrder - Orders results

Step 4: ApplyGroupingInMemory (Line 814)

• CRITICAL: Groups results IN MEMORY (not in database)
• Uses DistinctBy() with reflection on anonymous types
• For Series: DistinctBy(e => e.TvExtras!.SeriesPresentationUniqueKey)

Step 5: ApplyNavigations (Line 780)

• Eagerly loads related data via .Include():

  .Include(e => e.TvExtras)          // TV Series metadata
  .Include(e => e.LiveTvExtras)      // Live TV metadata  
  .Include(e => e.AudioExtras)       // Audio metadata
  .Include(e => e.Provider)          // Provider IDs
  .Include(e => e.LockedFields)      // Locked metadata fields
  .Include(e => e.UserData)          // Watch history (if EnableUserData=true)
  .Include(e => e.Images)            // Item images
  .Include(e => e.TrailerTypes)      // Trailer types (if requested)
  

---

2. Type Filtering for Series

Location: Lines 2719-2748

How Type Filtering Works:

var includeTypes = filter.IncludeItemTypes;

if (filter.IncludeItemTypes.Length == 0)
{
    var excludeTypes = filter.ExcludeItemTypes;
    if (excludeTypes.Length == 1)
    {
        baseQuery = baseQuery.Where(e => e.Type != excludeTypeName);
    }
    else if (excludeTypes.Length > 1)
    {
        var excludeTypeName = new List<string>();
        foreach (var excludeType in excludeTypes)
        {
            if (_itemTypeLookup.BaseItemKindNames.TryGetValue(excludeType, out var baseItemKindName))
            {
                excludeTypeName.Add(baseItemKindName!);
            }
        }
        baseQuery = baseQuery.Where(e => !excludeTypeName.Contains(e.Type));
    }
}
else
{
    string[] types = includeTypes.Select(f => _itemTypeLookup.BaseItemKindNames.GetValueOrDefault(f))
        .Where(e => e != null).ToArray()!;
    baseQuery = baseQuery.WhereOneOrMany(types, f => f.Type);
}

Problem: No index on Type column alone

• When filtering for Series: e.Type == "Series"
• Requires table scan or composite index
• Index Status: Only composite indexes exist:
  • baseitems_parentid_idx → (ParentId, Type)
  • baseitems_topparentid_idx → (TopParentId, Type)
  • Missing: Standalone Type index or (IsFolder, Type) composite

Workaround Optimization

The code uses IsFolder to optimize Series queries since Series are folders:

if (filter.IsFolder.HasValue)
{
    baseQuery = baseQuery.Where(e => e.IsFolder == filter.IsFolder);
}

But: This is manual and requires caller to specify IsFolder=true when filtering for Series.

---

3. Series-Specific Filtering: Child Count & Parent Relationships

Location: Lines 3416-3438

SeriesPresentationUniqueKey Filtering (Line 3416-3419)

if (!string.IsNullOrWhiteSpace(filter.SeriesPresentationUniqueKey))
{
    baseQuery = baseQuery
        .Where(e => e.TvExtras!.SeriesPresentationUniqueKey == filter.SeriesPresentationUniqueKey);
}

• Filters episodes by their parent Series
• Requires join to TvExtras table

Tag Inheritance for Episodes (Line 3424-3438)

if (filter.ExcludeInheritedTags.Length > 0)
{
    var excludedTags = filter.ExcludeInheritedTags;
    baseQuery = baseQuery.Where(e =>
        !e.ItemValues!.Any(f => f.ItemValue.Type == ItemValueType.Tags && excludedTags.Contains(f.ItemValue.CleanValue))
        && (!e.TvExtras!.SeriesId.HasValue || !context.ItemValuesMap.Any(f => 
            f.ItemId == e.TvExtras!.SeriesId.Value && 
            f.ItemValue.Type == ItemValueType.Tags && 
            excludedTags.Contains(f.ItemValue.CleanValue))));
}

if (filter.IncludeInheritedTags.Length > 0)
{
    var includeTags = filter.IncludeInheritedTags;
    var isPlaylistOnlyQuery = includeTypes.Length == 1 && includeTypes.FirstOrDefault() == BaseItemKind.Playlist;
    baseQuery = baseQuery.Where(e =>
        e.ItemValues!.Any(f => f.ItemValue.Type == ItemValueType.Tags && includeTags.Contains(f.ItemValue.CleanValue))
        
        // For seasons and episodes, we also need to check the parent series' tags.
        || (e.TvExtras!.SeriesId.HasValue && context.ItemValuesMap.Any(f => 
            f.ItemId == e.TvExtras!.SeriesId.Value && 
            f.ItemValue.Type == ItemValueType.Tags && 
            includeTags.Contains(f.ItemValue.CleanValue)))
        
        || (isPlaylistOnlyQuery && e.Data!.Contains($"OwnerUserId\":\"{filter.User!.Id:N}\"")));
}

Performance Issue:

• For each episode, the query checks the parent Series' tags
• Requires correlated subqueries in tag filtering
• Forces joins to TvExtras, ItemValuesMap, and ItemValues tables

IsPlayed Special Handling (Line 2995-3000)

if (filter.IsPlayed.HasValue)
{
    if (filter.IncludeItemTypes.Length == 1 && filter.IncludeItemTypes[0] == BaseItemKind.Series)
    {
        // SPECIAL CASE: For Series queries, check if ANY episode was played
        baseQuery = baseQuery.Where(e => 
            context.BaseItems.Where(e => e.Id != EF.Constant(PlaceholderId))
                .Where(e => e.IsFolder == false && e.IsVirtualItem == false)
                .Where(f => f.UserData!.FirstOrDefault(e => e.UserId == filter.User!.Id && e.Played)!.Played)
                .Any(f => f.TvExtras!.SeriesPresentationUniqueKey == e.PresentationUniqueKey) == filter.IsPlayed);
    }
    else
    {
        // Standard: Check if THIS item was played
        baseQuery = baseQuery
            .Select(e => new
            {
                IsPlayed = e.UserData!.Where(f => f.UserId == filter.User!.Id).Select(f => (bool?)f.Played).FirstOrDefault() ?? false,
                Item = e
            })
            .Where(e => e.IsPlayed == filter.IsPlayed)
            .Select(f => f.Item);
    }
}

The Problem:

• Correlated subquery inside WHERE clause
• For each Series, database checks ALL episodes to see if ANY were played
• This is a massive performance killer for Series queries
• Example: 100 Series × (each checks all episodes in library) = thousands of subqueries

---

4. N+1 Loading Patterns Specific to TV Shows

Location: Lines 450-530 (GetItemsAsync)

The Two-Query Pattern:

Query 1: Get IDs with Keys (Line 455-460)

var allIds = await dbQuery
    .Select(e => new { e.Id, e.PresentationUniqueKey, SeriesPresentationUniqueKey = e.TvExtras!.SeriesPresentationUniqueKey })
    .ToListAsync(cancellationToken)
    .ConfigureAwait(false);

• Gets all matching rows (including duplicates)
• Must load TvExtras for grouping

Query 2: Load Full Entities (Line 475-481)

var dbQueryWithNavs = ApplyNavigations(context.BaseItems.Where(e => pagedIds.Contains(e.Id)), filter);
var items = await dbQueryWithNavs
    .ToListAsync(cancellationToken)
    .ConfigureAwait(false);

• Second query: Loads full entities including all navigations

The N+1 Risk for TV Shows:

1. User has 500 Series with 10,000 Episodes total
2. When browsing "TV Shows" page:
   • User clicks "TV Shows" → wants to see list of Series (not episodes)
   • Query 1 loads ALL 10,000 episodes (to deduplicate by SeriesPresentationUniqueKey)
   • Only then groups to 500 Series
   • Query 2 loads 500 Series entities with all their data
3. If subsequent calls ask for "child count" or "episode count" per Series → additional queries per Series

Why This is Different Than Movies/Music

• Movies: No grouping needed, 1:1 with items displayed
• Music Albums: Grouped by Album, but typically fewer duplicates
• TV Shows: 1 Series + N Episodes = N+1 rows to load and deduplicate

---

5. In-Memory Deduplication (ApplyGroupingInMemory)

Location: Lines 814-850

private List<Guid> ApplyGroupingInMemory<T>(List<T> items, InternalItemsQuery filter)
    where T : class
{
    var enableGroupByPresentationUniqueKey = EnableGroupByPresentationUniqueKey(filter);

    var idProp = typeof(T).GetProperty("Id");
    var presentationKeyProp = typeof(T).GetProperty("PresentationUniqueKey");
    var seriesKeyProp = typeof(T).GetProperty("SeriesPresentationUniqueKey");

    IEnumerable<T> filtered = items;

    if (enableGroupByPresentationUniqueKey && filter.GroupBySeriesPresentationUniqueKey)
    {
        filtered = items.DistinctBy(e => new
        {
            PresentationKey = presentationKeyProp.GetValue(e),
            SeriesKey = seriesKeyProp.GetValue(e)
        });
    }
    else if (enableGroupByPresentationUniqueKey)
    {
        filtered = items.DistinctBy(e => presentationKeyProp.GetValue(e));
    }
    else if (filter.GroupBySeriesPresentationUniqueKey)
    {
        filtered = items.DistinctBy(e => seriesKeyProp.GetValue(e));
    }
    else
    {
        filtered = items.DistinctBy(e => idProp.GetValue(e));
    }

    return filtered.Select(e => (Guid)idProp.GetValue(e)!).ToList();
}

Why Not Database Grouping?

• Comment in code: "See docs/query-optimization-complete-story.md"
• EF Core limitation: DistinctBy() on complex types cannot be translated to SQL
• PostgreSQL issue: UUID constraints prevent some grouping patterns
• Workaround: Use DISTINCT ON via DistinctBy() in EF Core (translates correctly)
• But DistinctBy() within complex SELECT is not always translatable

Performance Impact for TV Shows:

• Must load ALL matching episodes from database
• Then deduplicate in C# code using reflection
• For large libraries: 10,000+ rows loaded just to get 500 unique Series
• Memory spike: Temporary list of 10,000 items before deduplication

---

6. Database Indexes: What Exists vs. What's Missing

Current Base Indexes on BaseItems

Migration: 20260226170000_AddBasePerformanceIndexes.cs

Indexes That Support Series Queries:

Index	Columns	Supports
baseitems_parentid_idx	(ParentId, Type)	Series grouped by parent folder
baseitems_topparentid_idx	(TopParentId, Type)	Series filtered by library
baseitems_seriespresentationuniquekey_idx	(SeriesPresentationUniqueKey, IndexNumber, ParentIndexNumber)	✅ Episodes by Series
baseitems_sortname_idx	(SortName)	Series sorted by name
baseitems_premieredate_idx	(PremiereDate DESC)	Series sorted by date

Critical Indexes Missing:

Index	Impact	Why Missing
Type (standalone)	Full table scan when filtering by Type alone	Not included in migration
(IsFolder, Type)	Fast Series queries without ParentId filter	Not included in migration
(Type, IsFolder)	Fast queries: "all Series" without folder hierarchy	Not included in migration
Type with filter on IsSeries=true	IsSeries column not indexed	Not included in migration

Why This Matters:

-- Current: Uses composite index (ParentId, Type) if available
SELECT * FROM BaseItems WHERE ParentId = $1 AND Type = 'Series';

-- Missing: NO index for this common query pattern
SELECT * FROM BaseItems WHERE Type = 'Series';

-- Missing: NO index for this optimization
SELECT * FROM BaseItems WHERE IsFolder = true AND Type = 'Series';

TvExtras Table Indexes

• Missing: Index on SeriesId (needed for tag inheritance queries)
• Missing: Composite (ItemId, SeriesPresentationUniqueKey)

---

7. ApplyGroupingFilter Logic

Location: Lines 740-778

private IQueryable<BaseItemEntity> ApplyGroupingFilter(JellyfinDbContext context, IQueryable<BaseItemEntity> dbQuery, InternalItemsQuery filter)
{
    var enableGroupByPresentationUniqueKey = EnableGroupByPresentationUniqueKey(filter);
    
    if (enableGroupByPresentationUniqueKey && filter.GroupBySeriesPresentationUniqueKey)
    {
        // PERFORMANCE FIX: Use DistinctBy() instead of GroupBy().Select().FirstOrDefault()
        dbQuery = dbQuery.DistinctBy(e => new { e.PresentationUniqueKey, SeriesPresentationUniqueKey = e.TvExtras!.SeriesPresentationUniqueKey });
    }
    else if (enableGroupByPresentationUniqueKey)
    {
        // PERFORMANCE FIX: Use DistinctBy() to generate DISTINCT ON instead of correlated subquery
        dbQuery = dbQuery.DistinctBy(e => e.PresentationUniqueKey);
    }
    else if (filter.GroupBySeriesPresentationUniqueKey)
    {
        // PERFORMANCE FIX: Use DistinctBy() instead of GroupBy().Select().FirstOrDefault()
        dbQuery = dbQuery.DistinctBy(e => e.TvExtras!.SeriesPresentationUniqueKey);
    }
    else
    {
        dbQuery = dbQuery.Distinct();
    }

    dbQuery = ApplyOrder(dbQuery, filter, context);
    return dbQuery;
}

Key Insight:

• DistinctBy() translates to PostgreSQL's DISTINCT ON clause
• This is much faster than correlated subqueries
• But still must fetch all matching rows first

For Series Queries:

• GroupBySeriesPresentationUniqueKey=true
• Translates to: SELECT DISTINCT ON (TvExtras.SeriesPresentationUniqueKey) ...
• This means: "Give me one row per unique Series"

The Problem:

• Must ORDER BY before using DISTINCT ON (PostgreSQL requirement)
• With many Episodes, sorting 10,000 rows then deduplicating is slower than database-level grouping

---

8. Complete Series Query Execution Example

Scenario: User Clicks "TV Shows" in Library

URL: /Items?IncludeItemTypes=Series&ParentId=X

What Happens:

1. GetItemsAsync() called with:
   - IncludeItemTypes: [Series]
   - ParentId: X (TV Shows folder)
   - EnableTotalRecordCount: true
   
2. PrepareItemQuery()
   └─ IQueryable<BaseItemEntity> dbQuery = context.BaseItems.AsNoTracking().AsSingleQuery()
   
3. TranslateQuery()
   └─ WHERE Type = 'Series'
   └─ WHERE ParentId = X
   └─ (If indexed: uses baseitems_parentid_idx)
   
4. ApplyOrder()
   └─ ORDER BY SortName (or other sort)
   
5. QUERY 1 (Lines 455-460):
   SELECT Id, PresentationUniqueKey, TvExtras.SeriesPresentationUniqueKey
   FROM BaseItems
   LEFT JOIN TvExtras ON BaseItems.Id = TvExtras.ItemId
   WHERE ParentId = X AND Type = 'Series'
   ORDER BY SortName
   └─ Result: 500 Series rows (1 row per Series, TvExtras joined)
   
6. ApplyGroupingInMemory()
   └─ Already unique, no deduplication needed for Series-only queries
   
7. ApplyPagingToIds()
   └─ Skip 0, Take 100 (if paginated)
   
8. QUERY 2 (Lines 475-481):
   SELECT * FROM BaseItems
   LEFT JOIN TvExtras ON BaseItems.Id = TvExtras.ItemId
   LEFT JOIN Provider ON BaseItems.Id = Provider.ItemId
   LEFT JOIN Images ON BaseItems.Id = Images.ItemId
   LEFT JOIN UserData ON BaseItems.Id = UserData.ItemId
   WHERE Id IN (500 Series IDs from paged list)
   └─ Result: Full Series entities with all related data
   
9. Deserialize to DTOs
   └─ Convert BaseItemEntity to BaseItemDto
   └─ Map UserData if available
   
10. Return to client

Total Queries: 2

• Query 1: Get Series IDs (fast, uses index)
• Query 2: Get full entities (includes multiple JOINs)

Bottleneck:

• If there are 10,000 Episodes with same Parent, Query 1 returns 10,000 rows (one per episode), then deduplicates
• This is rare for direct Series queries, but happens when:
  • Querying Episodes: IncludeItemTypes=Episode&ParentId=X
  • Querying all content: IncludeItemTypes=[Series,Episode,Movie]

---

9. Performance Comparison: Series vs. Movies vs. Music

Query Pattern Differences

Movies (Fast):

// No special handling, 1:1 with display
WHERE Type = 'Movie'
// Result: 100 rows = 100 Movies shown

Music Albums (Medium):

// Grouped by Album name
WHERE Type = 'MusicAlbum'
SELECT DISTINCT ON (Album) ...
// Result: 100 rows = 100 Albums

TV Shows (Slow):

// Two cases:
// Case 1: Query for Series themselves
WHERE Type = 'Series'
// Result: 500 rows = 500 Series (GOOD)

// Case 2: Query for all Episodes (common in "latest" or "browse all")
WHERE Type = 'Episode'
SELECT DISTINCT ON (TvExtras.SeriesPresentationUniqueKey) ...
// Result: 10,000 rows → deduplicate to 500 → return 100
// PROBLEM: Loaded 10,000, returned 100

Why Series Are Slower

Aspect	Movies	TV Shows	Impact
Deduplication	None	Required (Series ID)	10-100x more rows loaded
Table joins	1 (BaseItems)	3+ (BaseItems + TvExtras + Tags checks)	Multiple JOINs per query
Filter complexity	Simple	Complex (tag inheritance, IsPlayed special case)	More CPU, more conditions
Index usage	Direct on Type	Composite (ParentId, Type) usually needed	Fewer index options
Related data	Simple	Children (episodes), parent (series for episodes)	More eager-load navigations
In-memory ops	Minimal	Full list loaded for deduplication	Memory spike

---

10. Special Handling That Slows TV Shows

1. IsPlayed Special Case (Line 2995)

if (filter.IncludeItemTypes.Length == 1 && filter.IncludeItemTypes[0] == BaseItemKind.Series)
{
    // Correlated subquery: For each Series, check if ANY episode was played
    baseQuery = baseQuery.Where(e => 
        context.BaseItems
            .Where(f => f.TvExtras!.SeriesPresentationUniqueKey == e.PresentationUniqueKey)
            .Any(f => f.UserData!.FirstOrDefault()!.Played));
}

Impact: Extra subquery per Series

2. Tag Inheritance for Episodes (Line 3427-3438)

// Check episode's tags AND parent series' tags
&& (!e.TvExtras!.SeriesId.HasValue || 
    !context.ItemValuesMap.Any(f => 
        f.ItemId == e.TvExtras!.SeriesId.Value && 
        f.ItemValue.Type == ItemValueType.Tags))

Impact: Joins to ItemValuesMap for every tag filter on episodes

3. AsSingleQuery() (Line 914)

dbQuery = dbQuery.AsSingleQuery();

Impact: Prevents EF Core from splitting queries, but means all JOINs happen in one huge query

• Better: Avoids N+1
• Worse: One massive JOIN chain for all navigations

---

11. Recommendations for Optimization

Immediate (No Schema Changes)

1. Add Index on Type column

   CREATE INDEX CONCURRENTLY idx_baseitems_type 
   ON library."BaseItems" ("Type");
   

2. Add Composite Index for IsFolder + Type

   CREATE INDEX CONCURRENTLY idx_baseitems_isfolder_type 
   ON library."BaseItems" ("IsFolder", "Type");
   

3. Add Index on TvExtras.SeriesId

   CREATE INDEX CONCURRENTLY idx_baseitem_tv_extras_seriesid 
   ON library."BaseItemTvExtras" ("SeriesId");
   

Short-term (Application Changes)

1. Cache Series child counts to avoid repeated queries
2. Lazy-load TvExtras only when needed (not always via Include)
3. Optimize IsPlayed query for Series - precompute in separate query

Long-term (Architecture Changes)

1. Denormalize Series child count into BaseItems table
2. Create materialized view for Series with episode count, last played date
3. Use read-only replica for expensive Series queries
4. Implement Series-specific cache layer in BaseItemRepository

---

Summary: Why TV Shows Are Slower

Factor	Impact Level	Reason
Extra Table Joins	⭐⭐⭐⭐⭐	TvExtras, tag checks, user data
In-Memory Deduplication	⭐⭐⭐⭐	All episodes loaded, deduplicated in C#
Missing Indexes	⭐⭐⭐⭐	No Type or (IsFolder, Type) indexes
Correlated Subqueries	⭐⭐⭐⭐	IsPlayed, tag inheritance, child counts
Complex Filtering	⭐⭐⭐	Tag inheritance from parent Series
AsSingleQuery()	⭐⭐	One huge JOIN chain, not optimizable

Total Impact: TV Shows queries can be 10-100x slower than Movies, depending on library size and query type.