wjones
8914f4dce9
- Created `TV_SHOWS_SQL_QUERY_PATTERNS.md` to document SQL query patterns for TV shows, including performance issues and missing indexes. - Added `README.md` for Linux package building, detailing steps for creating Debian and Red Hat packages. - Implemented build scripts for Debian and Red Hat, including service files and post-installation hooks. - Added necessary scripts for managing Jellyfin service lifecycle on both Debian and Red Hat systems. - Included package specifications and installation instructions for both distributions.
586 lines
21 KiB
Markdown
586 lines
21 KiB
Markdown
# TV Shows/Series Query Performance Analysis
|
||
|
||
## Executive Summary
|
||
|
||
TV Shows queries are **inherently slower** than other item types due to:
|
||
1. **Extra table joins** - TvExtras table required for Series grouping/deduplication
|
||
2. **Complex filtering logic** - Series-specific handling in IsPlayed filter and tag inheritance
|
||
3. **In-memory deduplication** - All matching episodes must be loaded before grouping (N+1 risk)
|
||
4. **Missing composite indexes** - No index on `(Type, IsFolder)` for efficient Series filtering
|
||
5. **SeriesPresentationUniqueKey deduplication** - Requires loading all results before grouping
|
||
|
||
---
|
||
|
||
## 1. TV Shows Query Logic in BaseItemRepository
|
||
|
||
### Location
|
||
[BaseItemRepository.cs](Jellyfin.Server.Implementations/Item/BaseItemRepository.cs#L421)
|
||
|
||
### Query Flow for TV Shows
|
||
|
||
**Step 1: PrepareItemQuery** (Line 911)
|
||
```csharp
|
||
private IQueryable<BaseItemEntity> PrepareItemQuery(JellyfinDbContext context, InternalItemsQuery filter)
|
||
{
|
||
IQueryable<BaseItemEntity> dbQuery = context.BaseItems.AsNoTracking();
|
||
dbQuery = dbQuery.AsSingleQuery();
|
||
return dbQuery;
|
||
}
|
||
```
|
||
- Creates a base query from BaseItems table
|
||
- `AsSingleQuery()` - Prevents EF Core query splitting (important for TV Shows to avoid N+1)
|
||
|
||
**Step 2: TranslateQuery** (Line 2577)
|
||
- Applies filtering including:
|
||
- Type filtering: `e.Type != "Series"` (for excluding Series)
|
||
- IsSeries flag: `e.IsSeries == filter.IsSeries`
|
||
- IsPlayed special logic (line 2995-3000)
|
||
|
||
**Step 3: ApplyOrder** - Orders results
|
||
|
||
**Step 4: ApplyGroupingInMemory** (Line 814)
|
||
- **CRITICAL**: Groups results IN MEMORY (not in database)
|
||
- Uses `DistinctBy()` with reflection on anonymous types
|
||
- For Series: `DistinctBy(e => e.TvExtras!.SeriesPresentationUniqueKey)`
|
||
|
||
**Step 5: ApplyNavigations** (Line 780)
|
||
- Eagerly loads related data via `.Include()`:
|
||
```csharp
|
||
.Include(e => e.TvExtras) // TV Series metadata
|
||
.Include(e => e.LiveTvExtras) // Live TV metadata
|
||
.Include(e => e.AudioExtras) // Audio metadata
|
||
.Include(e => e.Provider) // Provider IDs
|
||
.Include(e => e.LockedFields) // Locked metadata fields
|
||
.Include(e => e.UserData) // Watch history (if EnableUserData=true)
|
||
.Include(e => e.Images) // Item images
|
||
.Include(e => e.TrailerTypes) // Trailer types (if requested)
|
||
```
|
||
|
||
---
|
||
|
||
## 2. Type Filtering for Series
|
||
|
||
### Location: Lines 2719-2748
|
||
|
||
**How Type Filtering Works:**
|
||
|
||
```csharp
|
||
var includeTypes = filter.IncludeItemTypes;
|
||
|
||
if (filter.IncludeItemTypes.Length == 0)
|
||
{
|
||
var excludeTypes = filter.ExcludeItemTypes;
|
||
if (excludeTypes.Length == 1)
|
||
{
|
||
baseQuery = baseQuery.Where(e => e.Type != excludeTypeName);
|
||
}
|
||
else if (excludeTypes.Length > 1)
|
||
{
|
||
var excludeTypeName = new List<string>();
|
||
foreach (var excludeType in excludeTypes)
|
||
{
|
||
if (_itemTypeLookup.BaseItemKindNames.TryGetValue(excludeType, out var baseItemKindName))
|
||
{
|
||
excludeTypeName.Add(baseItemKindName!);
|
||
}
|
||
}
|
||
baseQuery = baseQuery.Where(e => !excludeTypeName.Contains(e.Type));
|
||
}
|
||
}
|
||
else
|
||
{
|
||
string[] types = includeTypes.Select(f => _itemTypeLookup.BaseItemKindNames.GetValueOrDefault(f))
|
||
.Where(e => e != null).ToArray()!;
|
||
baseQuery = baseQuery.WhereOneOrMany(types, f => f.Type);
|
||
}
|
||
```
|
||
|
||
**Problem**: No index on `Type` column alone
|
||
- When filtering for Series: `e.Type == "Series"`
|
||
- Requires table scan or composite index
|
||
- **Index Status**: Only composite indexes exist:
|
||
- `baseitems_parentid_idx` → `(ParentId, Type)`
|
||
- `baseitems_topparentid_idx` → `(TopParentId, Type)`
|
||
- **Missing**: Standalone `Type` index or `(IsFolder, Type)` composite
|
||
|
||
### Workaround Optimization
|
||
The code uses `IsFolder` to optimize Series queries since Series are folders:
|
||
```csharp
|
||
if (filter.IsFolder.HasValue)
|
||
{
|
||
baseQuery = baseQuery.Where(e => e.IsFolder == filter.IsFolder);
|
||
}
|
||
```
|
||
**But**: This is manual and requires caller to specify `IsFolder=true` when filtering for Series.
|
||
|
||
---
|
||
|
||
## 3. Series-Specific Filtering: Child Count & Parent Relationships
|
||
|
||
### Location: Lines 3416-3438
|
||
|
||
**SeriesPresentationUniqueKey Filtering** (Line 3416-3419)
|
||
```csharp
|
||
if (!string.IsNullOrWhiteSpace(filter.SeriesPresentationUniqueKey))
|
||
{
|
||
baseQuery = baseQuery
|
||
.Where(e => e.TvExtras!.SeriesPresentationUniqueKey == filter.SeriesPresentationUniqueKey);
|
||
}
|
||
```
|
||
- Filters episodes by their parent Series
|
||
- Requires **join to TvExtras table**
|
||
|
||
**Tag Inheritance for Episodes** (Line 3424-3438)
|
||
```csharp
|
||
if (filter.ExcludeInheritedTags.Length > 0)
|
||
{
|
||
var excludedTags = filter.ExcludeInheritedTags;
|
||
baseQuery = baseQuery.Where(e =>
|
||
!e.ItemValues!.Any(f => f.ItemValue.Type == ItemValueType.Tags && excludedTags.Contains(f.ItemValue.CleanValue))
|
||
&& (!e.TvExtras!.SeriesId.HasValue || !context.ItemValuesMap.Any(f =>
|
||
f.ItemId == e.TvExtras!.SeriesId.Value &&
|
||
f.ItemValue.Type == ItemValueType.Tags &&
|
||
excludedTags.Contains(f.ItemValue.CleanValue))));
|
||
}
|
||
|
||
if (filter.IncludeInheritedTags.Length > 0)
|
||
{
|
||
var includeTags = filter.IncludeInheritedTags;
|
||
var isPlaylistOnlyQuery = includeTypes.Length == 1 && includeTypes.FirstOrDefault() == BaseItemKind.Playlist;
|
||
baseQuery = baseQuery.Where(e =>
|
||
e.ItemValues!.Any(f => f.ItemValue.Type == ItemValueType.Tags && includeTags.Contains(f.ItemValue.CleanValue))
|
||
|
||
// For seasons and episodes, we also need to check the parent series' tags.
|
||
|| (e.TvExtras!.SeriesId.HasValue && context.ItemValuesMap.Any(f =>
|
||
f.ItemId == e.TvExtras!.SeriesId.Value &&
|
||
f.ItemValue.Type == ItemValueType.Tags &&
|
||
includeTags.Contains(f.ItemValue.CleanValue)))
|
||
|
||
|| (isPlaylistOnlyQuery && e.Data!.Contains($"OwnerUserId\":\"{filter.User!.Id:N}\"")));
|
||
}
|
||
```
|
||
|
||
**Performance Issue**:
|
||
- **For each episode**, the query checks the parent Series' tags
|
||
- Requires correlated subqueries in tag filtering
|
||
- Forces joins to TvExtras, ItemValuesMap, and ItemValues tables
|
||
|
||
### IsPlayed Special Handling (Line 2995-3000)
|
||
```csharp
|
||
if (filter.IsPlayed.HasValue)
|
||
{
|
||
if (filter.IncludeItemTypes.Length == 1 && filter.IncludeItemTypes[0] == BaseItemKind.Series)
|
||
{
|
||
// SPECIAL CASE: For Series queries, check if ANY episode was played
|
||
baseQuery = baseQuery.Where(e =>
|
||
context.BaseItems.Where(e => e.Id != EF.Constant(PlaceholderId))
|
||
.Where(e => e.IsFolder == false && e.IsVirtualItem == false)
|
||
.Where(f => f.UserData!.FirstOrDefault(e => e.UserId == filter.User!.Id && e.Played)!.Played)
|
||
.Any(f => f.TvExtras!.SeriesPresentationUniqueKey == e.PresentationUniqueKey) == filter.IsPlayed);
|
||
}
|
||
else
|
||
{
|
||
// Standard: Check if THIS item was played
|
||
baseQuery = baseQuery
|
||
.Select(e => new
|
||
{
|
||
IsPlayed = e.UserData!.Where(f => f.UserId == filter.User!.Id).Select(f => (bool?)f.Played).FirstOrDefault() ?? false,
|
||
Item = e
|
||
})
|
||
.Where(e => e.IsPlayed == filter.IsPlayed)
|
||
.Select(f => f.Item);
|
||
}
|
||
}
|
||
```
|
||
|
||
**The Problem**:
|
||
- **Correlated subquery inside WHERE clause**
|
||
- For each Series, database checks ALL episodes to see if ANY were played
|
||
- This is a **massive performance killer** for Series queries
|
||
- Example: 100 Series × (each checks all episodes in library) = thousands of subqueries
|
||
|
||
---
|
||
|
||
## 4. N+1 Loading Patterns Specific to TV Shows
|
||
|
||
### Location: Lines 450-530 (GetItemsAsync)
|
||
|
||
**The Two-Query Pattern:**
|
||
|
||
**Query 1: Get IDs with Keys** (Line 455-460)
|
||
```csharp
|
||
var allIds = await dbQuery
|
||
.Select(e => new { e.Id, e.PresentationUniqueKey, SeriesPresentationUniqueKey = e.TvExtras!.SeriesPresentationUniqueKey })
|
||
.ToListAsync(cancellationToken)
|
||
.ConfigureAwait(false);
|
||
```
|
||
- Gets **all matching rows** (including duplicates)
|
||
- Must load TvExtras for grouping
|
||
|
||
**Query 2: Load Full Entities** (Line 475-481)
|
||
```csharp
|
||
var dbQueryWithNavs = ApplyNavigations(context.BaseItems.Where(e => pagedIds.Contains(e.Id)), filter);
|
||
var items = await dbQueryWithNavs
|
||
.ToListAsync(cancellationToken)
|
||
.ConfigureAwait(false);
|
||
```
|
||
- **Second query**: Loads full entities including all navigations
|
||
|
||
**The N+1 Risk for TV Shows:**
|
||
1. User has 500 Series with 10,000 Episodes total
|
||
2. When browsing "TV Shows" page:
|
||
- User clicks "TV Shows" → wants to see list of Series (not episodes)
|
||
- Query 1 loads ALL 10,000 episodes (to deduplicate by SeriesPresentationUniqueKey)
|
||
- Only then groups to 500 Series
|
||
- Query 2 loads 500 Series entities with all their data
|
||
3. If subsequent calls ask for "child count" or "episode count" per Series → additional queries per Series
|
||
|
||
### Why This is Different Than Movies/Music
|
||
- **Movies**: No grouping needed, 1:1 with items displayed
|
||
- **Music Albums**: Grouped by Album, but typically fewer duplicates
|
||
- **TV Shows**: 1 Series + N Episodes = N+1 rows to load and deduplicate
|
||
|
||
---
|
||
|
||
## 5. In-Memory Deduplication (ApplyGroupingInMemory)
|
||
|
||
### Location: Lines 814-850
|
||
|
||
```csharp
|
||
private List<Guid> ApplyGroupingInMemory<T>(List<T> items, InternalItemsQuery filter)
|
||
where T : class
|
||
{
|
||
var enableGroupByPresentationUniqueKey = EnableGroupByPresentationUniqueKey(filter);
|
||
|
||
var idProp = typeof(T).GetProperty("Id");
|
||
var presentationKeyProp = typeof(T).GetProperty("PresentationUniqueKey");
|
||
var seriesKeyProp = typeof(T).GetProperty("SeriesPresentationUniqueKey");
|
||
|
||
IEnumerable<T> filtered = items;
|
||
|
||
if (enableGroupByPresentationUniqueKey && filter.GroupBySeriesPresentationUniqueKey)
|
||
{
|
||
filtered = items.DistinctBy(e => new
|
||
{
|
||
PresentationKey = presentationKeyProp.GetValue(e),
|
||
SeriesKey = seriesKeyProp.GetValue(e)
|
||
});
|
||
}
|
||
else if (enableGroupByPresentationUniqueKey)
|
||
{
|
||
filtered = items.DistinctBy(e => presentationKeyProp.GetValue(e));
|
||
}
|
||
else if (filter.GroupBySeriesPresentationUniqueKey)
|
||
{
|
||
filtered = items.DistinctBy(e => seriesKeyProp.GetValue(e));
|
||
}
|
||
else
|
||
{
|
||
filtered = items.DistinctBy(e => idProp.GetValue(e));
|
||
}
|
||
|
||
return filtered.Select(e => (Guid)idProp.GetValue(e)!).ToList();
|
||
}
|
||
```
|
||
|
||
**Why Not Database Grouping?**
|
||
- Comment in code: "See docs/query-optimization-complete-story.md"
|
||
- **EF Core limitation**: `DistinctBy()` on complex types cannot be translated to SQL
|
||
- **PostgreSQL issue**: UUID constraints prevent some grouping patterns
|
||
- **Workaround**: Use `DISTINCT ON` via `DistinctBy()` in EF Core (translates correctly)
|
||
- But `DistinctBy()` within complex SELECT is not always translatable
|
||
|
||
**Performance Impact for TV Shows:**
|
||
- Must load **ALL matching episodes** from database
|
||
- Then deduplicate in C# code using reflection
|
||
- For large libraries: 10,000+ rows loaded just to get 500 unique Series
|
||
- **Memory spike**: Temporary list of 10,000 items before deduplication
|
||
|
||
---
|
||
|
||
## 6. Database Indexes: What Exists vs. What's Missing
|
||
|
||
### Current Base Indexes on BaseItems
|
||
|
||
[Migration: 20260226170000_AddBasePerformanceIndexes.cs](src/Jellyfin.Database/Jellyfin.Database.Providers.Postgres/Migrations/20260226170000_AddBasePerformanceIndexes.cs)
|
||
|
||
**Indexes That Support Series Queries:**
|
||
| Index | Columns | Supports |
|
||
|-------|---------|----------|
|
||
| `baseitems_parentid_idx` | `(ParentId, Type)` | Series grouped by parent folder |
|
||
| `baseitems_topparentid_idx` | `(TopParentId, Type)` | Series filtered by library |
|
||
| `baseitems_seriespresentationuniquekey_idx` | `(SeriesPresentationUniqueKey, IndexNumber, ParentIndexNumber)` | ✅ Episodes by Series |
|
||
| `baseitems_sortname_idx` | `(SortName)` | Series sorted by name |
|
||
| `baseitems_premieredate_idx` | `(PremiereDate DESC)` | Series sorted by date |
|
||
|
||
**Critical Indexes Missing:**
|
||
| Index | Impact | Why Missing |
|
||
|-------|--------|------------|
|
||
| `Type` (standalone) | Full table scan when filtering by Type alone | Not included in migration |
|
||
| `(IsFolder, Type)` | Fast Series queries without ParentId filter | Not included in migration |
|
||
| `(Type, IsFolder)` | Fast queries: "all Series" without folder hierarchy | Not included in migration |
|
||
| `Type` with filter on `IsSeries=true` | IsSeries column not indexed | Not included in migration |
|
||
|
||
**Why This Matters:**
|
||
```sql
|
||
-- Current: Uses composite index (ParentId, Type) if available
|
||
SELECT * FROM BaseItems WHERE ParentId = $1 AND Type = 'Series';
|
||
|
||
-- Missing: NO index for this common query pattern
|
||
SELECT * FROM BaseItems WHERE Type = 'Series';
|
||
|
||
-- Missing: NO index for this optimization
|
||
SELECT * FROM BaseItems WHERE IsFolder = true AND Type = 'Series';
|
||
```
|
||
|
||
### TvExtras Table Indexes
|
||
- **Missing**: Index on `SeriesId` (needed for tag inheritance queries)
|
||
- **Missing**: Composite `(ItemId, SeriesPresentationUniqueKey)`
|
||
|
||
---
|
||
|
||
## 7. ApplyGroupingFilter Logic
|
||
|
||
### Location: Lines 740-778
|
||
|
||
```csharp
|
||
private IQueryable<BaseItemEntity> ApplyGroupingFilter(JellyfinDbContext context, IQueryable<BaseItemEntity> dbQuery, InternalItemsQuery filter)
|
||
{
|
||
var enableGroupByPresentationUniqueKey = EnableGroupByPresentationUniqueKey(filter);
|
||
|
||
if (enableGroupByPresentationUniqueKey && filter.GroupBySeriesPresentationUniqueKey)
|
||
{
|
||
// PERFORMANCE FIX: Use DistinctBy() instead of GroupBy().Select().FirstOrDefault()
|
||
dbQuery = dbQuery.DistinctBy(e => new { e.PresentationUniqueKey, SeriesPresentationUniqueKey = e.TvExtras!.SeriesPresentationUniqueKey });
|
||
}
|
||
else if (enableGroupByPresentationUniqueKey)
|
||
{
|
||
// PERFORMANCE FIX: Use DistinctBy() to generate DISTINCT ON instead of correlated subquery
|
||
dbQuery = dbQuery.DistinctBy(e => e.PresentationUniqueKey);
|
||
}
|
||
else if (filter.GroupBySeriesPresentationUniqueKey)
|
||
{
|
||
// PERFORMANCE FIX: Use DistinctBy() instead of GroupBy().Select().FirstOrDefault()
|
||
dbQuery = dbQuery.DistinctBy(e => e.TvExtras!.SeriesPresentationUniqueKey);
|
||
}
|
||
else
|
||
{
|
||
dbQuery = dbQuery.Distinct();
|
||
}
|
||
|
||
dbQuery = ApplyOrder(dbQuery, filter, context);
|
||
return dbQuery;
|
||
}
|
||
```
|
||
|
||
**Key Insight**:
|
||
- `DistinctBy()` translates to PostgreSQL's `DISTINCT ON` clause
|
||
- This is **much faster** than correlated subqueries
|
||
- But still must fetch all matching rows first
|
||
|
||
**For Series Queries:**
|
||
- `GroupBySeriesPresentationUniqueKey=true`
|
||
- Translates to: `SELECT DISTINCT ON (TvExtras.SeriesPresentationUniqueKey) ...`
|
||
- This means: "Give me one row per unique Series"
|
||
|
||
**The Problem:**
|
||
- Must ORDER BY before using DISTINCT ON (PostgreSQL requirement)
|
||
- With many Episodes, sorting 10,000 rows then deduplicating is slower than database-level grouping
|
||
|
||
---
|
||
|
||
## 8. Complete Series Query Execution Example
|
||
|
||
### Scenario: User Clicks "TV Shows" in Library
|
||
|
||
**URL**: `/Items?IncludeItemTypes=Series&ParentId=X`
|
||
|
||
**What Happens:**
|
||
|
||
```
|
||
1. GetItemsAsync() called with:
|
||
- IncludeItemTypes: [Series]
|
||
- ParentId: X (TV Shows folder)
|
||
- EnableTotalRecordCount: true
|
||
|
||
2. PrepareItemQuery()
|
||
└─ IQueryable<BaseItemEntity> dbQuery = context.BaseItems.AsNoTracking().AsSingleQuery()
|
||
|
||
3. TranslateQuery()
|
||
└─ WHERE Type = 'Series'
|
||
└─ WHERE ParentId = X
|
||
└─ (If indexed: uses baseitems_parentid_idx)
|
||
|
||
4. ApplyOrder()
|
||
└─ ORDER BY SortName (or other sort)
|
||
|
||
5. QUERY 1 (Lines 455-460):
|
||
SELECT Id, PresentationUniqueKey, TvExtras.SeriesPresentationUniqueKey
|
||
FROM BaseItems
|
||
LEFT JOIN TvExtras ON BaseItems.Id = TvExtras.ItemId
|
||
WHERE ParentId = X AND Type = 'Series'
|
||
ORDER BY SortName
|
||
└─ Result: 500 Series rows (1 row per Series, TvExtras joined)
|
||
|
||
6. ApplyGroupingInMemory()
|
||
└─ Already unique, no deduplication needed for Series-only queries
|
||
|
||
7. ApplyPagingToIds()
|
||
└─ Skip 0, Take 100 (if paginated)
|
||
|
||
8. QUERY 2 (Lines 475-481):
|
||
SELECT * FROM BaseItems
|
||
LEFT JOIN TvExtras ON BaseItems.Id = TvExtras.ItemId
|
||
LEFT JOIN Provider ON BaseItems.Id = Provider.ItemId
|
||
LEFT JOIN Images ON BaseItems.Id = Images.ItemId
|
||
LEFT JOIN UserData ON BaseItems.Id = UserData.ItemId
|
||
WHERE Id IN (500 Series IDs from paged list)
|
||
└─ Result: Full Series entities with all related data
|
||
|
||
9. Deserialize to DTOs
|
||
└─ Convert BaseItemEntity to BaseItemDto
|
||
└─ Map UserData if available
|
||
|
||
10. Return to client
|
||
```
|
||
|
||
**Total Queries: 2**
|
||
- **Query 1**: Get Series IDs (fast, uses index)
|
||
- **Query 2**: Get full entities (includes multiple JOINs)
|
||
|
||
**Bottleneck**:
|
||
- If there are 10,000 Episodes with same Parent, Query 1 returns 10,000 rows (one per episode), then deduplicates
|
||
- This is rare for direct Series queries, but happens when:
|
||
- Querying Episodes: `IncludeItemTypes=Episode&ParentId=X`
|
||
- Querying all content: `IncludeItemTypes=[Series,Episode,Movie]`
|
||
|
||
---
|
||
|
||
## 9. Performance Comparison: Series vs. Movies vs. Music
|
||
|
||
### Query Pattern Differences
|
||
|
||
**Movies** (Fast):
|
||
```csharp
|
||
// No special handling, 1:1 with display
|
||
WHERE Type = 'Movie'
|
||
// Result: 100 rows = 100 Movies shown
|
||
```
|
||
|
||
**Music Albums** (Medium):
|
||
```csharp
|
||
// Grouped by Album name
|
||
WHERE Type = 'MusicAlbum'
|
||
SELECT DISTINCT ON (Album) ...
|
||
// Result: 100 rows = 100 Albums
|
||
```
|
||
|
||
**TV Shows** (Slow):
|
||
```csharp
|
||
// Two cases:
|
||
// Case 1: Query for Series themselves
|
||
WHERE Type = 'Series'
|
||
// Result: 500 rows = 500 Series (GOOD)
|
||
|
||
// Case 2: Query for all Episodes (common in "latest" or "browse all")
|
||
WHERE Type = 'Episode'
|
||
SELECT DISTINCT ON (TvExtras.SeriesPresentationUniqueKey) ...
|
||
// Result: 10,000 rows → deduplicate to 500 → return 100
|
||
// PROBLEM: Loaded 10,000, returned 100
|
||
```
|
||
|
||
### Why Series Are Slower
|
||
|
||
| Aspect | Movies | TV Shows | Impact |
|
||
|--------|--------|----------|--------|
|
||
| Deduplication | None | Required (Series ID) | 10-100x more rows loaded |
|
||
| Table joins | 1 (BaseItems) | 3+ (BaseItems + TvExtras + Tags checks) | Multiple JOINs per query |
|
||
| Filter complexity | Simple | Complex (tag inheritance, IsPlayed special case) | More CPU, more conditions |
|
||
| Index usage | Direct on Type | Composite (ParentId, Type) usually needed | Fewer index options |
|
||
| Related data | Simple | Children (episodes), parent (series for episodes) | More eager-load navigations |
|
||
| In-memory ops | Minimal | Full list loaded for deduplication | Memory spike |
|
||
|
||
---
|
||
|
||
## 10. Special Handling That Slows TV Shows
|
||
|
||
### 1. IsPlayed Special Case (Line 2995)
|
||
```csharp
|
||
if (filter.IncludeItemTypes.Length == 1 && filter.IncludeItemTypes[0] == BaseItemKind.Series)
|
||
{
|
||
// Correlated subquery: For each Series, check if ANY episode was played
|
||
baseQuery = baseQuery.Where(e =>
|
||
context.BaseItems
|
||
.Where(f => f.TvExtras!.SeriesPresentationUniqueKey == e.PresentationUniqueKey)
|
||
.Any(f => f.UserData!.FirstOrDefault()!.Played));
|
||
}
|
||
```
|
||
**Impact**: Extra subquery per Series
|
||
|
||
### 2. Tag Inheritance for Episodes (Line 3427-3438)
|
||
```csharp
|
||
// Check episode's tags AND parent series' tags
|
||
&& (!e.TvExtras!.SeriesId.HasValue ||
|
||
!context.ItemValuesMap.Any(f =>
|
||
f.ItemId == e.TvExtras!.SeriesId.Value &&
|
||
f.ItemValue.Type == ItemValueType.Tags))
|
||
```
|
||
**Impact**: Joins to ItemValuesMap for every tag filter on episodes
|
||
|
||
### 3. AsSingleQuery() (Line 914)
|
||
```csharp
|
||
dbQuery = dbQuery.AsSingleQuery();
|
||
```
|
||
**Impact**: Prevents EF Core from splitting queries, but means all JOINs happen in one huge query
|
||
- Better: Avoids N+1
|
||
- Worse: One massive JOIN chain for all navigations
|
||
|
||
---
|
||
|
||
## 11. Recommendations for Optimization
|
||
|
||
### Immediate (No Schema Changes)
|
||
1. **Add Index on Type column**
|
||
```sql
|
||
CREATE INDEX CONCURRENTLY idx_baseitems_type
|
||
ON library."BaseItems" ("Type");
|
||
```
|
||
|
||
2. **Add Composite Index for IsFolder + Type**
|
||
```sql
|
||
CREATE INDEX CONCURRENTLY idx_baseitems_isfolder_type
|
||
ON library."BaseItems" ("IsFolder", "Type");
|
||
```
|
||
|
||
3. **Add Index on TvExtras.SeriesId**
|
||
```sql
|
||
CREATE INDEX CONCURRENTLY idx_baseitem_tv_extras_seriesid
|
||
ON library."BaseItemTvExtras" ("SeriesId");
|
||
```
|
||
|
||
### Short-term (Application Changes)
|
||
1. **Cache Series child counts** to avoid repeated queries
|
||
2. **Lazy-load TvExtras** only when needed (not always via Include)
|
||
3. **Optimize IsPlayed query** for Series - precompute in separate query
|
||
|
||
### Long-term (Architecture Changes)
|
||
1. **Denormalize Series child count** into BaseItems table
|
||
2. **Create materialized view** for Series with episode count, last played date
|
||
3. **Use read-only replica** for expensive Series queries
|
||
4. **Implement Series-specific cache layer** in BaseItemRepository
|
||
|
||
---
|
||
|
||
## Summary: Why TV Shows Are Slower
|
||
|
||
| Factor | Impact Level | Reason |
|
||
|--------|--------------|--------|
|
||
| **Extra Table Joins** | ⭐⭐⭐⭐⭐ | TvExtras, tag checks, user data |
|
||
| **In-Memory Deduplication** | ⭐⭐⭐⭐ | All episodes loaded, deduplicated in C# |
|
||
| **Missing Indexes** | ⭐⭐⭐⭐ | No Type or (IsFolder, Type) indexes |
|
||
| **Correlated Subqueries** | ⭐⭐⭐⭐ | IsPlayed, tag inheritance, child counts |
|
||
| **Complex Filtering** | ⭐⭐⭐ | Tag inheritance from parent Series |
|
||
| **AsSingleQuery()** | ⭐⭐ | One huge JOIN chain, not optimizable |
|
||
|
||
**Total Impact**: TV Shows queries can be **10-100x slower** than Movies, depending on library size and query type.
|