Add SQL query patterns documentation and Linux package build scripts
- Created `TV_SHOWS_SQL_QUERY_PATTERNS.md` to document SQL query patterns for TV shows, including performance issues and missing indexes. - Added `README.md` for Linux package building, detailing steps for creating Debian and Red Hat packages. - Implemented build scripts for Debian and Red Hat, including service files and post-installation hooks. - Added necessary scripts for managing Jellyfin service lifecycle on both Debian and Red Hat systems. - Included package specifications and installation instructions for both distributions.
This commit is contained in:
@@ -0,0 +1,585 @@
|
||||
# TV Shows/Series Query Performance Analysis
|
||||
|
||||
## Executive Summary
|
||||
|
||||
TV Shows queries are **inherently slower** than other item types due to:
|
||||
1. **Extra table joins** - TvExtras table required for Series grouping/deduplication
|
||||
2. **Complex filtering logic** - Series-specific handling in IsPlayed filter and tag inheritance
|
||||
3. **In-memory deduplication** - All matching episodes must be loaded before grouping (N+1 risk)
|
||||
4. **Missing composite indexes** - No index on `(Type, IsFolder)` for efficient Series filtering
|
||||
5. **SeriesPresentationUniqueKey deduplication** - Requires loading all results before grouping
|
||||
|
||||
---
|
||||
|
||||
## 1. TV Shows Query Logic in BaseItemRepository
|
||||
|
||||
### Location
|
||||
[BaseItemRepository.cs](Jellyfin.Server.Implementations/Item/BaseItemRepository.cs#L421)
|
||||
|
||||
### Query Flow for TV Shows
|
||||
|
||||
**Step 1: PrepareItemQuery** (Line 911)
|
||||
```csharp
|
||||
private IQueryable<BaseItemEntity> PrepareItemQuery(JellyfinDbContext context, InternalItemsQuery filter)
|
||||
{
|
||||
IQueryable<BaseItemEntity> dbQuery = context.BaseItems.AsNoTracking();
|
||||
dbQuery = dbQuery.AsSingleQuery();
|
||||
return dbQuery;
|
||||
}
|
||||
```
|
||||
- Creates a base query from BaseItems table
|
||||
- `AsSingleQuery()` - Prevents EF Core query splitting (important for TV Shows to avoid N+1)
|
||||
|
||||
**Step 2: TranslateQuery** (Line 2577)
|
||||
- Applies filtering including:
|
||||
- Type filtering: `e.Type != "Series"` (for excluding Series)
|
||||
- IsSeries flag: `e.IsSeries == filter.IsSeries`
|
||||
- IsPlayed special logic (line 2995-3000)
|
||||
|
||||
**Step 3: ApplyOrder** - Orders results
|
||||
|
||||
**Step 4: ApplyGroupingInMemory** (Line 814)
|
||||
- **CRITICAL**: Groups results IN MEMORY (not in database)
|
||||
- Uses `DistinctBy()` with reflection on anonymous types
|
||||
- For Series: `DistinctBy(e => e.TvExtras!.SeriesPresentationUniqueKey)`
|
||||
|
||||
**Step 5: ApplyNavigations** (Line 780)
|
||||
- Eagerly loads related data via `.Include()`:
|
||||
```csharp
|
||||
.Include(e => e.TvExtras) // TV Series metadata
|
||||
.Include(e => e.LiveTvExtras) // Live TV metadata
|
||||
.Include(e => e.AudioExtras) // Audio metadata
|
||||
.Include(e => e.Provider) // Provider IDs
|
||||
.Include(e => e.LockedFields) // Locked metadata fields
|
||||
.Include(e => e.UserData) // Watch history (if EnableUserData=true)
|
||||
.Include(e => e.Images) // Item images
|
||||
.Include(e => e.TrailerTypes) // Trailer types (if requested)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Type Filtering for Series
|
||||
|
||||
### Location: Lines 2719-2748
|
||||
|
||||
**How Type Filtering Works:**
|
||||
|
||||
```csharp
|
||||
var includeTypes = filter.IncludeItemTypes;
|
||||
|
||||
if (filter.IncludeItemTypes.Length == 0)
|
||||
{
|
||||
var excludeTypes = filter.ExcludeItemTypes;
|
||||
if (excludeTypes.Length == 1)
|
||||
{
|
||||
baseQuery = baseQuery.Where(e => e.Type != excludeTypeName);
|
||||
}
|
||||
else if (excludeTypes.Length > 1)
|
||||
{
|
||||
var excludeTypeName = new List<string>();
|
||||
foreach (var excludeType in excludeTypes)
|
||||
{
|
||||
if (_itemTypeLookup.BaseItemKindNames.TryGetValue(excludeType, out var baseItemKindName))
|
||||
{
|
||||
excludeTypeName.Add(baseItemKindName!);
|
||||
}
|
||||
}
|
||||
baseQuery = baseQuery.Where(e => !excludeTypeName.Contains(e.Type));
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
string[] types = includeTypes.Select(f => _itemTypeLookup.BaseItemKindNames.GetValueOrDefault(f))
|
||||
.Where(e => e != null).ToArray()!;
|
||||
baseQuery = baseQuery.WhereOneOrMany(types, f => f.Type);
|
||||
}
|
||||
```
|
||||
|
||||
**Problem**: No index on `Type` column alone
|
||||
- When filtering for Series: `e.Type == "Series"`
|
||||
- Requires table scan or composite index
|
||||
- **Index Status**: Only composite indexes exist:
|
||||
- `baseitems_parentid_idx` → `(ParentId, Type)`
|
||||
- `baseitems_topparentid_idx` → `(TopParentId, Type)`
|
||||
- **Missing**: Standalone `Type` index or `(IsFolder, Type)` composite
|
||||
|
||||
### Workaround Optimization
|
||||
The code uses `IsFolder` to optimize Series queries since Series are folders:
|
||||
```csharp
|
||||
if (filter.IsFolder.HasValue)
|
||||
{
|
||||
baseQuery = baseQuery.Where(e => e.IsFolder == filter.IsFolder);
|
||||
}
|
||||
```
|
||||
**But**: This is manual and requires caller to specify `IsFolder=true` when filtering for Series.
|
||||
|
||||
---
|
||||
|
||||
## 3. Series-Specific Filtering: Child Count & Parent Relationships
|
||||
|
||||
### Location: Lines 3416-3438
|
||||
|
||||
**SeriesPresentationUniqueKey Filtering** (Line 3416-3419)
|
||||
```csharp
|
||||
if (!string.IsNullOrWhiteSpace(filter.SeriesPresentationUniqueKey))
|
||||
{
|
||||
baseQuery = baseQuery
|
||||
.Where(e => e.TvExtras!.SeriesPresentationUniqueKey == filter.SeriesPresentationUniqueKey);
|
||||
}
|
||||
```
|
||||
- Filters episodes by their parent Series
|
||||
- Requires **join to TvExtras table**
|
||||
|
||||
**Tag Inheritance for Episodes** (Line 3424-3438)
|
||||
```csharp
|
||||
if (filter.ExcludeInheritedTags.Length > 0)
|
||||
{
|
||||
var excludedTags = filter.ExcludeInheritedTags;
|
||||
baseQuery = baseQuery.Where(e =>
|
||||
!e.ItemValues!.Any(f => f.ItemValue.Type == ItemValueType.Tags && excludedTags.Contains(f.ItemValue.CleanValue))
|
||||
&& (!e.TvExtras!.SeriesId.HasValue || !context.ItemValuesMap.Any(f =>
|
||||
f.ItemId == e.TvExtras!.SeriesId.Value &&
|
||||
f.ItemValue.Type == ItemValueType.Tags &&
|
||||
excludedTags.Contains(f.ItemValue.CleanValue))));
|
||||
}
|
||||
|
||||
if (filter.IncludeInheritedTags.Length > 0)
|
||||
{
|
||||
var includeTags = filter.IncludeInheritedTags;
|
||||
var isPlaylistOnlyQuery = includeTypes.Length == 1 && includeTypes.FirstOrDefault() == BaseItemKind.Playlist;
|
||||
baseQuery = baseQuery.Where(e =>
|
||||
e.ItemValues!.Any(f => f.ItemValue.Type == ItemValueType.Tags && includeTags.Contains(f.ItemValue.CleanValue))
|
||||
|
||||
// For seasons and episodes, we also need to check the parent series' tags.
|
||||
|| (e.TvExtras!.SeriesId.HasValue && context.ItemValuesMap.Any(f =>
|
||||
f.ItemId == e.TvExtras!.SeriesId.Value &&
|
||||
f.ItemValue.Type == ItemValueType.Tags &&
|
||||
includeTags.Contains(f.ItemValue.CleanValue)))
|
||||
|
||||
|| (isPlaylistOnlyQuery && e.Data!.Contains($"OwnerUserId\":\"{filter.User!.Id:N}\"")));
|
||||
}
|
||||
```
|
||||
|
||||
**Performance Issue**:
|
||||
- **For each episode**, the query checks the parent Series' tags
|
||||
- Requires correlated subqueries in tag filtering
|
||||
- Forces joins to TvExtras, ItemValuesMap, and ItemValues tables
|
||||
|
||||
### IsPlayed Special Handling (Line 2995-3000)
|
||||
```csharp
|
||||
if (filter.IsPlayed.HasValue)
|
||||
{
|
||||
if (filter.IncludeItemTypes.Length == 1 && filter.IncludeItemTypes[0] == BaseItemKind.Series)
|
||||
{
|
||||
// SPECIAL CASE: For Series queries, check if ANY episode was played
|
||||
baseQuery = baseQuery.Where(e =>
|
||||
context.BaseItems.Where(e => e.Id != EF.Constant(PlaceholderId))
|
||||
.Where(e => e.IsFolder == false && e.IsVirtualItem == false)
|
||||
.Where(f => f.UserData!.FirstOrDefault(e => e.UserId == filter.User!.Id && e.Played)!.Played)
|
||||
.Any(f => f.TvExtras!.SeriesPresentationUniqueKey == e.PresentationUniqueKey) == filter.IsPlayed);
|
||||
}
|
||||
else
|
||||
{
|
||||
// Standard: Check if THIS item was played
|
||||
baseQuery = baseQuery
|
||||
.Select(e => new
|
||||
{
|
||||
IsPlayed = e.UserData!.Where(f => f.UserId == filter.User!.Id).Select(f => (bool?)f.Played).FirstOrDefault() ?? false,
|
||||
Item = e
|
||||
})
|
||||
.Where(e => e.IsPlayed == filter.IsPlayed)
|
||||
.Select(f => f.Item);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**The Problem**:
|
||||
- **Correlated subquery inside WHERE clause**
|
||||
- For each Series, database checks ALL episodes to see if ANY were played
|
||||
- This is a **massive performance killer** for Series queries
|
||||
- Example: 100 Series × (each checks all episodes in library) = thousands of subqueries
|
||||
|
||||
---
|
||||
|
||||
## 4. N+1 Loading Patterns Specific to TV Shows
|
||||
|
||||
### Location: Lines 450-530 (GetItemsAsync)
|
||||
|
||||
**The Two-Query Pattern:**
|
||||
|
||||
**Query 1: Get IDs with Keys** (Line 455-460)
|
||||
```csharp
|
||||
var allIds = await dbQuery
|
||||
.Select(e => new { e.Id, e.PresentationUniqueKey, SeriesPresentationUniqueKey = e.TvExtras!.SeriesPresentationUniqueKey })
|
||||
.ToListAsync(cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
```
|
||||
- Gets **all matching rows** (including duplicates)
|
||||
- Must load TvExtras for grouping
|
||||
|
||||
**Query 2: Load Full Entities** (Line 475-481)
|
||||
```csharp
|
||||
var dbQueryWithNavs = ApplyNavigations(context.BaseItems.Where(e => pagedIds.Contains(e.Id)), filter);
|
||||
var items = await dbQueryWithNavs
|
||||
.ToListAsync(cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
```
|
||||
- **Second query**: Loads full entities including all navigations
|
||||
|
||||
**The N+1 Risk for TV Shows:**
|
||||
1. User has 500 Series with 10,000 Episodes total
|
||||
2. When browsing "TV Shows" page:
|
||||
- User clicks "TV Shows" → wants to see list of Series (not episodes)
|
||||
- Query 1 loads ALL 10,000 episodes (to deduplicate by SeriesPresentationUniqueKey)
|
||||
- Only then groups to 500 Series
|
||||
- Query 2 loads 500 Series entities with all their data
|
||||
3. If subsequent calls ask for "child count" or "episode count" per Series → additional queries per Series
|
||||
|
||||
### Why This is Different Than Movies/Music
|
||||
- **Movies**: No grouping needed, 1:1 with items displayed
|
||||
- **Music Albums**: Grouped by Album, but typically fewer duplicates
|
||||
- **TV Shows**: 1 Series + N Episodes = N+1 rows to load and deduplicate
|
||||
|
||||
---
|
||||
|
||||
## 5. In-Memory Deduplication (ApplyGroupingInMemory)
|
||||
|
||||
### Location: Lines 814-850
|
||||
|
||||
```csharp
|
||||
private List<Guid> ApplyGroupingInMemory<T>(List<T> items, InternalItemsQuery filter)
|
||||
where T : class
|
||||
{
|
||||
var enableGroupByPresentationUniqueKey = EnableGroupByPresentationUniqueKey(filter);
|
||||
|
||||
var idProp = typeof(T).GetProperty("Id");
|
||||
var presentationKeyProp = typeof(T).GetProperty("PresentationUniqueKey");
|
||||
var seriesKeyProp = typeof(T).GetProperty("SeriesPresentationUniqueKey");
|
||||
|
||||
IEnumerable<T> filtered = items;
|
||||
|
||||
if (enableGroupByPresentationUniqueKey && filter.GroupBySeriesPresentationUniqueKey)
|
||||
{
|
||||
filtered = items.DistinctBy(e => new
|
||||
{
|
||||
PresentationKey = presentationKeyProp.GetValue(e),
|
||||
SeriesKey = seriesKeyProp.GetValue(e)
|
||||
});
|
||||
}
|
||||
else if (enableGroupByPresentationUniqueKey)
|
||||
{
|
||||
filtered = items.DistinctBy(e => presentationKeyProp.GetValue(e));
|
||||
}
|
||||
else if (filter.GroupBySeriesPresentationUniqueKey)
|
||||
{
|
||||
filtered = items.DistinctBy(e => seriesKeyProp.GetValue(e));
|
||||
}
|
||||
else
|
||||
{
|
||||
filtered = items.DistinctBy(e => idProp.GetValue(e));
|
||||
}
|
||||
|
||||
return filtered.Select(e => (Guid)idProp.GetValue(e)!).ToList();
|
||||
}
|
||||
```
|
||||
|
||||
**Why Not Database Grouping?**
|
||||
- Comment in code: "See docs/query-optimization-complete-story.md"
|
||||
- **EF Core limitation**: `DistinctBy()` on complex types cannot be translated to SQL
|
||||
- **PostgreSQL issue**: UUID constraints prevent some grouping patterns
|
||||
- **Workaround**: Use `DISTINCT ON` via `DistinctBy()` in EF Core (translates correctly)
|
||||
- But `DistinctBy()` within complex SELECT is not always translatable
|
||||
|
||||
**Performance Impact for TV Shows:**
|
||||
- Must load **ALL matching episodes** from database
|
||||
- Then deduplicate in C# code using reflection
|
||||
- For large libraries: 10,000+ rows loaded just to get 500 unique Series
|
||||
- **Memory spike**: Temporary list of 10,000 items before deduplication
|
||||
|
||||
---
|
||||
|
||||
## 6. Database Indexes: What Exists vs. What's Missing
|
||||
|
||||
### Current Base Indexes on BaseItems
|
||||
|
||||
[Migration: 20260226170000_AddBasePerformanceIndexes.cs](src/Jellyfin.Database/Jellyfin.Database.Providers.Postgres/Migrations/20260226170000_AddBasePerformanceIndexes.cs)
|
||||
|
||||
**Indexes That Support Series Queries:**
|
||||
| Index | Columns | Supports |
|
||||
|-------|---------|----------|
|
||||
| `baseitems_parentid_idx` | `(ParentId, Type)` | Series grouped by parent folder |
|
||||
| `baseitems_topparentid_idx` | `(TopParentId, Type)` | Series filtered by library |
|
||||
| `baseitems_seriespresentationuniquekey_idx` | `(SeriesPresentationUniqueKey, IndexNumber, ParentIndexNumber)` | ✅ Episodes by Series |
|
||||
| `baseitems_sortname_idx` | `(SortName)` | Series sorted by name |
|
||||
| `baseitems_premieredate_idx` | `(PremiereDate DESC)` | Series sorted by date |
|
||||
|
||||
**Critical Indexes Missing:**
|
||||
| Index | Impact | Why Missing |
|
||||
|-------|--------|------------|
|
||||
| `Type` (standalone) | Full table scan when filtering by Type alone | Not included in migration |
|
||||
| `(IsFolder, Type)` | Fast Series queries without ParentId filter | Not included in migration |
|
||||
| `(Type, IsFolder)` | Fast queries: "all Series" without folder hierarchy | Not included in migration |
|
||||
| `Type` with filter on `IsSeries=true` | IsSeries column not indexed | Not included in migration |
|
||||
|
||||
**Why This Matters:**
|
||||
```sql
|
||||
-- Current: Uses composite index (ParentId, Type) if available
|
||||
SELECT * FROM BaseItems WHERE ParentId = $1 AND Type = 'Series';
|
||||
|
||||
-- Missing: NO index for this common query pattern
|
||||
SELECT * FROM BaseItems WHERE Type = 'Series';
|
||||
|
||||
-- Missing: NO index for this optimization
|
||||
SELECT * FROM BaseItems WHERE IsFolder = true AND Type = 'Series';
|
||||
```
|
||||
|
||||
### TvExtras Table Indexes
|
||||
- **Missing**: Index on `SeriesId` (needed for tag inheritance queries)
|
||||
- **Missing**: Composite `(ItemId, SeriesPresentationUniqueKey)`
|
||||
|
||||
---
|
||||
|
||||
## 7. ApplyGroupingFilter Logic
|
||||
|
||||
### Location: Lines 740-778
|
||||
|
||||
```csharp
|
||||
private IQueryable<BaseItemEntity> ApplyGroupingFilter(JellyfinDbContext context, IQueryable<BaseItemEntity> dbQuery, InternalItemsQuery filter)
|
||||
{
|
||||
var enableGroupByPresentationUniqueKey = EnableGroupByPresentationUniqueKey(filter);
|
||||
|
||||
if (enableGroupByPresentationUniqueKey && filter.GroupBySeriesPresentationUniqueKey)
|
||||
{
|
||||
// PERFORMANCE FIX: Use DistinctBy() instead of GroupBy().Select().FirstOrDefault()
|
||||
dbQuery = dbQuery.DistinctBy(e => new { e.PresentationUniqueKey, SeriesPresentationUniqueKey = e.TvExtras!.SeriesPresentationUniqueKey });
|
||||
}
|
||||
else if (enableGroupByPresentationUniqueKey)
|
||||
{
|
||||
// PERFORMANCE FIX: Use DistinctBy() to generate DISTINCT ON instead of correlated subquery
|
||||
dbQuery = dbQuery.DistinctBy(e => e.PresentationUniqueKey);
|
||||
}
|
||||
else if (filter.GroupBySeriesPresentationUniqueKey)
|
||||
{
|
||||
// PERFORMANCE FIX: Use DistinctBy() instead of GroupBy().Select().FirstOrDefault()
|
||||
dbQuery = dbQuery.DistinctBy(e => e.TvExtras!.SeriesPresentationUniqueKey);
|
||||
}
|
||||
else
|
||||
{
|
||||
dbQuery = dbQuery.Distinct();
|
||||
}
|
||||
|
||||
dbQuery = ApplyOrder(dbQuery, filter, context);
|
||||
return dbQuery;
|
||||
}
|
||||
```
|
||||
|
||||
**Key Insight**:
|
||||
- `DistinctBy()` translates to PostgreSQL's `DISTINCT ON` clause
|
||||
- This is **much faster** than correlated subqueries
|
||||
- But still must fetch all matching rows first
|
||||
|
||||
**For Series Queries:**
|
||||
- `GroupBySeriesPresentationUniqueKey=true`
|
||||
- Translates to: `SELECT DISTINCT ON (TvExtras.SeriesPresentationUniqueKey) ...`
|
||||
- This means: "Give me one row per unique Series"
|
||||
|
||||
**The Problem:**
|
||||
- Must ORDER BY before using DISTINCT ON (PostgreSQL requirement)
|
||||
- With many Episodes, sorting 10,000 rows then deduplicating is slower than database-level grouping
|
||||
|
||||
---
|
||||
|
||||
## 8. Complete Series Query Execution Example
|
||||
|
||||
### Scenario: User Clicks "TV Shows" in Library
|
||||
|
||||
**URL**: `/Items?IncludeItemTypes=Series&ParentId=X`
|
||||
|
||||
**What Happens:**
|
||||
|
||||
```
|
||||
1. GetItemsAsync() called with:
|
||||
- IncludeItemTypes: [Series]
|
||||
- ParentId: X (TV Shows folder)
|
||||
- EnableTotalRecordCount: true
|
||||
|
||||
2. PrepareItemQuery()
|
||||
└─ IQueryable<BaseItemEntity> dbQuery = context.BaseItems.AsNoTracking().AsSingleQuery()
|
||||
|
||||
3. TranslateQuery()
|
||||
└─ WHERE Type = 'Series'
|
||||
└─ WHERE ParentId = X
|
||||
└─ (If indexed: uses baseitems_parentid_idx)
|
||||
|
||||
4. ApplyOrder()
|
||||
└─ ORDER BY SortName (or other sort)
|
||||
|
||||
5. QUERY 1 (Lines 455-460):
|
||||
SELECT Id, PresentationUniqueKey, TvExtras.SeriesPresentationUniqueKey
|
||||
FROM BaseItems
|
||||
LEFT JOIN TvExtras ON BaseItems.Id = TvExtras.ItemId
|
||||
WHERE ParentId = X AND Type = 'Series'
|
||||
ORDER BY SortName
|
||||
└─ Result: 500 Series rows (1 row per Series, TvExtras joined)
|
||||
|
||||
6. ApplyGroupingInMemory()
|
||||
└─ Already unique, no deduplication needed for Series-only queries
|
||||
|
||||
7. ApplyPagingToIds()
|
||||
└─ Skip 0, Take 100 (if paginated)
|
||||
|
||||
8. QUERY 2 (Lines 475-481):
|
||||
SELECT * FROM BaseItems
|
||||
LEFT JOIN TvExtras ON BaseItems.Id = TvExtras.ItemId
|
||||
LEFT JOIN Provider ON BaseItems.Id = Provider.ItemId
|
||||
LEFT JOIN Images ON BaseItems.Id = Images.ItemId
|
||||
LEFT JOIN UserData ON BaseItems.Id = UserData.ItemId
|
||||
WHERE Id IN (500 Series IDs from paged list)
|
||||
└─ Result: Full Series entities with all related data
|
||||
|
||||
9. Deserialize to DTOs
|
||||
└─ Convert BaseItemEntity to BaseItemDto
|
||||
└─ Map UserData if available
|
||||
|
||||
10. Return to client
|
||||
```
|
||||
|
||||
**Total Queries: 2**
|
||||
- **Query 1**: Get Series IDs (fast, uses index)
|
||||
- **Query 2**: Get full entities (includes multiple JOINs)
|
||||
|
||||
**Bottleneck**:
|
||||
- If there are 10,000 Episodes with same Parent, Query 1 returns 10,000 rows (one per episode), then deduplicates
|
||||
- This is rare for direct Series queries, but happens when:
|
||||
- Querying Episodes: `IncludeItemTypes=Episode&ParentId=X`
|
||||
- Querying all content: `IncludeItemTypes=[Series,Episode,Movie]`
|
||||
|
||||
---
|
||||
|
||||
## 9. Performance Comparison: Series vs. Movies vs. Music
|
||||
|
||||
### Query Pattern Differences
|
||||
|
||||
**Movies** (Fast):
|
||||
```csharp
|
||||
// No special handling, 1:1 with display
|
||||
WHERE Type = 'Movie'
|
||||
// Result: 100 rows = 100 Movies shown
|
||||
```
|
||||
|
||||
**Music Albums** (Medium):
|
||||
```csharp
|
||||
// Grouped by Album name
|
||||
WHERE Type = 'MusicAlbum'
|
||||
SELECT DISTINCT ON (Album) ...
|
||||
// Result: 100 rows = 100 Albums
|
||||
```
|
||||
|
||||
**TV Shows** (Slow):
|
||||
```csharp
|
||||
// Two cases:
|
||||
// Case 1: Query for Series themselves
|
||||
WHERE Type = 'Series'
|
||||
// Result: 500 rows = 500 Series (GOOD)
|
||||
|
||||
// Case 2: Query for all Episodes (common in "latest" or "browse all")
|
||||
WHERE Type = 'Episode'
|
||||
SELECT DISTINCT ON (TvExtras.SeriesPresentationUniqueKey) ...
|
||||
// Result: 10,000 rows → deduplicate to 500 → return 100
|
||||
// PROBLEM: Loaded 10,000, returned 100
|
||||
```
|
||||
|
||||
### Why Series Are Slower
|
||||
|
||||
| Aspect | Movies | TV Shows | Impact |
|
||||
|--------|--------|----------|--------|
|
||||
| Deduplication | None | Required (Series ID) | 10-100x more rows loaded |
|
||||
| Table joins | 1 (BaseItems) | 3+ (BaseItems + TvExtras + Tags checks) | Multiple JOINs per query |
|
||||
| Filter complexity | Simple | Complex (tag inheritance, IsPlayed special case) | More CPU, more conditions |
|
||||
| Index usage | Direct on Type | Composite (ParentId, Type) usually needed | Fewer index options |
|
||||
| Related data | Simple | Children (episodes), parent (series for episodes) | More eager-load navigations |
|
||||
| In-memory ops | Minimal | Full list loaded for deduplication | Memory spike |
|
||||
|
||||
---
|
||||
|
||||
## 10. Special Handling That Slows TV Shows
|
||||
|
||||
### 1. IsPlayed Special Case (Line 2995)
|
||||
```csharp
|
||||
if (filter.IncludeItemTypes.Length == 1 && filter.IncludeItemTypes[0] == BaseItemKind.Series)
|
||||
{
|
||||
// Correlated subquery: For each Series, check if ANY episode was played
|
||||
baseQuery = baseQuery.Where(e =>
|
||||
context.BaseItems
|
||||
.Where(f => f.TvExtras!.SeriesPresentationUniqueKey == e.PresentationUniqueKey)
|
||||
.Any(f => f.UserData!.FirstOrDefault()!.Played));
|
||||
}
|
||||
```
|
||||
**Impact**: Extra subquery per Series
|
||||
|
||||
### 2. Tag Inheritance for Episodes (Line 3427-3438)
|
||||
```csharp
|
||||
// Check episode's tags AND parent series' tags
|
||||
&& (!e.TvExtras!.SeriesId.HasValue ||
|
||||
!context.ItemValuesMap.Any(f =>
|
||||
f.ItemId == e.TvExtras!.SeriesId.Value &&
|
||||
f.ItemValue.Type == ItemValueType.Tags))
|
||||
```
|
||||
**Impact**: Joins to ItemValuesMap for every tag filter on episodes
|
||||
|
||||
### 3. AsSingleQuery() (Line 914)
|
||||
```csharp
|
||||
dbQuery = dbQuery.AsSingleQuery();
|
||||
```
|
||||
**Impact**: Prevents EF Core from splitting queries, but means all JOINs happen in one huge query
|
||||
- Better: Avoids N+1
|
||||
- Worse: One massive JOIN chain for all navigations
|
||||
|
||||
---
|
||||
|
||||
## 11. Recommendations for Optimization
|
||||
|
||||
### Immediate (No Schema Changes)
|
||||
1. **Add Index on Type column**
|
||||
```sql
|
||||
CREATE INDEX CONCURRENTLY idx_baseitems_type
|
||||
ON library."BaseItems" ("Type");
|
||||
```
|
||||
|
||||
2. **Add Composite Index for IsFolder + Type**
|
||||
```sql
|
||||
CREATE INDEX CONCURRENTLY idx_baseitems_isfolder_type
|
||||
ON library."BaseItems" ("IsFolder", "Type");
|
||||
```
|
||||
|
||||
3. **Add Index on TvExtras.SeriesId**
|
||||
```sql
|
||||
CREATE INDEX CONCURRENTLY idx_baseitem_tv_extras_seriesid
|
||||
ON library."BaseItemTvExtras" ("SeriesId");
|
||||
```
|
||||
|
||||
### Short-term (Application Changes)
|
||||
1. **Cache Series child counts** to avoid repeated queries
|
||||
2. **Lazy-load TvExtras** only when needed (not always via Include)
|
||||
3. **Optimize IsPlayed query** for Series - precompute in separate query
|
||||
|
||||
### Long-term (Architecture Changes)
|
||||
1. **Denormalize Series child count** into BaseItems table
|
||||
2. **Create materialized view** for Series with episode count, last played date
|
||||
3. **Use read-only replica** for expensive Series queries
|
||||
4. **Implement Series-specific cache layer** in BaseItemRepository
|
||||
|
||||
---
|
||||
|
||||
## Summary: Why TV Shows Are Slower
|
||||
|
||||
| Factor | Impact Level | Reason |
|
||||
|--------|--------------|--------|
|
||||
| **Extra Table Joins** | ⭐⭐⭐⭐⭐ | TvExtras, tag checks, user data |
|
||||
| **In-Memory Deduplication** | ⭐⭐⭐⭐ | All episodes loaded, deduplicated in C# |
|
||||
| **Missing Indexes** | ⭐⭐⭐⭐ | No Type or (IsFolder, Type) indexes |
|
||||
| **Correlated Subqueries** | ⭐⭐⭐⭐ | IsPlayed, tag inheritance, child counts |
|
||||
| **Complex Filtering** | ⭐⭐⭐ | Tag inheritance from parent Series |
|
||||
| **AsSingleQuery()** | ⭐⭐ | One huge JOIN chain, not optimizable |
|
||||
|
||||
**Total Impact**: TV Shows queries can be **10-100x slower** than Movies, depending on library size and query type.
|
||||
Reference in New Issue
Block a user