# Query Path Map - Specific SQL & Code Locations ## Query Execution Paths ### Path 1: Primary Item Query (Always Executed) ``` Queries: 2-3 depending on ApplyGroupingInMemory logic 1. QUERY: Get filtered item IDs SELECT e.Id, e.PresentationUniqueKey, e.TvExtras!.SeriesPresentationUniqueKey FROM BaseItems e WHERE [filter conditions] ORDER BY [sort fields] File: BaseItemRepository.cs:453-455 Code: dbQuery.Select(e => new { e.Id, e.PresentationUniqueKey, ... }) .ToListAsync() 2. QUERY: Load full items with navigations (INCLUDES UserData) SELECT [ALL COLUMNS] FROM BaseItems e LEFT JOIN UserData ON e.Id = UserData.ItemId LEFT JOIN Provider ON e.Id = Provider.ItemId LEFT JOIN Images ON e.Id = Images.ItemId WHERE e.Id IN (SELECT [paged IDs]) File: BaseItemRepository.cs:468-473 Code: ApplyNavigations(context.BaseItems.Where(...), filter) .Include(e => e.UserData) .Include(e => e.Provider) .Include(e => e.Images) .ToListAsync() 3. QUERY: (Optional) Get TV/Audio extras if needed Already included via: .Include(e => e.TvExtras) .Include(e => e.LiveTvExtras) .Include(e => e.AudioExtras) ``` --- ### Path 2: ItemCounts N+1 Pattern (IF ItemCounts field requested) ``` Queries: +1 per item in result set For each item: IF options.ContainsField(ItemFields.ItemCounts): QUERY: Get item counts for THIS ITEM (Query details depend on SetItemByNameInfo implementation) File: DtoService.cs:182-183 Code: for (int index = 0; index < accessibleItems.Count; index++) { if (options.ContainsField(ItemFields.ItemCounts)) { SetItemByNameInfo(dto, user); // ← CALLED PER ITEM } } Example: If 20 items returned: Base query: 2 queries ItemCounts: +20 queries TOTAL: 22 queries ``` --- ### Path 3: ChildCount Potential N+1 (IF ChildCount field requested) ``` Queries: +1 per folder item (not for non-folder items) For each folder item: IF options.ContainsField(ItemFields.ChildCount): IF folder is ICollectionFolder OR UserView: NO QUERY: return Random.Shared.Next(1, 10) ELSE: QUERY: Get child count for THIS folder File: DtoService.cs:530-533 Code: GetChildCount(folder, user) → folder.GetChildCount(user) Likely query pattern: SELECT COUNT(*) FROM BaseItems WHERE ParentId = @folderId AND [visibility conditions for user] ``` --- ### Path 4: UserData Attachment (NO N+1 - Eagerly Loaded) ``` NO ADDITIONAL QUERIES - UserData is already in memory from Path 1 For each item: AttachUserSpecificInfo(dto, item, user, options) └─ GetUserData(user, item) └─ item.UserData?.Where(e => e.UserId.Equals(user.Id)) This is an in-memory LINQ query on the eagerly-loaded collection. File: UserDataManager.cs:247-253 Code: public UserItemData? GetUserData(User user, BaseItem item) { return item.UserData?.Where(e => e.UserId.Equals(user.Id)) .Select(Map) .FirstOrDefault() ?? new UserItemData() { Key = item.GetUserDataKeys()[0], }; } NO DATABASE ACCESS - all data already loaded. ``` --- ## Detailed Query Analysis by Request Type ### Request Type A: Basic Items Listing ``` GET /Items?userId={id}&startIndex=0&limit=20 ``` **Queries Executed**: ``` 1. Get IDs (filtering/sorting/paging) 2. Load full items with navigations TOTAL: 2 queries ``` **SQL Pattern**: ```sql -- Query 1: Get filtered+sorted IDs SELECT e.Id, e.PresentationUniqueKey, e.TvExtras.SeriesPresentationUniqueKey FROM BaseItems e WHERE [filter conditions] ORDER BY [sort fields] OFFSET 0 ROWS FETCH NEXT 20 ROWS ONLY; -- Query 2: Load full entities with navigations SELECT b.*, ud.Id, ud.UserId, ud.PlayCount, ..., p.Id, p.ProviderName, ..., i.Id, i.ImageType, ..., tv.Id, ..., la.Id, ..., ae.Id, ... FROM BaseItems b LEFT JOIN UserData ud ON b.Id = ud.ItemId LEFT JOIN Provider p ON b.Id = p.ItemId LEFT JOIN Images i ON b.Id = i.ItemId LEFT JOIN TvExtras tv ON b.Id = tv.ItemId LEFT JOIN LiveTvExtras la ON b.Id = la.ItemId LEFT JOIN AudioExtras ae ON b.Id = ae.ItemId WHERE b.Id IN (...20 IDs...); ``` --- ### Request Type B: With ItemCounts Field ``` GET /Items?userId={id}&startIndex=0&limit=20&fields=ItemCounts ``` **Queries Executed**: ``` 1. Get IDs (filtering/sorting/paging) 2. Load full items with navigations 3-22. SetItemByNameInfo for each of 20 items TOTAL: 22 queries ``` **Performance Impact**: 10x slower than basic listing --- ### Request Type C: With ChildCount Field ``` GET /Items?userId={id}&startIndex=0&limit=20&fields=ChildCount&includeItemTypes=Folder ``` **Queries Executed**: ``` 1. Get IDs (filtering/sorting/paging) 2. Load full items with navigations 3-N. GetChildCount for non-collection folders TOTAL: 2 + (count of non-collection folders) queries ``` --- ### Request Type D: Multiple Concurrent Calls (Real Page Load) ``` GET /Items?includeItemTypes=Movie&limit=16 GET /Items?includeItemTypes=Series&limit=16 GET /Items?filters=IsResumable&limit=16 GET /Users/{userId}/Items/Resume GET /Playlists ``` **Queries Executed** (assuming no ItemCounts/ChildCount): ``` Call 1 (Movies): 2 queries Call 2 (Series): 2 queries Call 3 (Resumable): 2 queries Call 4 (Resume items): 2 queries Call 5 (Playlists): 2 queries TOTAL: 10 queries ``` **Plus if ItemCounts requested**: ``` Call 1: 2 + 16 = 18 queries Call 2: 2 + 16 = 18 queries Call 3: 2 + 16 = 18 queries Call 4: 2 + 16 = 18 queries Call 5: 2 + 16 = 18 queries TOTAL: 90 queries ``` --- ## How to Trace Specific Queries ### Enable EF Core Command Logging **File**: [Jellyfin.Server/Resources/Configuration/logging.json](Jellyfin.Server/Resources/Configuration/logging.json) ```json { "Serilog": { "MinimumLevel": { "Default": "Information", "Override": { "Microsoft.EntityFrameworkCore.Database.Command": "Debug" } } } } ``` ### Restart and Capture Logs ```bash # Clear existing logs sudo rm /var/log/jellyfin/log_*.log # Restart service sudo systemctl restart jellyfin # Load web page # (navigate to Jellyfin in browser) # Examine logs tail -1000 /var/log/jellyfin/log_*.log | grep "Database.Command" # Or search for specific query patterns grep "FROM BaseItems" /var/log/jellyfin/log_*.log | wc -l grep "FROM UserData" /var/log/jellyfin/log_*.log | wc -l ``` --- ## Code Sections Responsible for Each Query | Query | Triggered By | Code Location | Frequency | |-------|--------------|---------------|-----------| | **Primary ID Query** | BaseItemRepository.GetItemsAsync() | BaseItemRepository.cs:453-455 | Always (1 per request) | | **Load Full Items** | ApplyNavigations() | BaseItemRepository.cs:468-473 | Always (1 per request) | | **SetItemByNameInfo** | ItemCounts field | DtoService.cs:182-183 | Per item if requested | | **GetChildCount** | ChildCount field | DtoService.cs:497-506 | Per folder if requested | | **AttachPeople** | People field | DtoService.cs:241-244 | Per item if requested | | **GetStaticMediaSources** | MediaSources field | DtoService.cs:264-268 | Per item if requested | --- ## Memory vs. Database Access ``` ┌─ BaseItemRepository.GetItemsAsync() │ ├─ Query 1: Get ID list │ └─ Database access │ ├─ Query 2: Load full items + navigations │ ├─ Database access │ └─ Result: BaseItem objects in memory with: │ ├─ UserData collection (loaded) │ ├─ Provider data (loaded) │ ├─ Images collection (loaded) │ └─ Etc. │ └─ DtoService.GetBaseItemDtos() │ └─ For each item in memory: ├─ GetBaseItemDtoInternal() │ ├─ AttachUserSpecificInfo() │ │ └─ GetUserData() │ │ └─ Access item.UserData (memory - NO DATABASE) │ │ │ ├─ AttachStudios() (memory) │ ├─ AttachBasicFields() (memory) │ └─ Etc. │ ├─ SetItemByNameInfo() (POTENTIAL DATABASE if requested) │ └─ GetChildCount() (POTENTIAL DATABASE if requested) ``` --- ## Query Optimization Opportunities ### Optimization 1: Batch ItemCounts **Current Cost**: N queries (one per item) **Optimized Cost**: 1 query (batch operation) ```csharp // BEFORE (Current - N queries) foreach (var item in items) { SetItemByNameInfo(dto, user); // Database query per item } // AFTER (Optimized - 1 query) var itemIds = items.Select(i => i.Id).ToArray(); var allCounts = await GetItemCountsBatch(itemIds, user); foreach (var dto in dtos) { if (allCounts.TryGetValue(dto.Id, out var counts)) { dto.ItemCounts = counts; } } ``` ### Optimization 2: Cache ChildCount **Current Cost**: N queries (one per folder per request) **Optimized Cost**: 1 query (one-time cache) ```csharp // Use a memory cache keyed by FolderId + UserId private readonly IMemoryCache _childCountCache; private int GetChildCount(Folder folder, User user) { var cacheKey = $"ChildCount_{folder.Id}_{user.Id}"; if (_childCountCache.TryGetValue(cacheKey, out int count)) { return count; } count = folder.GetChildCount(user); _childCountCache.Set(cacheKey, count, TimeSpan.FromMinutes(5)); return count; } ``` ### Optimization 3: Consolidate UI Requests **Current Cost**: 5+ simultaneous requests = 10+ queries **Optimized Cost**: 1 batch request = 2 queries ```javascript // BEFORE (Current - 5 API calls) Promise.all([ fetch('/Items?includeItemTypes=Movie&limit=16'), fetch('/Items?includeItemTypes=Series&limit=16'), fetch('/Items?filters=IsResumable&limit=16'), fetch('/Users/{userId}/Items/Resume'), fetch('/Playlists') ]); // AFTER (Optimized - 1 API call to get all) fetch('/Items?batch=true&includeItemTypes=Movie,Series&limit=16') ``` --- ## Monitoring Script ```bash #!/bin/bash # Capture query statistics during page load echo "Query execution started at $(date)" tail -f /var/log/jellyfin/log_*.log | \ grep -E "Database.Command|Executing|milliseconds" | \ while read line; do if [[ $line == *"FROM"* ]]; then echo "📊 QUERY: $(echo $line | grep -oE 'FROM [^ ]+')" elif [[ $line == *"Executing"* ]]; then echo "⏱️ TIMING: $line" fi done ``` Usage: ```bash chmod +x monitor_queries.sh ./monitor_queries.sh & # In another terminal, load the page firefox http://localhost:8096 # Stop monitoring kill %1 ``` --- ## References - **Main Analysis**: [QUERY_EXECUTION_ANALYSIS.md](QUERY_EXECUTION_ANALYSIS.md) - **Remediation Guide**: [QUERY_ISSUES_REMEDIATION.md](QUERY_ISSUES_REMEDIATION.md) - **Source Files**: - BaseItemRepository: [Jellyfin.Server.Implementations/Item/BaseItemRepository.cs](Jellyfin.Server.Implementations/Item/BaseItemRepository.cs) - DtoService: [Emby.Server.Implementations/Dto/DtoService.cs](Emby.Server.Implementations/Dto/DtoService.cs) - ItemsController: [Jellyfin.Api/Controllers/ItemsController.cs](Jellyfin.Api/Controllers/ItemsController.cs)