pgsql-jellyfin/QUERY_PATH_MAP.md

# Query Path Map - Specific SQL & Code Locations

## Query Execution Paths

### Path 1: Primary Item Query (Always Executed)
```
Queries: 2-3 depending on ApplyGroupingInMemory logic

1. QUERY: Get filtered item IDs
   SELECT e.Id, e.PresentationUniqueKey, 
          e.TvExtras!.SeriesPresentationUniqueKey
   FROM BaseItems e
   WHERE [filter conditions]
   ORDER BY [sort fields]
   
   File: BaseItemRepository.cs:453-455
   Code: dbQuery.Select(e => new { e.Id, e.PresentationUniqueKey, ... })
                .ToListAsync()

2. QUERY: Load full items with navigations (INCLUDES UserData)
   SELECT [ALL COLUMNS] FROM BaseItems e
   LEFT JOIN UserData ON e.Id = UserData.ItemId
   LEFT JOIN Provider ON e.Id = Provider.ItemId
   LEFT JOIN Images ON e.Id = Images.ItemId
   WHERE e.Id IN (SELECT [paged IDs])
   
   File: BaseItemRepository.cs:468-473
   Code: ApplyNavigations(context.BaseItems.Where(...), filter)
                .Include(e => e.UserData)
                .Include(e => e.Provider)
                .Include(e => e.Images)
                .ToListAsync()

3. QUERY: (Optional) Get TV/Audio extras if needed
   Already included via:
   .Include(e => e.TvExtras)
   .Include(e => e.LiveTvExtras)
   .Include(e => e.AudioExtras)
```

---

### Path 2: ItemCounts N+1 Pattern (IF ItemCounts field requested)
```
Queries: +1 per item in result set

For each item:
  IF options.ContainsField(ItemFields.ItemCounts):
    QUERY: Get item counts for THIS ITEM
    (Query details depend on SetItemByNameInfo implementation)
    
    File: DtoService.cs:182-183
    Code: for (int index = 0; index < accessibleItems.Count; index++)
          {
              if (options.ContainsField(ItemFields.ItemCounts))
              {
                  SetItemByNameInfo(dto, user);  // ← CALLED PER ITEM
              }
          }

Example: If 20 items returned:
  Base query: 2 queries
  ItemCounts: +20 queries
  TOTAL: 22 queries
```

---

### Path 3: ChildCount Potential N+1 (IF ChildCount field requested)
```
Queries: +1 per folder item (not for non-folder items)

For each folder item:
  IF options.ContainsField(ItemFields.ChildCount):
    IF folder is ICollectionFolder OR UserView:
      NO QUERY: return Random.Shared.Next(1, 10)
    ELSE:
      QUERY: Get child count for THIS folder
      File: DtoService.cs:530-533
      Code: GetChildCount(folder, user)
            → folder.GetChildCount(user)
            
      Likely query pattern:
      SELECT COUNT(*) FROM BaseItems 
      WHERE ParentId = @folderId 
            AND [visibility conditions for user]
```

---

### Path 4: UserData Attachment (NO N+1 - Eagerly Loaded)
```
NO ADDITIONAL QUERIES - UserData is already in memory from Path 1

For each item:
  AttachUserSpecificInfo(dto, item, user, options)
    └─ GetUserData(user, item)
       └─ item.UserData?.Where(e => e.UserId.Equals(user.Id))
          
This is an in-memory LINQ query on the eagerly-loaded collection.

File: UserDataManager.cs:247-253
Code: public UserItemData? GetUserData(User user, BaseItem item)
      {
          return item.UserData?.Where(e => e.UserId.Equals(user.Id))
                             .Select(Map)
                             .FirstOrDefault() ?? new UserItemData()
          {
              Key = item.GetUserDataKeys()[0],
          };
      }

NO DATABASE ACCESS - all data already loaded.
```

---

## Detailed Query Analysis by Request Type

### Request Type A: Basic Items Listing
```
GET /Items?userId={id}&startIndex=0&limit=20
```

**Queries Executed**:
```
1. Get IDs (filtering/sorting/paging)
2. Load full items with navigations
TOTAL: 2 queries
```

**SQL Pattern**:
```sql
-- Query 1: Get filtered+sorted IDs
SELECT e.Id, e.PresentationUniqueKey, e.TvExtras.SeriesPresentationUniqueKey
FROM BaseItems e
WHERE [filter conditions]
ORDER BY [sort fields]
OFFSET 0 ROWS FETCH NEXT 20 ROWS ONLY;

-- Query 2: Load full entities with navigations
SELECT b.*, 
       ud.Id, ud.UserId, ud.PlayCount, ...,
       p.Id, p.ProviderName, ...,
       i.Id, i.ImageType, ...,
       tv.Id, ...,
       la.Id, ...,
       ae.Id, ...
FROM BaseItems b
LEFT JOIN UserData ud ON b.Id = ud.ItemId
LEFT JOIN Provider p ON b.Id = p.ItemId
LEFT JOIN Images i ON b.Id = i.ItemId
LEFT JOIN TvExtras tv ON b.Id = tv.ItemId
LEFT JOIN LiveTvExtras la ON b.Id = la.ItemId
LEFT JOIN AudioExtras ae ON b.Id = ae.ItemId
WHERE b.Id IN (...20 IDs...);
```

---

### Request Type B: With ItemCounts Field
```
GET /Items?userId={id}&startIndex=0&limit=20&fields=ItemCounts
```

**Queries Executed**:
```
1. Get IDs (filtering/sorting/paging)
2. Load full items with navigations
3-22. SetItemByNameInfo for each of 20 items
TOTAL: 22 queries
```

**Performance Impact**: 10x slower than basic listing

---

### Request Type C: With ChildCount Field
```
GET /Items?userId={id}&startIndex=0&limit=20&fields=ChildCount&includeItemTypes=Folder
```

**Queries Executed**:
```
1. Get IDs (filtering/sorting/paging)
2. Load full items with navigations
3-N. GetChildCount for non-collection folders
TOTAL: 2 + (count of non-collection folders) queries
```

---

### Request Type D: Multiple Concurrent Calls (Real Page Load)
```
GET /Items?includeItemTypes=Movie&limit=16
GET /Items?includeItemTypes=Series&limit=16  
GET /Items?filters=IsResumable&limit=16
GET /Users/{userId}/Items/Resume
GET /Playlists
```

**Queries Executed** (assuming no ItemCounts/ChildCount):
```
Call 1 (Movies): 2 queries
Call 2 (Series): 2 queries
Call 3 (Resumable): 2 queries
Call 4 (Resume items): 2 queries
Call 5 (Playlists): 2 queries
TOTAL: 10 queries
```

**Plus if ItemCounts requested**:
```
Call 1: 2 + 16 = 18 queries
Call 2: 2 + 16 = 18 queries
Call 3: 2 + 16 = 18 queries
Call 4: 2 + 16 = 18 queries
Call 5: 2 + 16 = 18 queries
TOTAL: 90 queries
```

---

## How to Trace Specific Queries

### Enable EF Core Command Logging
**File**: [Jellyfin.Server/Resources/Configuration/logging.json](Jellyfin.Server/Resources/Configuration/logging.json)

```json
{
    "Serilog": {
        "MinimumLevel": {
            "Default": "Information",
            "Override": {
                "Microsoft.EntityFrameworkCore.Database.Command": "Debug"
            }
        }
    }
}
```

### Restart and Capture Logs
```bash
# Clear existing logs
sudo rm /var/log/jellyfin/log_*.log

# Restart service
sudo systemctl restart jellyfin

# Load web page
# (navigate to Jellyfin in browser)

# Examine logs
tail -1000 /var/log/jellyfin/log_*.log | grep "Database.Command"

# Or search for specific query patterns
grep "FROM BaseItems" /var/log/jellyfin/log_*.log | wc -l
grep "FROM UserData" /var/log/jellyfin/log_*.log | wc -l
```

---

## Code Sections Responsible for Each Query

| Query | Triggered By | Code Location | Frequency |
|-------|--------------|---------------|-----------|
| **Primary ID Query** | BaseItemRepository.GetItemsAsync() | BaseItemRepository.cs:453-455 | Always (1 per request) |
| **Load Full Items** | ApplyNavigations() | BaseItemRepository.cs:468-473 | Always (1 per request) |
| **SetItemByNameInfo** | ItemCounts field | DtoService.cs:182-183 | Per item if requested |
| **GetChildCount** | ChildCount field | DtoService.cs:497-506 | Per folder if requested |
| **AttachPeople** | People field | DtoService.cs:241-244 | Per item if requested |
| **GetStaticMediaSources** | MediaSources field | DtoService.cs:264-268 | Per item if requested |

---

## Memory vs. Database Access

```
┌─ BaseItemRepository.GetItemsAsync()
│
├─ Query 1: Get ID list
│  └─ Database access
│
├─ Query 2: Load full items + navigations
│  ├─ Database access
│  └─ Result: BaseItem objects in memory with:
│     ├─ UserData collection (loaded)
│     ├─ Provider data (loaded)
│     ├─ Images collection (loaded)
│     └─ Etc.
│
└─ DtoService.GetBaseItemDtos()
   │
   └─ For each item in memory:
      ├─ GetBaseItemDtoInternal()
      │  ├─ AttachUserSpecificInfo()
      │  │  └─ GetUserData()
      │  │     └─ Access item.UserData (memory - NO DATABASE)
      │  │
      │  ├─ AttachStudios() (memory)
      │  ├─ AttachBasicFields() (memory)
      │  └─ Etc.
      │
      ├─ SetItemByNameInfo() (POTENTIAL DATABASE if requested)
      │
      └─ GetChildCount() (POTENTIAL DATABASE if requested)
```

---

## Query Optimization Opportunities

### Optimization 1: Batch ItemCounts
**Current Cost**: N queries (one per item)
**Optimized Cost**: 1 query (batch operation)

```csharp
// BEFORE (Current - N queries)
foreach (var item in items)
{
    SetItemByNameInfo(dto, user);  // Database query per item
}

// AFTER (Optimized - 1 query)
var itemIds = items.Select(i => i.Id).ToArray();
var allCounts = await GetItemCountsBatch(itemIds, user);
foreach (var dto in dtos)
{
    if (allCounts.TryGetValue(dto.Id, out var counts))
    {
        dto.ItemCounts = counts;
    }
}
```

### Optimization 2: Cache ChildCount
**Current Cost**: N queries (one per folder per request)
**Optimized Cost**: 1 query (one-time cache)

```csharp
// Use a memory cache keyed by FolderId + UserId
private readonly IMemoryCache _childCountCache;

private int GetChildCount(Folder folder, User user)
{
    var cacheKey = $"ChildCount_{folder.Id}_{user.Id}";
    if (_childCountCache.TryGetValue(cacheKey, out int count))
    {
        return count;
    }
    
    count = folder.GetChildCount(user);
    _childCountCache.Set(cacheKey, count, TimeSpan.FromMinutes(5));
    return count;
}
```

### Optimization 3: Consolidate UI Requests
**Current Cost**: 5+ simultaneous requests = 10+ queries
**Optimized Cost**: 1 batch request = 2 queries

```javascript
// BEFORE (Current - 5 API calls)
Promise.all([
    fetch('/Items?includeItemTypes=Movie&limit=16'),
    fetch('/Items?includeItemTypes=Series&limit=16'),
    fetch('/Items?filters=IsResumable&limit=16'),
    fetch('/Users/{userId}/Items/Resume'),
    fetch('/Playlists')
]);

// AFTER (Optimized - 1 API call to get all)
fetch('/Items?batch=true&includeItemTypes=Movie,Series&limit=16')
```

---

## Monitoring Script

```bash
#!/bin/bash
# Capture query statistics during page load

echo "Query execution started at $(date)"
tail -f /var/log/jellyfin/log_*.log | \
grep -E "Database.Command|Executing|milliseconds" | \
while read line; do
    if [[ $line == *"FROM"* ]]; then
        echo "📊 QUERY: $(echo $line | grep -oE 'FROM [^ ]+')"
    elif [[ $line == *"Executing"* ]]; then
        echo "⏱️  TIMING: $line"
    fi
done
```

Usage:
```bash
chmod +x monitor_queries.sh
./monitor_queries.sh &

# In another terminal, load the page
firefox http://localhost:8096

# Stop monitoring
kill %1
```

---

## References

- **Main Analysis**: [QUERY_EXECUTION_ANALYSIS.md](QUERY_EXECUTION_ANALYSIS.md)
- **Remediation Guide**: [QUERY_ISSUES_REMEDIATION.md](QUERY_ISSUES_REMEDIATION.md)
- **Source Files**:
  - BaseItemRepository: [Jellyfin.Server.Implementations/Item/BaseItemRepository.cs](Jellyfin.Server.Implementations/Item/BaseItemRepository.cs)
  - DtoService: [Emby.Server.Implementations/Dto/DtoService.cs](Emby.Server.Implementations/Dto/DtoService.cs)
  - ItemsController: [Jellyfin.Api/Controllers/ItemsController.cs](Jellyfin.Api/Controllers/ItemsController.cs)