pgsql-jellyfin/docs/QUERY_ISSUES_REMEDIATION.md

# Query Execution - Quick Reference & Remediation Guide

## The Query Flow in 30 Seconds

```
Browser loads page
    ↓
Multiple API calls (likely different item types/filters)
    ↓
ItemsController.GetItems() × N
    ↓
Build InternalItemsQuery (with DtoOptions.EnableUserData=true by default)
    ↓
BaseItemRepository.GetItemsAsync()
    ├─ Query 1: Get filtered item IDs
    └─ Query 2: Load full items with eager loading:
       ├─ .Include(e => e.UserData) ← UserData loaded HERE
       ├─ .Include(e => e.Provider)
       ├─ .Include(e => e.Images)
       ├─ .Include(e => e.TvExtras)
       ├─ .Include(e => e.LiveTvExtras)
       └─ .Include(e => e.AudioExtras)
    ↓
DtoService.GetBaseItemDtos() - For each item:
    ├─ GetBaseItemDtoInternal()
    │  └─ AttachUserSpecificInfo()
    │     └─ GetUserDataDto()
    │        └─ GetUserData() ← Accesses in-memory UserData (NO QUERY)
    └─ [Potential N+1] SetItemByNameInfo() if ItemCounts requested
    └─ [Potential N+1] GetChildCount() if ChildCount requested
    ↓
Return DTOs to browser
```

## Identified Query Issues & Solutions

### 1. Multiple API Endpoints During Page Load
**Status**: ✓ **Normal behavior, not a bug**

**What's Happening**:
The web UI calls multiple GetItems endpoints simultaneously for:
- Movies library
- Series library  
- Recently added
- Resume items
- Playlists

Each is a separate API call → separate database query.

**Evidence to Confirm**:
```bash
# Check network tab in browser DevTools
# Look for multiple /Items calls with different parameters
```

**Solution**: Consolidate UI requests or add caching if same queries repeat.

---

### 2. ItemCounts Field N+1 Pattern
**Status**: ⚠️ **Actual N+1 detected**

**What's Happening** at [Emby.Server.Implementations/Dto/DtoService.cs](Emby.Server.Implementations/Dto/DtoService.cs#L182):
```csharp
for (int index = 0; index < accessibleItems.Count; index++)
{
    var item = accessibleItems[index];
    var dto = GetBaseItemDtoInternal(item, options, user, owner);
    
    if (options.ContainsField(ItemFields.ItemCounts))  // ← If requested
    {
        SetItemByNameInfo(dto, user);  // ← Called for EACH item
    }
}
```

**Risk Level**: HIGH if ItemCounts is frequently requested

**To Check**: Look for `ItemCounts` in API calls:
```bash
# Browser DevTools → Network → GetItems request
# Check if query parameters include ItemCounts
```

**Fix**: Batch SetItemByNameInfo operations instead of per-item:
```csharp
if (options.ContainsField(ItemFields.ItemCounts))
{
    // Batch operation - collect all IDs first
    var itemIds = returnItems.Select(r => r.Id).ToList();
    var itemCountsMap = GetItemCountsForItems(itemIds, user);
    
    foreach (var dto in returnItems)
    {
        if (itemCountsMap.TryGetValue(dto.Id, out var counts))
        {
            dto.ItemCounts = counts;
        }
    }
}
```

---

### 3. ChildCount Field Potential N+1
**Status**: ⚠️ **Partially optimized**

**What's Happening** at [Emby.Server.Implementations/Dto/DtoService.cs](Emby.Server.Implementations/Dto/DtoService.cs#L497):
```csharp
private static int GetChildCount(Folder folder, User user)
{
    if (folder is ICollectionFolder || folder is UserView)
    {
        return Random.Shared.Next(1, 10);  // ✓ NO QUERY
    }

    return folder.GetChildCount(user);  // ⚠️ POTENTIAL QUERY
}
```

**Current Status**:
- ✓ Collection folders & user views: return random (optimized, no query)
- ⚠️ Regular folders: call GetChildCount() which MAY query

**To Check**: 
```bash
# Enable EF logging to see if GetChildCount() queries per-item
"Microsoft.EntityFrameworkCore.Database.Command": "Debug"
```

**Potential Fix**: Cache folder child counts:
```csharp
private static readonly Dictionary<Guid, int> _childCountCache = new();

private static int GetChildCount(Folder folder, User user)
{
    if (folder is ICollectionFolder || folder is UserView)
    {
        return Random.Shared.Next(1, 10);
    }

    if (_childCountCache.TryGetValue(folder.Id, out var count))
    {
        return count;
    }

    count = folder.GetChildCount(user);
    _childCountCache[folder.Id] = count;
    return count;
}
```

---

### 4. People Field Processing
**Status**: ⚠️ **Per-item operation**

**What's Happening** at [Emby.Server.Implementations/Dto/DtoService.cs](Emby.Server.Implementations/Dto/DtoService.cs#L241):
```csharp
if (options.ContainsField(ItemFields.People))
{
    AttachPeople(dto, item, user);  // ← Called for EACH item
}
```

**Risk Level**: MEDIUM - depends on AttachPeople implementation

**To Investigate**:
- Search for AttachPeople implementation
- Check if it queries database per-item

---

### 5. MediaSources Retrieval
**Status**: ⚠️ **Per-item operation**

**What's Happening** at [Emby.Server.Implementations/Dto/DtoService.cs](Emby.Server.Implementations/Dto/DtoService.cs#L264):
```csharp
if (item is IHasMediaSources
    && options.ContainsField(ItemFields.MediaSources))
{
    dto.MediaSources = _mediaSourceManager.GetStaticMediaSources(item, true, user).ToArray();
}
```

**Risk Level**: MEDIUM - depends on GetStaticMediaSources implementation

**To Investigate**:
- Check if GetStaticMediaSources queries database
- If it does, batch retrieve all media sources

---

## How to Identify Which N+1 Pattern Is Occurring

### Step 1: Enable SQL Logging
Modify [Jellyfin.Server/Resources/Configuration/logging.json](Jellyfin.Server/Resources/Configuration/logging.json):
```json
{
    "Serilog": {
        "MinimumLevel": {
            "Override": {
                "Microsoft.EntityFrameworkCore.Database.Command": "Debug"
            }
        }
    }
}
```

### Step 2: Clear Logs
```bash
rm /var/log/jellyfin/log_*.log
# or for development: clear logs folder
```

### Step 3: Load Web UI Page
Navigate to the slow page and note the time.

### Step 4: Analyze Logs
```bash
# Look for timestamp of the page load
grep "SELECT" /var/log/jellyfin/log_*.log | tail -50

# Look for patterns like:
# - Same query repeated multiple times
# - "SELECT COUNT(*)" followed by "SELECT * FROM [BaseItems]"
# - Queries with same WHERE clause but different parameters
```

### Step 5: Correlate with Fields Requested
Check browser DevTools for API parameters:
```javascript
// In browser console
fetch("/Items?userId=...&fields=ItemCounts,ChildCount&...").then(r => r.json()).then(console.log)
```

---

## Performance Impact Summary

| Issue | Queries per Page Load | User Impact | Fix Priority |
|-------|---------------------|------------|--------------|
| Multiple API endpoints | 2-5 | Slow page load | Medium |
| ItemCounts N+1 | +20-100 | Very slow for large libraries | High |
| ChildCount N+1 | +10-50 | Slow for folder listings | Medium |
| People field | +5-20 | Slow if many items | Low |
| MediaSources | +5-20 | Slow if many video items | Low |

---

## Recommended Testing

### Test Case 1: Verify UserData is Not N+1
```
✓ Enable EF logging
✓ Load library page with 50 items
✓ Search logs for "SELECT.*FROM.*UserData"
✓ Should see only 1-2 queries for UserData (not 50)
```

### Test Case 2: Check ItemCounts Impact
```
✓ Note count of queries in normal page load
✓ Remove ItemCounts from request
✓ Reload and compare query count
✓ Difference = cost of ItemCounts feature
```

### Test Case 3: Monitor API Calls
```
✓ Open DevTools Network tab
✓ Clear filters
✓ Load home page
✓ Observe and count /Items calls
✓ Note different query parameters
```

---

## Action Items Priority

### 🔴 HIGH Priority
- [ ] Investigate if ItemCounts field causes N+1 queries
- [ ] Batch SetItemByNameInfo operations
- [ ] Profile library page load with 1000+ items

### 🟡 MEDIUM Priority
- [ ] Investigate ChildCount field N+1 potential
- [ ] Add caching to GetChildCount
- [ ] Verify AttachPeople doesn't query per-item

### 🟢 LOW Priority
- [ ] Investigate MediaSources retrieval
- [ ] Consider batching other per-item operations
- [ ] Monitor performance over time

---

## References

- Main Query Execution: [QUERY_EXECUTION_ANALYSIS.md](QUERY_EXECUTION_ANALYSIS.md)
- ItemsController: [Jellyfin.Api/Controllers/ItemsController.cs](Jellyfin.Api/Controllers/ItemsController.cs#L166)
- DtoService: [Emby.Server.Implementations/Dto/DtoService.cs](Emby.Server.Implementations/Dto/DtoService.cs#L160)
- BaseItemRepository: [Jellyfin.Server.Implementations/Item/BaseItemRepository.cs](Jellyfin.Server.Implementations/Item/BaseItemRepository.cs#L421)