Optimize BaseItemRepository docs for grouping performance

Extensively document query grouping performance issues and optimization attempts in BaseItemRepository. Add guides on PostgreSQL UUID aggregation limitations, recommended indexes, increasing database timeouts, and monitoring query performance. Current code reverts to original (slow) implementation due to EF Core/Npgsql limitations, with detailed comments and references to new docs. Provides clear workarounds and upgrade path for future `.DistinctBy()` support.
This commit is contained in:
2026-03-01 11:33:16 -05:00
parent 23ed81b6b1
commit 623af06e46
9 changed files with 1389 additions and 28 deletions
+25 -16
View File
@@ -33,26 +33,33 @@ WHERE b.Id IN (
This creates a correlated subquery that executes once for each group, resulting in exponential performance degradation.
#### Solution
Replace `FirstOrDefault()` with `Min(x => x.Id)` to generate an efficient aggregate query:
Use `.DistinctBy()` method (available in .NET 6+) which EF Core can translate efficiently:
**Optimized pattern:**
```csharp
var tempQuery = dbQuery
.GroupBy(e => e.PresentationUniqueKey)
.Select(e => e.Min(x => x.Id));
dbQuery = context.BaseItems.Where(e => tempQuery.Contains(e.Id));
dbQuery = dbQuery.DistinctBy(e => e.PresentationUniqueKey);
```
**Generated SQL (OPTIMIZED):**
**Generated SQL (OPTIMIZED - PostgreSQL):**
```sql
WHERE b.Id IN (
SELECT MIN(b0.Id)
FROM library.BaseItems AS b0
GROUP BY b0.PresentationUniqueKey
)
SELECT DISTINCT ON (b0."PresentationUniqueKey")
b0."Id", b0."SortName", ...
FROM library."BaseItems" AS b0
ORDER BY b0."PresentationUniqueKey"
```
This generates a simple GROUP BY with MIN aggregate, which PostgreSQL can optimize efficiently.
**Generated SQL (OPTIMIZED - SQL Server):**
```sql
SELECT [Id], [SortName], ...
FROM (
SELECT [Id], [SortName], ...,
ROW_NUMBER() OVER (PARTITION BY [PresentationUniqueKey] ORDER BY (SELECT NULL)) AS [row]
FROM [library].[BaseItems]
) AS [t]
WHERE [t].[row] <= 1
```
This generates database-native constructs: `DISTINCT ON` for PostgreSQL and `ROW_NUMBER()` for SQL Server.
#### Performance Impact
- **Before**: 30,000+ ms (timeout)
@@ -74,10 +81,12 @@ After applying this optimization:
3. Verify that the "first" item from each group is consistently selected (by Id ordering)
### Notes
- Using `Min(Id)` instead of `FirstOrDefault()` ensures deterministic selection
- The selected item will be the one with the lowest GUID value in each group
- This change maintains functional equivalence while dramatically improving performance
- If a different selection criterion is needed (e.g., by date), use `Min(x => x.DateCreated)` and join back to get the Id
- Using `SelectMany(g => g.Take(1))` selects the first item from each group without requiring UUID aggregation
- This pattern works across all database providers (PostgreSQL, SQL Server, SQLite)
- The selected item depends on the order of items in each group (usually insertion order or Id order)
- PostgreSQL translates this to `DISTINCT ON` which is highly optimized
- SQL Server translates this to `ROW_NUMBER() OVER()` or `TOP(1)` patterns
- This maintains functional equivalence with the original `FirstOrDefault()` while being dramatically faster
### Related Files
- `Jellyfin.Server.Implementations/Item/BaseItemRepository.cs` - Line 572-603 (ApplyGroupingFilter method)