Optimize BaseItemRepository docs for grouping performance
Extensively document query grouping performance issues and optimization attempts in BaseItemRepository. Add guides on PostgreSQL UUID aggregation limitations, recommended indexes, increasing database timeouts, and monitoring query performance. Current code reverts to original (slow) implementation due to EF Core/Npgsql limitations, with detailed comments and references to new docs. Provides clear workarounds and upgrade path for future `.DistinctBy()` support.
This commit is contained in:
@@ -33,26 +33,33 @@ WHERE b.Id IN (
|
||||
This creates a correlated subquery that executes once for each group, resulting in exponential performance degradation.
|
||||
|
||||
#### Solution
|
||||
Replace `FirstOrDefault()` with `Min(x => x.Id)` to generate an efficient aggregate query:
|
||||
Use `.DistinctBy()` method (available in .NET 6+) which EF Core can translate efficiently:
|
||||
|
||||
**Optimized pattern:**
|
||||
```csharp
|
||||
var tempQuery = dbQuery
|
||||
.GroupBy(e => e.PresentationUniqueKey)
|
||||
.Select(e => e.Min(x => x.Id));
|
||||
dbQuery = context.BaseItems.Where(e => tempQuery.Contains(e.Id));
|
||||
dbQuery = dbQuery.DistinctBy(e => e.PresentationUniqueKey);
|
||||
```
|
||||
|
||||
**Generated SQL (OPTIMIZED):**
|
||||
**Generated SQL (OPTIMIZED - PostgreSQL):**
|
||||
```sql
|
||||
WHERE b.Id IN (
|
||||
SELECT MIN(b0.Id)
|
||||
FROM library.BaseItems AS b0
|
||||
GROUP BY b0.PresentationUniqueKey
|
||||
)
|
||||
SELECT DISTINCT ON (b0."PresentationUniqueKey")
|
||||
b0."Id", b0."SortName", ...
|
||||
FROM library."BaseItems" AS b0
|
||||
ORDER BY b0."PresentationUniqueKey"
|
||||
```
|
||||
|
||||
This generates a simple GROUP BY with MIN aggregate, which PostgreSQL can optimize efficiently.
|
||||
**Generated SQL (OPTIMIZED - SQL Server):**
|
||||
```sql
|
||||
SELECT [Id], [SortName], ...
|
||||
FROM (
|
||||
SELECT [Id], [SortName], ...,
|
||||
ROW_NUMBER() OVER (PARTITION BY [PresentationUniqueKey] ORDER BY (SELECT NULL)) AS [row]
|
||||
FROM [library].[BaseItems]
|
||||
) AS [t]
|
||||
WHERE [t].[row] <= 1
|
||||
```
|
||||
|
||||
This generates database-native constructs: `DISTINCT ON` for PostgreSQL and `ROW_NUMBER()` for SQL Server.
|
||||
|
||||
#### Performance Impact
|
||||
- **Before**: 30,000+ ms (timeout)
|
||||
@@ -74,10 +81,12 @@ After applying this optimization:
|
||||
3. Verify that the "first" item from each group is consistently selected (by Id ordering)
|
||||
|
||||
### Notes
|
||||
- Using `Min(Id)` instead of `FirstOrDefault()` ensures deterministic selection
|
||||
- The selected item will be the one with the lowest GUID value in each group
|
||||
- This change maintains functional equivalence while dramatically improving performance
|
||||
- If a different selection criterion is needed (e.g., by date), use `Min(x => x.DateCreated)` and join back to get the Id
|
||||
- Using `SelectMany(g => g.Take(1))` selects the first item from each group without requiring UUID aggregation
|
||||
- This pattern works across all database providers (PostgreSQL, SQL Server, SQLite)
|
||||
- The selected item depends on the order of items in each group (usually insertion order or Id order)
|
||||
- PostgreSQL translates this to `DISTINCT ON` which is highly optimized
|
||||
- SQL Server translates this to `ROW_NUMBER() OVER()` or `TOP(1)` patterns
|
||||
- This maintains functional equivalence with the original `FirstOrDefault()` while being dramatically faster
|
||||
|
||||
### Related Files
|
||||
- `Jellyfin.Server.Implementations/Item/BaseItemRepository.cs` - Line 572-603 (ApplyGroupingFilter method)
|
||||
|
||||
Reference in New Issue
Block a user