pgsql-jellyfin/NPGSQL_COMMAND_IN_PROGRESS_FIX.md

PostgreSQL "Command Already in Progress" - Fix

Issue Description

Error: Npgsql.NpgsqlOperationInProgressException: A command is already in progress

Location: Jellyfin.Server\Migrations\Routines\MigrateRatingLevels.cs

Affected Operation: Migration routine for recalculating parental rating levels

---

Root Cause

The migration routine was causing a PostgreSQL connection conflict by:

1. Creating a lazy query: var ratings = context.BaseItems.Select(e => e.OfficialRating).Distinct();
2. Iterating over this query with foreach
3. While the iteration is reading from the database, executing ExecuteUpdate commands on the same context/connection

The Problem

// ❌ WRONG - Lazy query execution
var ratings = context.BaseItems.AsNoTracking().Select(e => e.OfficialRating).Distinct();

foreach (var rating in ratings)  // ← SELECT query is reading here
{
    context.BaseItems
        .Where(...)
        .ExecuteUpdate(...);  // ← UPDATE command tries to execute here
    
    // ERROR: Cannot execute UPDATE while SELECT is in progress!
}

PostgreSQL/Npgsql Limitation

PostgreSQL (via Npgsql) does not allow multiple active commands on the same connection simultaneously:

• The foreach loop is actively reading from a SELECT DISTINCT query
• The ExecuteUpdate tries to run an UPDATE command
• Result: NpgsqlOperationInProgressException

---

Solution

Materialize the query first using .ToList() before entering the loop:

// ✅ CORRECT - Materialize query first
var ratings = context.BaseItems.AsNoTracking()
    .Select(e => e.OfficialRating)
    .Distinct()
    .ToList();  // ← Execute the SELECT and load data into memory

foreach (var rating in ratings)  // ← Now iterating over in-memory list
{
    context.BaseItems
        .Where(...)
        .ExecuteUpdate(...);  // ← UPDATE can now execute freely
}

Why This Works

1. .ToList() forces immediate execution of the SELECT query
2. The database connection is released after the SELECT completes
3. The foreach loop now iterates over an in-memory list
4. Each ExecuteUpdate can use the connection independently
5. No conflict: No active SELECT when UPDATE runs

---

Code Changes

File: Jellyfin.Server\Migrations\Routines\MigrateRatingLevels.cs

Line 44 - Before:

var ratings = context.BaseItems.AsNoTracking().Select(e => e.OfficialRating).Distinct();

Line 44 - After:

// Materialize the ratings list first to avoid "command in progress" error
var ratings = context.BaseItems.AsNoTracking().Select(e => e.OfficialRating).Distinct().ToList();

Single character change: Added .ToList() to materialize the query

---

Impact

Performance

• Negligible: The distinct ratings list is typically very small (10-50 items)
• Memory: Minimal additional memory usage
• Speed: Query executes once upfront instead of being streamed

Reliability

• Before: Guaranteed failure with PostgreSQL
• After: Works correctly with PostgreSQL and all other database providers

Compatibility

• SQLite: Already worked (SQLite is more permissive)
• PostgreSQL: Now works correctly ✅
• SQL Server: Would also benefit from this fix

---

Best Practices

When Using EF Core Queries with ExecuteUpdate/ExecuteDelete

Rule: Always materialize queries (.ToList(), .ToArray()) before using them in loops that execute updates/deletes.

❌ Avoid (Lazy Execution)

var items = context.Items.Where(...).Select(...).Distinct();
foreach (var item in items)
{
    context.Items.Where(...).ExecuteUpdate(...);  // FAILS with PostgreSQL
}

✅ Correct (Eager Execution)

var items = context.Items.Where(...).Select(...).Distinct().ToList();
foreach (var item in items)
{
    context.Items.Where(...).ExecuteUpdate(...);  // WORKS
}

---

Related Issues

Similar Pattern to Watch For

This pattern can appear in:

• Migration routines
• Batch update operations
• Data cleanup jobs
• Background tasks

Detection

Look for:

1. A query without .ToList() / .ToArray()
2. Used in a foreach loop
3. With ExecuteUpdate / ExecuteDelete / SaveChanges inside the loop

PostgreSQL Error Messages

If you see:

Npgsql.NpgsqlOperationInProgressException: A command is already in progress: SELECT ...

Solution: Materialize the query before the loop!

---

Testing

Verify Fix

1. Restart Jellyfin server
2. Allow migration to run
3. Check logs for successful completion
4. Verify no NpgsqlOperationInProgressException errors

Expected Behavior

[INFO] Recalculating parental rating levels based on rating string.
[INFO] Migration completed successfully.

---

Additional Notes

Why SQLite Didn't Show This Issue

SQLite is more permissive and allows nested command execution on the same connection. This masked the issue during development, but PostgreSQL (correctly) enforces stricter connection usage rules.

PostgreSQL Multiplexing

Even with multiplexing enabled, you cannot execute multiple commands on the same transaction. This fix ensures only one command is active at a time within the transaction.

Long-term Solution

For very large datasets (1000+ distinct ratings), consider:

1. Batch processing
2. Separate queries for each rating
3. Parallel processing (outside transaction)

However, for this specific case (distinct ratings), the in-memory list is the optimal solution.

---

Summary

✅ Fixed: PostgreSQL connection conflict in rating migration
✅ Method: Added .ToList() to materialize query
✅ Impact: Migration now works correctly with PostgreSQL
✅ Build: Successful with zero errors

Status: Ready for testing and deployment

---

Document Version: 1.0
Date: 2025-01-15
Issue: NpgsqlOperationInProgressException
Status: ✅ FIXED