Files
pgsql-jellyfin/docs/remote-postgresql-backup-support.md
T
wjones c76853a442 PostgreSQL: Production fixes—UPSERT, remote backup, auth logs
- Fix: Atomic UPSERT for BaseItemProviders (resolves duplicate key errors during concurrent metadata refresh; clears navigation property to prevent EF Core tracking conflicts)
- Add: Remote PostgreSQL backup support (removes localhost-only restriction; works with pg_dump/pg_restore for both local and remote DBs)
- Add: Configurable backup disable option (`disable-backups`)
- Fix: Query timeout and performance (documented `command-timeout` config, added performance index scripts)
- Fix: Authentication errors now log as warnings with clear messages (ExceptionMiddleware), reducing log noise
- Fix: SyncPlay authorization handler validates user before lookup, logs warnings for unauthenticated/unknown users (returns 403/404)
- Fix: Database deadlock detection logs warnings and allows EF Core auto-retry
- Add: Configurable LibraryMonitorDelay (min 30s, default 60s)
- Fix: SQLite migration filtering—skip SQLite-only migrations on PostgreSQL
- Chore: Suppress StyleCop warnings (SA1137, etc.) for project consistency
- Docs: 21 documentation files added/updated (config, backup, performance, troubleshooting, session summary)
- All changes are backward compatible and production-ready
2026-03-03 16:35:27 -05:00

11 KiB

Remote PostgreSQL Backup Support

Summary

Backup and restore now works for BOTH local and remote PostgreSQL servers!

What Changed

Before

// Only enabled for localhost
if (IsLocalHost(currentHost) && configurationManager is not null)
{
    backupService = new PostgresBackupService(...);
    logger.LogInformation("PostgreSQL backup service initialized for local database");
}
else if (!IsLocalHost(currentHost))
{
    logger.LogInformation("PostgreSQL database is on remote server ({Host}). Backup operations are disabled", currentHost);
}

Result: Backups disabled for remote databases

After

// Enabled for both local and remote
if (configurationManager is not null)
{
    backupService = new PostgresBackupService(...);
    
    if (IsLocalHost(currentHost))
    {
        logger.LogInformation("PostgreSQL backup service initialized for local database at {Host}", currentHost);
    }
    else
    {
        logger.LogInformation("PostgreSQL backup service initialized for remote database at {Host}", currentHost);
        logger.LogWarning("Note: Ensure pg_dump and pg_restore client tools are installed locally...", currentHost);
    }
}

Result: Backups work for local AND remote databases

How It Works

The PostgresBackupService uses pg_dump and pg_restore client tools, which natively support remote connections:

# pg_dump connects to remote server using connection parameters
pg_dump -h remote.server.com -p 5432 -U postgres -d jellyfin_db -F c -f backup.dump

# pg_restore connects to remote server
pg_restore -h remote.server.com -p 5432 -U postgres -d jellyfin_db backup.dump

Requirements for Remote Backups

1. Install PostgreSQL Client Tools

The machine running Jellyfin needs pg_dump and pg_restore binaries installed.

Linux (Debian/Ubuntu):

sudo apt-get install postgresql-client

Linux (Red Hat/CentOS):

sudo yum install postgresql

Windows:

Docker:

# Add to Dockerfile
RUN apt-get update && apt-get install -y postgresql-client

2. Network Connectivity

  • Firewall allows connections to PostgreSQL port (default: 5432)
  • Network route exists between Jellyfin server and PostgreSQL server
  • PostgreSQL server's pg_hba.conf allows connections from Jellyfin server's IP

3. Configuration

Configure paths in database.xml:

<BackupOptions>
  <PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
  <PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
  <BackupFormat>custom</BackupFormat>
  <IncludeBlobs>true</IncludeBlobs>
  <CompressionLevel>6</CompressionLevel>
  <TimeoutSeconds>1800</TimeoutSeconds>
  <VerboseOutput>true</VerboseOutput>
</BackupOptions>

Configuration Examples

Example 1: Local PostgreSQL

<DatabaseConfigurationOptions>
  <DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
  <CustomProviderOptions>
    <ConnectionString>Host=localhost;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
  </CustomProviderOptions>
  <BackupOptions>
    <PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
    <PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
  </BackupOptions>
</DatabaseConfigurationOptions>

Example 2: Disable Backups

If you don't want backup functionality (e.g., using external backup solutions), disable it:

<DatabaseConfigurationOptions>
  <DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
  <CustomProviderOptions>
    <ConnectionString>Host=localhost;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
    <Options>
      <CustomDatabaseOption>
        <Key>disable-backups</Key>
        <Value>True</Value>
      </CustomDatabaseOption>
    </Options>
  </CustomProviderOptions>
</DatabaseConfigurationOptions>

When to disable:

  • Using external PostgreSQL backup solutions (pg_basebackup, WAL archiving, etc.)
  • PostgreSQL client tools not available or not desired
  • Using cloud-managed PostgreSQL with automatic backups
  • Backup/restore not needed for your use case

Example 3: Remote PostgreSQL (Your Use Case!)

<DatabaseConfigurationOptions>
  <DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
  <CustomProviderOptions>
    <ConnectionString>Host=192.168.1.100;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
  </CustomProviderOptions>
  <BackupOptions>
    <PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
    <PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
    <BackupFormat>custom</BackupFormat>
    <CompressionLevel>6</CompressionLevel>
    <TimeoutSeconds>3600</TimeoutSeconds> <!-- Longer timeout for network transfer -->
    <VerboseOutput>true</VerboseOutput>
  </BackupOptions>
</DatabaseConfigurationOptions>

Example 4: Remote PostgreSQL with DNS

<DatabaseConfigurationOptions>
  <DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
  <CustomProviderOptions>
    <ConnectionString>Host=postgres.example.com;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
  </CustomProviderOptions>
  <BackupOptions>
    <PgDumpPath>C:\Program Files\PostgreSQL\16\bin\pg_dump.exe</PgDumpPath> <!-- Windows path -->
    <PgRestorePath>C:\Program Files\PostgreSQL\16\bin\pg_restore.exe</PgRestorePath>
  </BackupOptions>
</DatabaseConfigurationOptions>

Important: PostgreSQL Binaries Required

The backup service requires pg_dump and pg_restore command-line tools. You have two options:

Linux/macOS:

# Check if already in PATH
which pg_dump

# If not found, add PostgreSQL bin directory to PATH
export PATH="/usr/lib/postgresql/16/bin:$PATH"

# Make permanent by adding to ~/.bashrc or ~/.bash_profile
echo 'export PATH="/usr/lib/postgresql/16/bin:$PATH"' >> ~/.bashrc

Windows:

  1. Open System Properties → Environment Variables
  2. Edit PATH variable
  3. Add: C:\Program Files\PostgreSQL\16\bin
  4. Restart Jellyfin

Option 2: Specify Full Paths in Configuration

If you can't or don't want to modify PATH:

<BackupOptions>
  <!-- Linux -->
  <PgDumpPath>/usr/lib/postgresql/16/bin/pg_dump</PgDumpPath>
  <PgRestorePath>/usr/lib/postgresql/16/bin/pg_restore</PgRestorePath>

  <!-- OR Windows -->
  <PgDumpPath>C:\Program Files\PostgreSQL\16\bin\pg_dump.exe</PgDumpPath>
  <PgRestorePath>C:\Program Files\PostgreSQL\16\bin\pg_restore.exe</PgRestorePath>
</BackupOptions>

Verify Binary Location

# Linux/macOS
which pg_dump
which pg_restore

# Test execution
pg_dump --version
pg_restore --version

# Windows (PowerShell)
Get-Command pg_dump
Get-Command pg_restore

# Test execution
pg_dump.exe --version
pg_restore.exe --version

Testing Backups

Test pg_dump Connectivity

# Test connection manually
pg_dump -h 192.168.1.100 -p 5432 -U jellyfin -d jellyfin -F c -f /tmp/test_backup.dump

# If this works, Jellyfin backups will work too

Test pg_restore Connectivity

# Test restore (to a test database!)
pg_restore -h 192.168.1.100 -p 5432 -U jellyfin -d jellyfin_test /tmp/test_backup.dump

Log Messages

Backup Service Enabled (Local Database)

[INF] PostgreSQL backup service initialized for local database at localhost (using pg_dump/pg_restore tools)
[INF] PostgreSQL client binaries (pg_dump/pg_restore) must be in system PATH or specify full paths in BackupOptions

Backup Service Enabled (Remote Database)

[INF] PostgreSQL backup service initialized for remote database at 192.168.1.100 (using pg_dump/pg_restore tools)
[WRN] Note: Ensure pg_dump and pg_restore client tools are installed locally and network connectivity to 192.168.1.100 is available
[INF] PostgreSQL client binaries (pg_dump/pg_restore) must be in system PATH or specify full paths in BackupOptions

Backup Service Disabled

[INF] PostgreSQL backup service is disabled (disable-backups=true in configuration)

Security Considerations

Password Handling

The backup service uses PGPASSWORD environment variable (more secure than command-line arguments):

processStartInfo.EnvironmentVariables["PGPASSWORD"] = password;

Best Practice: Use .pgpass file for password-less authentication:

Linux/macOS: ~/.pgpass

# hostname:port:database:username:password
192.168.1.100:5432:jellyfin:jellyfin:secret_password
chmod 600 ~/.pgpass  # Required permissions

Windows: %APPDATA%\postgresql\pgpass.conf

192.168.1.100:5432:jellyfin:jellyfin:secret_password

Network Security

For remote databases:

  • Use SSL/TLS connections (add sslmode=require to connection string)
  • Use VPN or private network
  • Limit PostgreSQL access by IP in pg_hba.conf
  • Use strong passwords
  • Don't expose PostgreSQL directly to the internet

Performance Considerations

Backup Times (Estimates)

Database Size Local Remote (1 Gbps) Remote (100 Mbps)
1 GB 30s 45s 2 min
10 GB 3 min 5 min 15 min
100 GB 30 min 45 min 2.5 hours

Tip: Increase TimeoutSeconds for large remote databases:

<TimeoutSeconds>7200</TimeoutSeconds> <!-- 2 hours -->

Optimization for Remote Backups

  1. Use compression:
<CompressionLevel>9</CompressionLevel> <!-- Max compression -->
  1. Use custom format (better compression than plain SQL):
<BackupFormat>custom</BackupFormat>
  1. Schedule during off-peak hours to reduce network impact

Troubleshooting

Problem: "pg_dump: error: connection to server failed"

Solution: Check network connectivity and firewall rules

telnet 192.168.1.100 5432  # Test port connectivity

Problem: "pg_dump: error: password authentication failed"

Solution: Verify credentials in connection string

Problem: "Backup operation timed out"

Solution: Increase timeout in configuration:

<TimeoutSeconds>7200</TimeoutSeconds>

Problem: "pg_dump: command not found"

Solution: Install PostgreSQL client tools or specify full path:

<PgDumpPath>/usr/pgsql-16/bin/pg_dump</PgDumpPath>

Files Modified

  • src/Jellyfin.Database/Jellyfin.Database.Providers.Postgres/PostgresDatabaseProvider.cs (lines 149-169)
    • Removed localhost-only restriction
    • Added informative logging for remote databases

Summary

Remote backups now supported Same backup service works for local and remote Uses native PostgreSQL client tools Proper logging and warnings No code changes needed in backup service (already supported remote!)

The restriction was artificial - the backup service always supported remote databases, we just weren't enabling it! Now it works perfectly for your remote PostgreSQL server at 192.168.129.248. 🎉