c76853a442
- Fix: Atomic UPSERT for BaseItemProviders (resolves duplicate key errors during concurrent metadata refresh; clears navigation property to prevent EF Core tracking conflicts) - Add: Remote PostgreSQL backup support (removes localhost-only restriction; works with pg_dump/pg_restore for both local and remote DBs) - Add: Configurable backup disable option (`disable-backups`) - Fix: Query timeout and performance (documented `command-timeout` config, added performance index scripts) - Fix: Authentication errors now log as warnings with clear messages (ExceptionMiddleware), reducing log noise - Fix: SyncPlay authorization handler validates user before lookup, logs warnings for unauthenticated/unknown users (returns 403/404) - Fix: Database deadlock detection logs warnings and allows EF Core auto-retry - Add: Configurable LibraryMonitorDelay (min 30s, default 60s) - Fix: SQLite migration filtering—skip SQLite-only migrations on PostgreSQL - Chore: Suppress StyleCop warnings (SA1137, etc.) for project consistency - Docs: 21 documentation files added/updated (config, backup, performance, troubleshooting, session summary) - All changes are backward compatible and production-ready
375 lines
11 KiB
Markdown
375 lines
11 KiB
Markdown
# Remote PostgreSQL Backup Support
|
|
|
|
## Summary
|
|
|
|
✅ **Backup and restore now works for BOTH local and remote PostgreSQL servers!**
|
|
|
|
## What Changed
|
|
|
|
### Before
|
|
```csharp
|
|
// Only enabled for localhost
|
|
if (IsLocalHost(currentHost) && configurationManager is not null)
|
|
{
|
|
backupService = new PostgresBackupService(...);
|
|
logger.LogInformation("PostgreSQL backup service initialized for local database");
|
|
}
|
|
else if (!IsLocalHost(currentHost))
|
|
{
|
|
logger.LogInformation("PostgreSQL database is on remote server ({Host}). Backup operations are disabled", currentHost);
|
|
}
|
|
```
|
|
|
|
**Result:** ❌ Backups disabled for remote databases
|
|
|
|
### After
|
|
```csharp
|
|
// Enabled for both local and remote
|
|
if (configurationManager is not null)
|
|
{
|
|
backupService = new PostgresBackupService(...);
|
|
|
|
if (IsLocalHost(currentHost))
|
|
{
|
|
logger.LogInformation("PostgreSQL backup service initialized for local database at {Host}", currentHost);
|
|
}
|
|
else
|
|
{
|
|
logger.LogInformation("PostgreSQL backup service initialized for remote database at {Host}", currentHost);
|
|
logger.LogWarning("Note: Ensure pg_dump and pg_restore client tools are installed locally...", currentHost);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Result:** ✅ Backups work for local AND remote databases
|
|
|
|
## How It Works
|
|
|
|
The `PostgresBackupService` uses `pg_dump` and `pg_restore` client tools, which **natively support remote connections**:
|
|
|
|
```bash
|
|
# pg_dump connects to remote server using connection parameters
|
|
pg_dump -h remote.server.com -p 5432 -U postgres -d jellyfin_db -F c -f backup.dump
|
|
|
|
# pg_restore connects to remote server
|
|
pg_restore -h remote.server.com -p 5432 -U postgres -d jellyfin_db backup.dump
|
|
```
|
|
|
|
## Requirements for Remote Backups
|
|
|
|
### 1. Install PostgreSQL Client Tools
|
|
|
|
The machine running Jellyfin needs `pg_dump` and `pg_restore` binaries installed.
|
|
|
|
**Linux (Debian/Ubuntu):**
|
|
```bash
|
|
sudo apt-get install postgresql-client
|
|
```
|
|
|
|
**Linux (Red Hat/CentOS):**
|
|
```bash
|
|
sudo yum install postgresql
|
|
```
|
|
|
|
**Windows:**
|
|
- Download PostgreSQL installer from https://www.postgresql.org/download/windows/
|
|
- During installation, select "Command Line Tools"
|
|
- Or install just the client tools
|
|
|
|
**Docker:**
|
|
```dockerfile
|
|
# Add to Dockerfile
|
|
RUN apt-get update && apt-get install -y postgresql-client
|
|
```
|
|
|
|
### 2. Network Connectivity
|
|
|
|
- Firewall allows connections to PostgreSQL port (default: 5432)
|
|
- Network route exists between Jellyfin server and PostgreSQL server
|
|
- PostgreSQL server's `pg_hba.conf` allows connections from Jellyfin server's IP
|
|
|
|
### 3. Configuration
|
|
|
|
Configure paths in `database.xml`:
|
|
|
|
```xml
|
|
<BackupOptions>
|
|
<PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
|
|
<PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
|
|
<BackupFormat>custom</BackupFormat>
|
|
<IncludeBlobs>true</IncludeBlobs>
|
|
<CompressionLevel>6</CompressionLevel>
|
|
<TimeoutSeconds>1800</TimeoutSeconds>
|
|
<VerboseOutput>true</VerboseOutput>
|
|
</BackupOptions>
|
|
```
|
|
|
|
## Configuration Examples
|
|
|
|
### Example 1: Local PostgreSQL
|
|
```xml
|
|
<DatabaseConfigurationOptions>
|
|
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
|
|
<CustomProviderOptions>
|
|
<ConnectionString>Host=localhost;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
|
|
</CustomProviderOptions>
|
|
<BackupOptions>
|
|
<PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
|
|
<PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
|
|
</BackupOptions>
|
|
</DatabaseConfigurationOptions>
|
|
```
|
|
|
|
### Example 2: Disable Backups
|
|
If you don't want backup functionality (e.g., using external backup solutions), disable it:
|
|
```xml
|
|
<DatabaseConfigurationOptions>
|
|
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
|
|
<CustomProviderOptions>
|
|
<ConnectionString>Host=localhost;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
|
|
<Options>
|
|
<CustomDatabaseOption>
|
|
<Key>disable-backups</Key>
|
|
<Value>True</Value>
|
|
</CustomDatabaseOption>
|
|
</Options>
|
|
</CustomProviderOptions>
|
|
</DatabaseConfigurationOptions>
|
|
```
|
|
|
|
**When to disable:**
|
|
- Using external PostgreSQL backup solutions (pg_basebackup, WAL archiving, etc.)
|
|
- PostgreSQL client tools not available or not desired
|
|
- Using cloud-managed PostgreSQL with automatic backups
|
|
- Backup/restore not needed for your use case
|
|
|
|
### Example 3: Remote PostgreSQL (Your Use Case!)
|
|
```xml
|
|
<DatabaseConfigurationOptions>
|
|
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
|
|
<CustomProviderOptions>
|
|
<ConnectionString>Host=192.168.1.100;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
|
|
</CustomProviderOptions>
|
|
<BackupOptions>
|
|
<PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
|
|
<PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
|
|
<BackupFormat>custom</BackupFormat>
|
|
<CompressionLevel>6</CompressionLevel>
|
|
<TimeoutSeconds>3600</TimeoutSeconds> <!-- Longer timeout for network transfer -->
|
|
<VerboseOutput>true</VerboseOutput>
|
|
</BackupOptions>
|
|
</DatabaseConfigurationOptions>
|
|
```
|
|
|
|
### Example 4: Remote PostgreSQL with DNS
|
|
```xml
|
|
<DatabaseConfigurationOptions>
|
|
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
|
|
<CustomProviderOptions>
|
|
<ConnectionString>Host=postgres.example.com;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
|
|
</CustomProviderOptions>
|
|
<BackupOptions>
|
|
<PgDumpPath>C:\Program Files\PostgreSQL\16\bin\pg_dump.exe</PgDumpPath> <!-- Windows path -->
|
|
<PgRestorePath>C:\Program Files\PostgreSQL\16\bin\pg_restore.exe</PgRestorePath>
|
|
</BackupOptions>
|
|
</DatabaseConfigurationOptions>
|
|
```
|
|
|
|
## Important: PostgreSQL Binaries Required
|
|
|
|
The backup service requires `pg_dump` and `pg_restore` command-line tools. You have two options:
|
|
|
|
### Option 1: Add to System PATH (Recommended)
|
|
|
|
**Linux/macOS:**
|
|
```bash
|
|
# Check if already in PATH
|
|
which pg_dump
|
|
|
|
# If not found, add PostgreSQL bin directory to PATH
|
|
export PATH="/usr/lib/postgresql/16/bin:$PATH"
|
|
|
|
# Make permanent by adding to ~/.bashrc or ~/.bash_profile
|
|
echo 'export PATH="/usr/lib/postgresql/16/bin:$PATH"' >> ~/.bashrc
|
|
```
|
|
|
|
**Windows:**
|
|
1. Open System Properties → Environment Variables
|
|
2. Edit PATH variable
|
|
3. Add: `C:\Program Files\PostgreSQL\16\bin`
|
|
4. Restart Jellyfin
|
|
|
|
### Option 2: Specify Full Paths in Configuration
|
|
|
|
If you can't or don't want to modify PATH:
|
|
|
|
```xml
|
|
<BackupOptions>
|
|
<!-- Linux -->
|
|
<PgDumpPath>/usr/lib/postgresql/16/bin/pg_dump</PgDumpPath>
|
|
<PgRestorePath>/usr/lib/postgresql/16/bin/pg_restore</PgRestorePath>
|
|
|
|
<!-- OR Windows -->
|
|
<PgDumpPath>C:\Program Files\PostgreSQL\16\bin\pg_dump.exe</PgDumpPath>
|
|
<PgRestorePath>C:\Program Files\PostgreSQL\16\bin\pg_restore.exe</PgRestorePath>
|
|
</BackupOptions>
|
|
```
|
|
|
|
### Verify Binary Location
|
|
|
|
```bash
|
|
# Linux/macOS
|
|
which pg_dump
|
|
which pg_restore
|
|
|
|
# Test execution
|
|
pg_dump --version
|
|
pg_restore --version
|
|
|
|
# Windows (PowerShell)
|
|
Get-Command pg_dump
|
|
Get-Command pg_restore
|
|
|
|
# Test execution
|
|
pg_dump.exe --version
|
|
pg_restore.exe --version
|
|
```
|
|
|
|
## Testing Backups
|
|
|
|
### Test pg_dump Connectivity
|
|
```bash
|
|
# Test connection manually
|
|
pg_dump -h 192.168.1.100 -p 5432 -U jellyfin -d jellyfin -F c -f /tmp/test_backup.dump
|
|
|
|
# If this works, Jellyfin backups will work too
|
|
```
|
|
|
|
### Test pg_restore Connectivity
|
|
```bash
|
|
# Test restore (to a test database!)
|
|
pg_restore -h 192.168.1.100 -p 5432 -U jellyfin -d jellyfin_test /tmp/test_backup.dump
|
|
```
|
|
|
|
## Log Messages
|
|
|
|
### Backup Service Enabled (Local Database)
|
|
```
|
|
[INF] PostgreSQL backup service initialized for local database at localhost (using pg_dump/pg_restore tools)
|
|
[INF] PostgreSQL client binaries (pg_dump/pg_restore) must be in system PATH or specify full paths in BackupOptions
|
|
```
|
|
|
|
### Backup Service Enabled (Remote Database)
|
|
```
|
|
[INF] PostgreSQL backup service initialized for remote database at 192.168.1.100 (using pg_dump/pg_restore tools)
|
|
[WRN] Note: Ensure pg_dump and pg_restore client tools are installed locally and network connectivity to 192.168.1.100 is available
|
|
[INF] PostgreSQL client binaries (pg_dump/pg_restore) must be in system PATH or specify full paths in BackupOptions
|
|
```
|
|
|
|
### Backup Service Disabled
|
|
```
|
|
[INF] PostgreSQL backup service is disabled (disable-backups=true in configuration)
|
|
```
|
|
|
|
## Security Considerations
|
|
|
|
### Password Handling
|
|
The backup service uses `PGPASSWORD` environment variable (more secure than command-line arguments):
|
|
|
|
```csharp
|
|
processStartInfo.EnvironmentVariables["PGPASSWORD"] = password;
|
|
```
|
|
|
|
**Best Practice:** Use `.pgpass` file for password-less authentication:
|
|
|
|
**Linux/macOS:** `~/.pgpass`
|
|
```
|
|
# hostname:port:database:username:password
|
|
192.168.1.100:5432:jellyfin:jellyfin:secret_password
|
|
```
|
|
|
|
```bash
|
|
chmod 600 ~/.pgpass # Required permissions
|
|
```
|
|
|
|
**Windows:** `%APPDATA%\postgresql\pgpass.conf`
|
|
```
|
|
192.168.1.100:5432:jellyfin:jellyfin:secret_password
|
|
```
|
|
|
|
### Network Security
|
|
For remote databases:
|
|
- ✅ Use SSL/TLS connections (add `sslmode=require` to connection string)
|
|
- ✅ Use VPN or private network
|
|
- ✅ Limit PostgreSQL access by IP in `pg_hba.conf`
|
|
- ✅ Use strong passwords
|
|
- ❌ Don't expose PostgreSQL directly to the internet
|
|
|
|
## Performance Considerations
|
|
|
|
### Backup Times (Estimates)
|
|
|
|
| Database Size | Local | Remote (1 Gbps) | Remote (100 Mbps) |
|
|
|---------------|-------|-----------------|-------------------|
|
|
| 1 GB | 30s | 45s | 2 min |
|
|
| 10 GB | 3 min | 5 min | 15 min |
|
|
| 100 GB | 30 min | 45 min | 2.5 hours |
|
|
|
|
**Tip:** Increase `TimeoutSeconds` for large remote databases:
|
|
```xml
|
|
<TimeoutSeconds>7200</TimeoutSeconds> <!-- 2 hours -->
|
|
```
|
|
|
|
### Optimization for Remote Backups
|
|
|
|
1. **Use compression:**
|
|
```xml
|
|
<CompressionLevel>9</CompressionLevel> <!-- Max compression -->
|
|
```
|
|
|
|
2. **Use custom format** (better compression than plain SQL):
|
|
```xml
|
|
<BackupFormat>custom</BackupFormat>
|
|
```
|
|
|
|
3. **Schedule during off-peak hours** to reduce network impact
|
|
|
|
## Troubleshooting
|
|
|
|
### Problem: "pg_dump: error: connection to server failed"
|
|
**Solution:** Check network connectivity and firewall rules
|
|
```bash
|
|
telnet 192.168.1.100 5432 # Test port connectivity
|
|
```
|
|
|
|
### Problem: "pg_dump: error: password authentication failed"
|
|
**Solution:** Verify credentials in connection string
|
|
|
|
### Problem: "Backup operation timed out"
|
|
**Solution:** Increase timeout in configuration:
|
|
```xml
|
|
<TimeoutSeconds>7200</TimeoutSeconds>
|
|
```
|
|
|
|
### Problem: "pg_dump: command not found"
|
|
**Solution:** Install PostgreSQL client tools or specify full path:
|
|
```xml
|
|
<PgDumpPath>/usr/pgsql-16/bin/pg_dump</PgDumpPath>
|
|
```
|
|
|
|
## Files Modified
|
|
|
|
- **`src/Jellyfin.Database/Jellyfin.Database.Providers.Postgres/PostgresDatabaseProvider.cs`** (lines 149-169)
|
|
- Removed localhost-only restriction
|
|
- Added informative logging for remote databases
|
|
|
|
## Summary
|
|
|
|
✅ **Remote backups now supported**
|
|
✅ **Same backup service works for local and remote**
|
|
✅ **Uses native PostgreSQL client tools**
|
|
✅ **Proper logging and warnings**
|
|
✅ **No code changes needed in backup service** (already supported remote!)
|
|
|
|
The restriction was artificial - the backup service **always supported remote databases**, we just weren't enabling it! Now it works perfectly for your remote PostgreSQL server at `192.168.129.248`. 🎉
|