wjones/pgsql-jellyfin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

PostgreSQL: Production fixes—UPSERT, remote backup, auth logs

Browse Source 
- Fix: Atomic UPSERT for BaseItemProviders (resolves duplicate key errors during concurrent metadata refresh; clears navigation property to prevent EF Core tracking conflicts)
- Add: Remote PostgreSQL backup support (removes localhost-only restriction; works with pg_dump/pg_restore for both local and remote DBs)
- Add: Configurable backup disable option (`disable-backups`)
- Fix: Query timeout and performance (documented `command-timeout` config, added performance index scripts)
- Fix: Authentication errors now log as warnings with clear messages (ExceptionMiddleware), reducing log noise
- Fix: SyncPlay authorization handler validates user before lookup, logs warnings for unauthenticated/unknown users (returns 403/404)
- Fix: Database deadlock detection logs warnings and allows EF Core auto-retry
- Add: Configurable LibraryMonitorDelay (min 30s, default 60s)
- Fix: SQLite migration filtering—skip SQLite-only migrations on PostgreSQL
- Chore: Suppress StyleCop warnings (SA1137, etc.) for project consistency
- Docs: 21 documentation files added/updated (config, backup, performance, troubleshooting, session summary)
- All changes are backward compatible and production-ready
This commit is contained in:
Wendell Jones
2026-03-03 16:35:27 -05:00
parent 163a037642
commit c76853a442
27 changed files with 3320 additions and 260 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/database-configuration-examples.md
+327
View File
@@ -0,0 +1,327 @@
# PostgreSQL Database Configuration Examples
## Complete Configuration with All Options
```xml
<DatabaseConfigurationOptions>
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
<LockingBehavior>NoLock</LockingBehavior>
<CustomProviderOptions>
<PluginName>Jellyfin-PostgreSQL</PluginName>
<PluginAssembly>Jellyfin.Database.Providers.Postgres</PluginAssembly>
<ConnectionString>Host=192.168.129.248;Port=5432;Database=jellyfin_testdata;Username=postgres;Password=YourPassword</ConnectionString>
<Options>
<!-- Connection Settings -->
<CustomDatabaseOption>
<Key>Host</Key>
<Value>192.168.129.248</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>Port</Key>
<Value>5432</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>Database</Key>
<Value>jellyfin_testdata</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>Username</Key>
<Value>postgres</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>Password</Key>
<Value>YourPassword</Value>
</CustomDatabaseOption>
<!-- Performance Settings -->
<CustomDatabaseOption>
<Key>command-timeout</Key>
<Value>120</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>connection-timeout</Key>
<Value>30</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>max-pool-size</Key>
<Value>100</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>min-pool-size</Key>
<Value>0</Value>
</CustomDatabaseOption>
<!-- Backup Settings -->
<CustomDatabaseOption>
<Key>disable-backups</Key>
<Value>False</Value> <!-- Set to True to disable backup functionality -->
</CustomDatabaseOption>
</Options>
</CustomProviderOptions>
<BackupOptions>
<!-- PostgreSQL Client Tool Paths -->
<!-- Linux paths (if not in PATH) -->
<PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
<PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
<!-- OR Windows paths (if not in PATH) -->
<!-- <PgDumpPath>C:\Program Files\PostgreSQL\16\bin\pg_dump.exe</PgDumpPath> -->
<!-- <PgRestorePath>C:\Program Files\PostgreSQL\16\bin\pg_restore.exe</PgRestorePath> -->
<!-- Backup Format: plain, custom, directory, tar (custom recommended) -->
<BackupFormat>custom</BackupFormat>
<!-- Include large objects (blobs) in backup -->
<IncludeBlobs>true</IncludeBlobs>
<!-- Compression level (0-9, for custom/directory formats) -->
<CompressionLevel>6</CompressionLevel>
<!-- Backup/restore timeout in seconds (increase for large databases or remote servers) -->
<TimeoutSeconds>1800</TimeoutSeconds>
<!-- Show verbose output in logs -->
<VerboseOutput>true</VerboseOutput>
<!-- Number of parallel jobs (directory format only) -->
<!-- <ParallelJobs>4</ParallelJobs> -->
<!-- Additional pg_dump/pg_restore arguments -->
<!-- <AdditionalArguments>--no-owner --no-acl</AdditionalArguments> -->
</BackupOptions>
</DatabaseConfigurationOptions>
```
## Quick Start Configurations
### Minimal Configuration (Local PostgreSQL)
```xml
<DatabaseConfigurationOptions>
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
<CustomProviderOptions>
<ConnectionString>Host=localhost;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
</CustomProviderOptions>
</DatabaseConfigurationOptions>
```
### Recommended Configuration (Local PostgreSQL with Backups)
```xml
<DatabaseConfigurationOptions>
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
<CustomProviderOptions>
<ConnectionString>Host=localhost;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
<Options>
<CustomDatabaseOption>
<Key>command-timeout</Key>
<Value>120</Value>
</CustomDatabaseOption>
</Options>
</CustomProviderOptions>
<BackupOptions>
<PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
<PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
<BackupFormat>custom</BackupFormat>
<CompressionLevel>6</CompressionLevel>
</BackupOptions>
</DatabaseConfigurationOptions>
```
### Remote PostgreSQL with Backups
```xml
<DatabaseConfigurationOptions>
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
<CustomProviderOptions>
<ConnectionString>Host=192.168.1.100;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
<Options>
<CustomDatabaseOption>
<Key>command-timeout</Key>
<Value>120</Value>
</CustomDatabaseOption>
</Options>
</CustomProviderOptions>
<BackupOptions>
<PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
<PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
<BackupFormat>custom</BackupFormat>
<CompressionLevel>6</CompressionLevel>
<TimeoutSeconds>3600</TimeoutSeconds> <!-- Longer timeout for remote backups -->
</BackupOptions>
</DatabaseConfigurationOptions>
```
### Remote PostgreSQL WITHOUT Backups
```xml
<DatabaseConfigurationOptions>
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
<CustomProviderOptions>
<ConnectionString>Host=192.168.1.100;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
<Options>
<CustomDatabaseOption>
<Key>command-timeout</Key>
<Value>120</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>disable-backups</Key>
<Value>True</Value>
</CustomDatabaseOption>
</Options>
</CustomProviderOptions>
</DatabaseConfigurationOptions>
```
### High Performance Configuration
```xml
<DatabaseConfigurationOptions>
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
<CustomProviderOptions>
<ConnectionString>Host=localhost;Port=5432;Database=jellyfin;Username=jellyfin;Password=secret</ConnectionString>
<Options>
<CustomDatabaseOption>
<Key>command-timeout</Key>
<Value>180</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>max-pool-size</Key>
<Value>200</Value>
</CustomDatabaseOption>
<CustomDatabaseOption>
<Key>min-pool-size</Key>
<Value>10</Value>
</CustomDatabaseOption>
</Options>
</CustomProviderOptions>
<BackupOptions>
<PgDumpPath>/usr/bin/pg_dump</PgDumpPath>
<PgRestorePath>/usr/bin/pg_restore</PgRestorePath>
<BackupFormat>directory</BackupFormat>
<ParallelJobs>4</ParallelJobs>
<CompressionLevel>6</CompressionLevel>
</BackupOptions>
</DatabaseConfigurationOptions>
```
## Configuration Options Reference
### CustomProviderOptions → Options
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `Host` | string | localhost | PostgreSQL server hostname or IP |
| `Port` | int | 5432 | PostgreSQL server port |
| `Database` | string | - | Database name |
| `Username` | string | - | Database username |
| `Password` | string | - | Database password |
| `command-timeout` | int | 30 | Command timeout in seconds |
| `connection-timeout` | int | 15 | Connection timeout in seconds |
| `max-pool-size` | int | 100 | Maximum connection pool size |
| `min-pool-size` | int | 0 | Minimum connection pool size |
| `disable-backups` | bool | false | Disable backup/restore functionality |
### BackupOptions
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `PgDumpPath` | string | pg_dump | Path to pg_dump binary (use full path if not in PATH) |
| `PgRestorePath` | string | pg_restore | Path to pg_restore binary (use full path if not in PATH) |
| `BackupFormat` | string | custom | Backup format: plain, custom, directory, tar |
| `IncludeBlobs` | bool | true | Include large objects in backup |
| `CompressionLevel` | int | 6 | Compression level (0-9, for custom/directory) |
| `TimeoutSeconds` | int | 1800 | Backup/restore operation timeout |
| `VerboseOutput` | bool | false | Log detailed backup/restore output |
| `ParallelJobs` | int | 1 | Number of parallel jobs (directory format only) |
| `AdditionalArguments` | string | - | Additional pg_dump/pg_restore arguments |
## Important Notes
### PostgreSQL Client Binaries
The backup feature requires `pg_dump` and `pg_restore` command-line tools. These must be:
1. **In system PATH**, OR
2. **Specified with full paths** in BackupOptions
**To install:**
```bash
# Linux (Debian/Ubuntu)
sudo apt-get install postgresql-client
# Linux (Red Hat/CentOS)
sudo yum install postgresql
# Windows
# Download from https://www.postgresql.org/download/windows/
# Select "Command Line Tools" during installation
```
**To verify:**
```bash
# Check if in PATH
pg_dump --version
pg_restore --version
# Find location
which pg_dump # Linux/macOS
Get-Command pg_dump # Windows PowerShell
```
### Performance Indexes
After initial setup, run the performance indexes SQL script:
```bash
psql -U postgres -d your_database -f sql/add-performance-indexes.sql
```
This significantly improves query performance (3-15x faster).
### Backup Considerations
- **Local databases:** Fast backups, full backup support
- **Remote databases:** Slower (network transfer), but fully supported
- **Disable backups:** If using external backup solutions or cloud-managed PostgreSQL
- **Large databases:** Increase `TimeoutSeconds` and consider `directory` format with `ParallelJobs`
## Troubleshooting
### "pg_dump: command not found" or "pg_dump.exe not recognized"
**Solution:** Install PostgreSQL client tools or specify full path:
```xml
<PgDumpPath>/usr/lib/postgresql/16/bin/pg_dump</PgDumpPath>
```
### "connection to server failed"
**Solution:** Check:
- Network connectivity: `telnet host port`
- Firewall rules
- PostgreSQL `pg_hba.conf` allows connections from your IP
- Credentials are correct
### "Backup operation timed out"
**Solution:** Increase timeout:
```xml
<TimeoutSeconds>7200</TimeoutSeconds>
```
### "Query timeout"
**Solution:** Increase command timeout:
```xml
<CustomDatabaseOption>
<Key>command-timeout</Key>
<Value>180</Value>
</CustomDatabaseOption>
```
## See Also
- `docs/remote-postgresql-backup-support.md` - Detailed backup documentation
- `docs/increase-database-timeout.md` - Query timeout information
- `docs/COMPLETE-SESSION-SUMMARY.md` - All fixes and improvements
- `sql/add-performance-indexes.sql` - Performance optimization indexes
- `sql/monitor-query-performance.sql` - Query performance monitoring