Refactor SQLite Database Provider
- Removed unused classes and files related to SQLite database provider, including SqliteDesignTimeJellyfinDbFactory, ModelBuilderExtensions, PragmaConnectionInterceptor, AssemblyInfo, SqliteDatabaseProvider, DateTimeKindValueConverter. - Updated tests to remove dependencies on removed classes and adjusted mocking for configuration sections. - Added Microsoft.EntityFrameworkCore.Sqlite package reference to test project. - Improved string handling in tests for better consistency and clarity. - Refactored logging methods in JellyfinApplicationFactory for better readability and maintainability.
This commit is contained in:
@@ -0,0 +1,244 @@
|
||||
# EF Core Removal Analysis — pgsql-jellyfin Fork
|
||||
|
||||
**Date**: 2025
|
||||
**Scope**: Full codebase analysis of whether EF Core can and should be removed from this PostgreSQL-only fork.
|
||||
**Verdict**: **Do not remove EF Core.** Address specific friction points instead.
|
||||
|
||||
---
|
||||
|
||||
## 1. Usage Inventory
|
||||
|
||||
EF Core (`IDbContextFactory<JellyfinDbContext>`) is injected into **82 call sites across ~35 source files**, permeating every layer:
|
||||
|
||||
| Layer | Files |
|
||||
|---|---|
|
||||
| API | `DatabaseViewsController` |
|
||||
| Repository (Item) | `BaseItemRepository`, `PeopleRepository`, `MediaStreamRepository`, `MediaAttachmentRepository`, `KeyframeRepository`, `ChapterRepository` |
|
||||
| Repository (Other) | `ActivityManager`, `DeviceManager`, `DisplayPreferencesManager`, `UserManager`, `TrickplayManager`, `MediaSegmentManager`, `AuthenticationManager`, `AuthorizationContext`, `LibraryOptionsRepository`, `BackupService` |
|
||||
| Scheduled Tasks | `PeopleValidationTask`, `CleanupUserDataTask`, `CleanDatabaseScheduledTask`, `UserDataManager` |
|
||||
| Migration Routines | 13 routines: `MigrateActivityLogDb`, `MigrateAuthenticationDb`, `MigrateDisplayPreferencesDb`, `MigrateKeyframeData`, `MigrateLibraryDb`, `MigrateLibraryUserData`, `MigrateRatingLevels`, `MoveExtractedFiles`, `RefreshCleanNames`, `RefreshInternalDateModified`, `FixDates`, `CleanMusicArtist`, (+ more) |
|
||||
| Infrastructure | `JellyfinMigrationService`, `PostgresDatabaseProvider`, `DbContextFactoryHealthCheck`, `Program.cs`, `ServiceCollectionExtensions` |
|
||||
|
||||
The `JellyfinDbContext` itself defines **~35 DbSets** spanning 5 schemas (`activitylog`, `authentication`, `displaypreferences`, `library`, `users`).
|
||||
|
||||
---
|
||||
|
||||
## 2. Query Complexity — What EF Core Actually Does
|
||||
|
||||
### 2.1 BaseItemRepository.cs — The Core Case
|
||||
|
||||
`BaseItemRepository.cs` is **3,583 lines / 152 KB**, the largest file in the repository. It is the performance-critical code path for all library browsing, search, and playback-related queries. It is built entirely on EF Core LINQ.
|
||||
|
||||
**Key EF-dependent sub-systems inside it:**
|
||||
|
||||
| Method | EF Feature Used | Purpose |
|
||||
|---|---|---|
|
||||
| `TranslateQuery()` (lines 2548+, ~500 lines) | LINQ predicate composition over `IQueryable<BaseItemEntity>` | Translates `InternalItemsQuery` (60+ filter parameters) to SQL WHERE clauses |
|
||||
| `ApplyOrder()` (~80 lines) | `OrderBy` / `ThenBy` over EF expression trees, `OrderMapper.MapOrderByField()` | Translates `ItemSortBy` enum to typed SQL ORDER BY |
|
||||
| `ApplyNavigations()` | Conditional `Include()` / `ThenInclude()` | Eagerly loads TvExtras, AudioExtras, LiveTvExtras, Provider, Images, UserData per DTO field set |
|
||||
| `ApplyGroupingFilter()` | `DistinctBy()` → PostgreSQL `DISTINCT ON` | Deduplication for presentation unique keys |
|
||||
| `GetNextUpSeriesKeys()` | `Join()` + `GroupBy()` + `Max()` | Complex TV "next up" episode query |
|
||||
| `GetItemValues()` | Nested `Join()` + `SelectMany()` + correlated subquery pattern | Artist/Genre/Studio item-value lookups |
|
||||
| `UpdateOrInsertItemsAsync()` | `CreateExecutionStrategy().ExecuteAsync()`, `SaveChangesAsync()`, `Attach().State = EntityState.Modified`, `AddRange()` | Transactional batch upsert |
|
||||
| Entity state management | `context.Entry(placeholder).State = EntityState.Detached` | Upsert conflict resolution |
|
||||
| `EF.Constant()` | PostgreSQL-specific plan hint | Prevents parameterization of the placeholder UUID to allow index scans |
|
||||
|
||||
**LINQ operator counts in BaseItemRepository.cs** (approximate from code review):
|
||||
- `Where()`, `Select()`, `Join()`, `GroupBy()`, `Include()`, `ThenInclude()`, `OrderBy()`: **200+ call sites**
|
||||
- `ExecuteDelete()` / `ExecuteUpdate()` bulk ops: **~25 call sites**
|
||||
- `FromSqlRaw` / `FromSqlInterpolated`: **0** — reads are pure LINQ
|
||||
- `ExecuteSqlAsync()` raw SQL upserts: **~8** (ON CONFLICT upserts for extension tables)
|
||||
|
||||
The LINQ query composition is not superficial mapping sugar — `TranslateQuery` conditionally applies 30+ predicate branches (user data filters, rating filters, type filters, ancestor traversal, search term scoring, tag inheritance, etc.) based on `InternalItemsQuery` flags. This is the equivalent of a 500-line dynamic SQL builder, expressed in type-safe C#.
|
||||
|
||||
### 2.2 Other Repositories
|
||||
|
||||
The other repositories (`ActivityManager`, `UserManager`, `DeviceManager`, etc.) use EF more simply — primarily `Where()` + `Select()` + `SaveChanges()` patterns — but collectively represent hundreds more LINQ call sites.
|
||||
|
||||
### 2.3 Hybrid Usage Already Present
|
||||
|
||||
`BaseItemRepository.UpdateOrInsertItemsAsync()` already uses a hybrid approach: EF Core for reads and change-tracking, **raw `ExecuteSqlAsync()` for ON CONFLICT upserts**. This tells us the team is already reaching EF's limits on writes.
|
||||
|
||||
---
|
||||
|
||||
## 3. Migration Infrastructure
|
||||
|
||||
### 3.1 EF Schema Migrations Are DISABLED
|
||||
|
||||
Per `JellyfinMigrationService.cs` (line ~275):
|
||||
```csharp
|
||||
// DISABLED for .NET 11 preview: EF Core database migrations don't work with preview SDK
|
||||
// Database schema is initialized from SQL scripts in PostgresDatabaseProvider instead
|
||||
/*
|
||||
if (stage is JellyfinMigrationStageTypes.CoreInitialisation)
|
||||
{
|
||||
pendingDatabaseMigrations = migrationsAssembly.Migrations...
|
||||
}
|
||||
*/
|
||||
```
|
||||
|
||||
**EF's primary justification — schema migration management — is already bypassed.** Schema is initialized via `sql/schema_init/create_database_schema.sql` + `psql`.
|
||||
|
||||
### 3.2 EF Is Still Used for Code Migration Tracking
|
||||
|
||||
`JellyfinMigrationService` still uses EF to track *which code routines have run* via `__EFMigrationsHistory`:
|
||||
|
||||
```csharp
|
||||
var historyRepository = dbContext.GetService<IHistoryRepository>();
|
||||
await historyRepository.CreateIfNotExistsAsync();
|
||||
var appliedMigrations = await dbContext.Database.GetAppliedMigrationsAsync();
|
||||
await dbContext.Database.ExecuteSqlRawAsync(
|
||||
historyRepository.GetInsertScript(new HistoryRow(migrationId, version)));
|
||||
```
|
||||
|
||||
This usage of `IHistoryRepository` is what required the `SnakeCaseHistoryRepository` workaround (custom class overriding `MigrationIdColumnName` and `ProductVersionColumnName`, using `#pragma warning disable EF1001` because the base class is internal EF infrastructure).
|
||||
|
||||
---
|
||||
|
||||
## 4. Known Friction Points
|
||||
|
||||
These are the specific problems that prompted this analysis:
|
||||
|
||||
### 4.1 Snake_case Naming Impedance (FIXED, but fragile)
|
||||
|
||||
EF Core defaults to PascalCase. This fork uses snake_case PostgreSQL conventions. Three workarounds are required:
|
||||
|
||||
1. **`SnakeCaseNamingConvention`** — `IPropertyAddedConvention` that converts all property names to snake_case at model build time
|
||||
2. **`SnakeCaseHistoryRepository`** — custom `NpgsqlHistoryRepository` subclass overriding column name properties; requires `#pragma warning disable EF1001` (breaking internal EF API)
|
||||
3. **Manual table names in model configurations** — 31 `IEntityTypeConfiguration` files; two (`LibraryOptionsEntityConfiguration`, `HomeSectionConfiguration`) had PascalCase table names that overrode the convention and caused `relation "library.LibraryOptions" does not exist` errors at runtime
|
||||
|
||||
**Risk**: Every future upstream EF Core upgrade may break `SnakeCaseHistoryRepository` since it depends on internal EF infrastructure.
|
||||
|
||||
### 4.2 Execution Strategy Transaction Friction (FIXED, but imposed on all writers)
|
||||
|
||||
`EnableRetryOnFailure` in the PostgreSQL retry strategy makes EF Core incompatible with user-initiated `BeginTransaction()`. All transactional code must wrap in `context.Database.CreateExecutionStrategy().Execute(...)`. This has already been fixed in `MigrateRatingLevels` and `MigrateKeyframeData` but is an ongoing constraint.
|
||||
|
||||
### 4.3 EF-PostgreSQL Translation Gaps
|
||||
|
||||
Multiple code comments document EF Core translation failures:
|
||||
|
||||
- `DistinctBy()` on complex types sometimes cannot be translated → fallback to `ApplyGroupingInMemory()` using **reflection on anonymous types**
|
||||
- `GetItemValues()` in some paths falls back to `.AsEnumerable()` for in-memory filtering
|
||||
- `GetLatestItemList()` had to be split into two queries to avoid untranslatable nested `Min()` subqueries
|
||||
- A documented 165s query regression was caused by EF generating correlated subqueries — fixed by switching to `DistinctBy()` → `DISTINCT ON`
|
||||
- `AsSingleQuery()` required explicitly to prevent N+1 for multi-table includes
|
||||
|
||||
### 4.4 Inconsistent Table Naming in Raw SQL
|
||||
|
||||
`UpdateOrInsertItemsAsync()` uses raw `ExecuteSqlAsync()` with PascalCase quoted names:
|
||||
|
||||
```csharp
|
||||
$@"INSERT INTO library.""BaseItemProviders"" (""ItemId"", ""ProviderId"", ""ProviderValue"")..."
|
||||
$@"INSERT INTO library.""BaseItemTvExtras"" (""ItemId"", ..."
|
||||
```
|
||||
|
||||
These PascalCase quoted names (`"BaseItemProviders"`, `"BaseItemTvExtras"`) are inconsistent with the snake_case goal of the fork. The actual table names in PostgreSQL must match these quoted strings exactly — implying the schema init SQL creates them with PascalCase quoted names, not the snake_case convention applied to EF properties.
|
||||
|
||||
### 4.5 Known Performance Anti-Pattern in `GetItemValues`
|
||||
|
||||
```csharp
|
||||
// TODO: This is bad refactor!
|
||||
itemCount = new ItemCounts() {
|
||||
SeriesCount = itemCountQuery!.Count(f => f.Type == seriesTypeName),
|
||||
EpisodeCount = itemCountQuery!.Count(f => f.Type == episodeTypeName),
|
||||
// ... 5 more COUNT queries
|
||||
}
|
||||
```
|
||||
|
||||
This executes 7 separate `COUNT` queries per item in a result set — a classic N+1 problem embedded in an EF projection. This is an EF translation limitation (aggregate counts in a nested `SELECT` project).
|
||||
|
||||
---
|
||||
|
||||
## 5. Can EF Core Be Removed?
|
||||
|
||||
**Technically: Yes. Practically: The cost is prohibitive.**
|
||||
|
||||
A full removal would require:
|
||||
|
||||
1. **Rewriting `BaseItemRepository.cs`** (3,583 lines) as raw SQL — either Dapper or direct `NpgsqlCommand`. The `TranslateQuery` method alone is a ~500-line dynamic query builder that would need to be rewritten as a SQL string builder. All `Include()` chains would become explicit JOINs. All `ExecuteUpdate`/`ExecuteDelete` calls would become parameterized SQL.
|
||||
|
||||
2. **Replacing 31 ModelConfiguration files** with SQL schema definitions (partially done via `sql/schema_init/`).
|
||||
|
||||
3. **Replacing the migration tracking system** (`IHistoryRepository`) with a simple SQL table + custom tracker.
|
||||
|
||||
4. **Rewriting all other repositories** (~15 files) as Dapper or raw Npgsql.
|
||||
|
||||
5. **Replacing all `IDbContextFactory` injection points** (~35 files, 82 sites) with new connection/command factories.
|
||||
|
||||
**Estimated effort: 3–6 person-months of careful rewrite, plus extensive testing coverage.**
|
||||
|
||||
Alternatives assessed:
|
||||
|
||||
| Alternative | Pros | Cons |
|
||||
|---|---|---|
|
||||
| **Full removal (Dapper + Npgsql)** | Full control of SQL; no EF friction; better PostgreSQL-native features | 3-6 month rewrite; no dynamic LINQ query composition; lose `ExecuteUpdate`/`ExecuteDelete`; high regression risk |
|
||||
| **Hybrid (EF reads, Dapper writes)** | Eliminates write friction; keeps LINQ query composition | Two persistence paradigms; still need EF for reads; does not eliminate snake_case friction |
|
||||
| **Keep EF, address friction points** | Minimal disruption; preserves LINQ query composition | Ongoing EF+PostgreSQL compatibility work; preview SDK risk |
|
||||
| **Replace only migration tracking** | Eliminates `SnakeCaseHistoryRepository` hack | Small gain; does not address execution strategy or translation friction |
|
||||
|
||||
---
|
||||
|
||||
## 6. Should EF Core Be Removed?
|
||||
|
||||
**No — but specific friction points should be addressed deliberately.**
|
||||
|
||||
### 6.1 Why EF Core Should Stay
|
||||
|
||||
**The LINQ query composition in `TranslateQuery` / `ApplyOrder` is EF Core's core contribution to this codebase.** It provides:
|
||||
|
||||
- **Type safety** across 60+ filter parameters in `InternalItemsQuery`
|
||||
- **Composability** — predicates are built incrementally, each branch independently testable
|
||||
- **Maintainability** — 500 lines of C# expression trees is significantly more maintainable than equivalent raw SQL string building
|
||||
- **`ExecuteUpdate`/`ExecuteDelete`** — these EF 7+ bulk operations map directly to `UPDATE ... WHERE` and `DELETE ... WHERE` without loading entities; replacing these with raw SQL would produce functionally identical code with more boilerplate
|
||||
- **Navigation property loading** (`Include`) — the conditional navigation loading in `ApplyNavigations` maps cleanly to the DTO field selection pattern used by the API layer
|
||||
|
||||
EF Core's value in this codebase is almost entirely in the **read path** (LINQ-to-SQL translation) and **bulk operations** (`ExecuteUpdate`/`ExecuteDelete`). Its weaknesses are in the write path and infrastructure (migration history, naming conventions, execution strategy).
|
||||
|
||||
### 6.2 Recommended Targeted Actions
|
||||
|
||||
In priority order:
|
||||
|
||||
#### Priority 1 — Replace Migration History Tracking (Low effort, high safety gain)
|
||||
|
||||
The `SnakeCaseHistoryRepository` workaround depends on EF internal APIs (`#pragma warning disable EF1001`). Replace the code migration tracking with a simple dedicated table and Dapper/Npgsql:
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS public.jellyfin_applied_migrations (
|
||||
migration_id VARCHAR(150) PRIMARY KEY,
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
product_version VARCHAR(32) NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
Remove `JellyfinMigrationService`'s dependency on `IHistoryRepository` and `GetAppliedMigrationsAsync()`. This eliminates the most fragile EF workaround.
|
||||
|
||||
#### Priority 2 — Fix Raw SQL Table Names to Match snake_case Schema (Low effort, correctness)
|
||||
|
||||
`UpdateOrInsertItemsAsync` uses `library."BaseItemProviders"`, `library."BaseItemTvExtras"`, etc. If the fork's goal is snake_case naming, verify whether these tables exist as quoted PascalCase or as snake_case in PostgreSQL, and make the raw SQL consistent. Quoted PascalCase in PostgreSQL is case-sensitive and is a footgun.
|
||||
|
||||
#### Priority 3 — Replace Write Path with Raw SQL/Dapper (Medium effort, eliminates execution strategy friction)
|
||||
|
||||
The `UpdateOrInsertItemsAsync` method is already a hybrid (raw SQL for ON CONFLICT upserts + EF for the surrounding logic). Converting the remaining EF write operations (`SaveChangesAsync`, `Attach().State = EntityState.Modified`, `AddRange()`) in this method to direct Npgsql/Dapper calls would:
|
||||
|
||||
- Eliminate the `CreateExecutionStrategy().Execute()` wrappers
|
||||
- Enable direct PostgreSQL `ON CONFLICT DO UPDATE` throughout
|
||||
- Remove the impedance between EF change tracking and PostgreSQL's native upsert semantics
|
||||
|
||||
**Do not extend this to reads** — the LINQ query composition is worth keeping.
|
||||
|
||||
#### Priority 4 — Fix the N+1 in `GetItemValues` (Medium effort, performance)
|
||||
|
||||
The 7 separate `COUNT` queries per item should be replaced with a single `GROUP BY type, COUNT(*)` query executed once and pivoted in memory. This is a correctness/performance issue independent of whether EF is removed.
|
||||
|
||||
---
|
||||
|
||||
## 7. Summary
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| **Can EF Core be removed?** | Yes, technically. Estimated 3–6 person-months of work. |
|
||||
| **Should EF Core be removed?** | No. The LINQ query composition in `BaseItemRepository` is EF's core value and cost-prohibitive to replace. |
|
||||
| **Is EF Core being used optimally?** | No. The write path, migration tracking, and naming convention workarounds are sources of ongoing friction. |
|
||||
| **What should be done instead?** | Replace migration tracking with a native SQL table; convert the write path in `BaseItemRepository` to raw SQL; keep EF for the LINQ read path. |
|
||||
| **Biggest ongoing risk?** | `SnakeCaseHistoryRepository` uses internal EF APIs (`EF1001`) that may break on EF Core version bumps. Replacing migration tracking eliminates this. |
|
||||
| **Second biggest risk?** | EF Core + Npgsql preview packages (`11.0.0-preview.1`) on `net11.0`. Schema migrations are already disabled because of this. Pin to stable as soon as net11.0 releases. |
|
||||
@@ -0,0 +1,535 @@
|
||||
# Jellyfin with PostgreSQL Support
|
||||
|
||||
This is a fork of Jellyfin that adds PostgreSQL database support for .NET 11 preview.
|
||||
|
||||
## Features
|
||||
|
||||
- ✅ PostgreSQL database provider implementation
|
||||
- ✅ Automatic database initialization from SQL scripts
|
||||
- ✅ Remote database support with connection string configuration
|
||||
- ✅ Database backup/restore via pg_dump/pg_restore
|
||||
- ✅ Migration system disabled in favor of SQL scripts
|
||||
- ✅ Support for both local and remote PostgreSQL servers
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
### FFmpeg Requirements
|
||||
|
||||
**jellyfin-ffmpeg is highly preferred and recommended** for use with Jellyfin Media Server. While standard FFmpeg can be used, the custom `jellyfin-ffmpeg` fork includes specifically targeted fixes, optimizations, and enhanced hardware acceleration support for media streaming that may not be available or prioritized in the upstream version.
|
||||
|
||||
#### Why jellyfin-ffmpeg is Preferred:
|
||||
|
||||
- **Improved Hardware Acceleration**: Better support for hardware acceleration (NVENC, QuickSync, VA-API) across a wider range of GPUs, reducing the risk of partial acceleration
|
||||
- **Reduced Playback Issues**: Specifically built to handle HDR/Dolby Vision content, HLS/fMP4/MPEG-TS streaming, and complex transcoding scenarios better than standard FFmpeg
|
||||
- **Optimized Performance**: Often includes more efficient software decoders (like dav1d for AV1)
|
||||
- **Pre-configured**: Automatically included with official Docker images, deb packages, and Windows installers
|
||||
|
||||
**Link**: [jellyfin-ffmpeg repository](https://github.com/jellyfin/jellyfin-ffmpeg)
|
||||
|
||||
#### Installation:
|
||||
|
||||
**Debian/Ubuntu** (using Jellyfin repository):
|
||||
```bash
|
||||
# Add Jellyfin repository (if not already added)
|
||||
sudo apt-get install -y software-properties-common
|
||||
sudo add-apt-repository universe
|
||||
|
||||
# Install jellyfin-ffmpeg
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y jellyfin-ffmpeg6
|
||||
```
|
||||
|
||||
**Manual Download**:
|
||||
Download from [jellyfin-ffmpeg releases](https://github.com/jellyfin/jellyfin-ffmpeg/releases) and configure the path in Jellyfin settings.
|
||||
|
||||
#### Using Standard FFmpeg (Not Recommended):
|
||||
|
||||
If using a custom or unofficial installation where jellyfin-ffmpeg is not available, standard FFmpeg might work but could lead to issues with specific transcoding tasks:
|
||||
|
||||
```bash
|
||||
# Debian/Ubuntu
|
||||
sudo apt-get install -y ffmpeg
|
||||
|
||||
# RHEL/Fedora
|
||||
sudo dnf install -y ffmpeg
|
||||
|
||||
# Arch Linux
|
||||
sudo pacman -S ffmpeg
|
||||
```
|
||||
|
||||
**Note**: You may need to manually set the FFmpeg path in Jellyfin's dashboard under **Dashboard → Playback → FFmpeg path**.
|
||||
|
||||
### System Dependencies (Linux)
|
||||
|
||||
Jellyfin requires certain system libraries to be installed:
|
||||
|
||||
#### Debian/Ubuntu-based systems:
|
||||
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y \
|
||||
libfontconfig1 \
|
||||
libfreetype6 \
|
||||
libssl3 \
|
||||
libicu-dev
|
||||
```
|
||||
|
||||
**Note**: See the [FFmpeg Requirements](#ffmpeg-requirements) section above for installing `jellyfin-ffmpeg`.
|
||||
|
||||
#### RHEL/CentOS/Fedora-based systems:
|
||||
|
||||
```bash
|
||||
sudo dnf install -y \
|
||||
fontconfig \
|
||||
freetype \
|
||||
openssl \
|
||||
libicu
|
||||
```
|
||||
|
||||
**Note**: See the [FFmpeg Requirements](#ffmpeg-requirements) section above for installing FFmpeg.
|
||||
|
||||
#### Arch Linux:
|
||||
|
||||
```bash
|
||||
sudo pacman -S fontconfig freetype2 openssl icu
|
||||
```
|
||||
|
||||
**Note**: See the [FFmpeg Requirements](#ffmpeg-requirements) section above for installing FFmpeg.
|
||||
|
||||
**Note**: The `libfontconfig1` library is **required** for SkiaSharp (image processing). Without it, Jellyfin will fail to start with:
|
||||
```
|
||||
libfontconfig.so.1: cannot open shared object file: No such file or directory
|
||||
```
|
||||
|
||||
### PostgreSQL Requirements
|
||||
|
||||
- PostgreSQL 12 or higher (tested with PostgreSQL 18.3)
|
||||
- `psql` client tools (for automatic schema initialization)
|
||||
- Network access to PostgreSQL server (if using remote database)
|
||||
|
||||
#### Install PostgreSQL client tools:
|
||||
|
||||
**Debian/Ubuntu**:
|
||||
```bash
|
||||
sudo apt-get install -y postgresql-client
|
||||
```
|
||||
|
||||
**RHEL/CentOS/Fedora**:
|
||||
```bash
|
||||
sudo dnf install -y postgresql
|
||||
```
|
||||
|
||||
**Verify psql is available**:
|
||||
```bash
|
||||
psql --version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Configuration
|
||||
|
||||
### Option 1: Using ConnectionString (Recommended)
|
||||
|
||||
Create or edit `config/database.xml` in your Jellyfin data directory:
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<DatabaseConfigurationOptions>
|
||||
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
|
||||
<LockingBehavior>NoLock</LockingBehavior>
|
||||
|
||||
<CustomProviderOptions>
|
||||
<PluginName>Jellyfin-PostgreSQL</PluginName>
|
||||
<PluginAssembly>Jellyfin.Database.Providers.Postgres</PluginAssembly>
|
||||
|
||||
<!-- Connection string with all parameters -->
|
||||
<ConnectionString>Host=192.168.1.100;Port=5432;Database=jellyfin;Username=jellyfin;Password=your_password_here</ConnectionString>
|
||||
</CustomProviderOptions>
|
||||
|
||||
<!-- Optional: Backup configuration (works for both local and remote databases) -->
|
||||
<BackupOptions>
|
||||
<PgDumpPath>pg_dump</PgDumpPath>
|
||||
<PgRestorePath>pg_restore</PgRestorePath>
|
||||
<BackupFormat>custom</BackupFormat>
|
||||
<IncludeBlobs>true</IncludeBlobs>
|
||||
<CompressionLevel>6</CompressionLevel>
|
||||
<TimeoutSeconds>1800</TimeoutSeconds>
|
||||
<VerboseOutput>true</VerboseOutput>
|
||||
</BackupOptions>
|
||||
</DatabaseConfigurationOptions>
|
||||
```
|
||||
|
||||
### Option 2: Using Individual Options
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<DatabaseConfigurationOptions>
|
||||
<DatabaseType>Jellyfin-PostgreSQL</DatabaseType>
|
||||
|
||||
<CustomProviderOptions>
|
||||
<PluginName>Jellyfin-PostgreSQL</PluginName>
|
||||
<PluginAssembly>Jellyfin.Database.Providers.Postgres</PluginAssembly>
|
||||
|
||||
<Options>
|
||||
<CustomDatabaseOption>
|
||||
<Key>Host</Key>
|
||||
<Value>192.168.1.100</Value>
|
||||
</CustomDatabaseOption>
|
||||
|
||||
<CustomDatabaseOption>
|
||||
<Key>Port</Key>
|
||||
<Value>5432</Value>
|
||||
</CustomDatabaseOption>
|
||||
|
||||
<CustomDatabaseOption>
|
||||
<Key>Database</Key>
|
||||
<Value>jellyfin</Value>
|
||||
</CustomDatabaseOption>
|
||||
|
||||
<CustomDatabaseOption>
|
||||
<Key>Username</Key>
|
||||
<Value>jellyfin</Value>
|
||||
</CustomDatabaseOption>
|
||||
|
||||
<CustomDatabaseOption>
|
||||
<Key>Password</Key>
|
||||
<Value>your_password_here</Value>
|
||||
</CustomDatabaseOption>
|
||||
</Options>
|
||||
</CustomProviderOptions>
|
||||
</DatabaseConfigurationOptions>
|
||||
```
|
||||
|
||||
### Option 3: Hybrid (ConnectionString + Overrides)
|
||||
|
||||
Individual options take precedence over ConnectionString:
|
||||
|
||||
```xml
|
||||
<CustomProviderOptions>
|
||||
<!-- Base connection -->
|
||||
<ConnectionString>Host=192.168.1.100;Port=5432;Database=jellyfin;Username=jellyfin</ConnectionString>
|
||||
|
||||
<!-- Override just the password -->
|
||||
<Options>
|
||||
<CustomDatabaseOption>
|
||||
<Key>Password</Key>
|
||||
<Value>production_password</Value>
|
||||
</CustomDatabaseOption>
|
||||
</Options>
|
||||
</CustomProviderOptions>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Initialization
|
||||
|
||||
### Automatic Initialization
|
||||
|
||||
When Jellyfin starts with an empty database:
|
||||
|
||||
1. **Database Check**: Checks if the database exists (creates if missing)
|
||||
2. **Table Check**: Checks if `users.Users` table exists
|
||||
3. **Auto-Initialize**: If tables don't exist, automatically runs `sql/schema_init/create_database_schema.sql` via `psql`
|
||||
4. **Schema Verification**: Verifies all required schemas and tables are present
|
||||
|
||||
**Requirements for automatic initialization**:
|
||||
- ✅ `psql` command must be in system PATH
|
||||
- ✅ Database must exist (created automatically or manually)
|
||||
- ✅ SQL script must be deployed: `sql/schema_init/create_database_schema.sql`
|
||||
|
||||
### Manual Initialization
|
||||
|
||||
If `psql` is not available or automatic initialization fails:
|
||||
|
||||
```bash
|
||||
# Create database first
|
||||
createdb -U postgres jellyfin
|
||||
|
||||
# Run schema creation script
|
||||
psql -U postgres -d jellyfin -f /path/to/jellyfin/sql/schema_init/create_database_schema.sql
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Startup Logs
|
||||
|
||||
### Successful Startup (Empty Database):
|
||||
|
||||
```
|
||||
[INF] PostgreSQL connection: Host="192.168.1.100", Port=5432, Database="jellyfin", ...
|
||||
[INF] Database tables not found (users.Users table missing). Initializing from SQL script...
|
||||
[INF] Found schema script at: .../sql/schema_init/create_database_schema.sql
|
||||
[INF] Executing create_database_schema.sql (this may take several minutes)...
|
||||
[INF] Executing via psql: psql -h 192.168.1.100 -p 5432 -U jellyfin -d jellyfin -f "..."
|
||||
[INF] ✅ Successfully initialized database from SQL script via psql
|
||||
[INF] Database is ready. You can now start Jellyfin.
|
||||
[DBG] Schema 'users' contains 4 tables
|
||||
[DBG] Schema 'library' contains 45 tables
|
||||
[DBG] Schema 'activitylog' contains 1 tables
|
||||
[INF] ✅ Database schema verification complete
|
||||
[INF] There are 0 migrations for stage CoreInitialisation
|
||||
[INF] There are 24 migrations for stage AppInitialisation
|
||||
```
|
||||
|
||||
### Successful Startup (Existing Database):
|
||||
|
||||
```
|
||||
[INF] PostgreSQL connection: Host="192.168.1.100", Port=5432, Database="jellyfin", ...
|
||||
[INF] Database tables exist (users.Users table found)
|
||||
[DBG] Schema 'users' contains 4 tables
|
||||
[DBG] Schema 'library' contains 45 tables
|
||||
[INF] ✅ Database schema verification complete
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Error: `libfontconfig.so.1: cannot open shared object file`
|
||||
|
||||
**Cause**: Missing system dependency
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Debian/Ubuntu
|
||||
sudo apt-get install -y libfontconfig1
|
||||
|
||||
# RHEL/Fedora
|
||||
sudo dnf install -y fontconfig
|
||||
```
|
||||
|
||||
### Error: `psql command not found`
|
||||
|
||||
**Cause**: PostgreSQL client tools not installed
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Debian/Ubuntu
|
||||
sudo apt-get install -y postgresql-client
|
||||
|
||||
# RHEL/Fedora
|
||||
sudo dnf install -y postgresql
|
||||
```
|
||||
|
||||
### Error: `Failed to connect to 127.0.0.1:5432`
|
||||
|
||||
**Cause**: Configuration not loading correctly (using default localhost)
|
||||
|
||||
**Solution**:
|
||||
- Verify `database.xml` is in the correct location (`/var/lib/jellyfin/config/database.xml` on Linux)
|
||||
- Check that ConnectionString or individual Options are properly configured
|
||||
- Ensure Host is set to your remote server IP/hostname
|
||||
|
||||
### Error: `database "jellyfin" does not exist`
|
||||
|
||||
**Cause**: Database wasn't created automatically (rare)
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Create database manually
|
||||
createdb -U postgres -h your-server jellyfin
|
||||
|
||||
# Grant privileges
|
||||
psql -U postgres -h your-server -c "GRANT ALL PRIVILEGES ON DATABASE jellyfin TO jellyfin;"
|
||||
|
||||
# Restart Jellyfin (it will create tables automatically)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Database Schema
|
||||
|
||||
- `activitylog` - Activity log entries
|
||||
- `authentication` - API keys, devices, device options
|
||||
- `displaypreferences` - User display preferences
|
||||
- `library` - Media library data (BaseItems, metadata, etc.)
|
||||
- `users` - User accounts, permissions, preferences
|
||||
|
||||
### Migration System
|
||||
|
||||
**Entity Framework migrations are DISABLED** for .NET 11 preview. Schema changes are managed via SQL scripts:
|
||||
|
||||
- **Core migrations**: Disabled (no database structure changes via EF)
|
||||
- **Application migrations**: Enabled (configuration and data migrations only)
|
||||
|
||||
**Why**: EF migrations in preview builds can cause schema inconsistencies. SQL scripts provide reliable schema management.
|
||||
|
||||
---
|
||||
|
||||
## Backup and Restore
|
||||
|
||||
### Automatic Backups (via pg_dump)
|
||||
|
||||
Configured in `database.xml`:
|
||||
|
||||
```xml
|
||||
<BackupOptions>
|
||||
<PgDumpPath>pg_dump</PgDumpPath>
|
||||
<BackupFormat>custom</BackupFormat>
|
||||
<CompressionLevel>6</CompressionLevel>
|
||||
</BackupOptions>
|
||||
```
|
||||
|
||||
**Works for both local and remote databases!**
|
||||
|
||||
### Manual Backup
|
||||
|
||||
```bash
|
||||
# Backup to custom format (compressed)
|
||||
pg_dump -h your-server -U jellyfin -d jellyfin -Fc -f jellyfin_backup.dump
|
||||
|
||||
# Backup to SQL format
|
||||
pg_dump -h your-server -U jellyfin -d jellyfin -f jellyfin_backup.sql
|
||||
```
|
||||
|
||||
### Manual Restore
|
||||
|
||||
```bash
|
||||
# Restore from custom format
|
||||
pg_restore -h your-server -U jellyfin -d jellyfin jellyfin_backup.dump
|
||||
|
||||
# Restore from SQL format
|
||||
psql -h your-server -U jellyfin -d jellyfin -f jellyfin_backup.sql
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting Connection Issues
|
||||
|
||||
### Error: `57P01: terminating connection due to administrator command`
|
||||
|
||||
**Cause**: PostgreSQL server forcibly terminated the connection during an operation
|
||||
|
||||
**Common reasons**:
|
||||
- PostgreSQL server restart or reload
|
||||
- Connection timeout (statement_timeout, idle_in_transaction_session_timeout)
|
||||
- Manual connection termination via `pg_terminate_backend()`
|
||||
- Network instability or firewall issues
|
||||
- Resource limits exceeded (max_connections, memory pressure)
|
||||
|
||||
**Solutions**:
|
||||
|
||||
#### 1. Configure Automatic Retry (Already Enabled)
|
||||
|
||||
Jellyfin now includes automatic retry logic for transient failures:
|
||||
- **Max retries**: 3 attempts
|
||||
- **Max delay**: 5 seconds between retries
|
||||
- **Retryable errors**: Includes `57P01` (connection termination)
|
||||
|
||||
#### 2. Increase PostgreSQL Timeouts
|
||||
|
||||
Edit PostgreSQL configuration (`/etc/postgresql/*/main/postgresql.conf`):
|
||||
|
||||
```
|
||||
# Increase statement timeout
|
||||
statement_timeout = 300000 # 5 minutes (in milliseconds)
|
||||
|
||||
# Increase idle transaction timeout
|
||||
idle_in_transaction_session_timeout = 600000 # 10 minutes
|
||||
|
||||
# Enable TCP keepalives (prevent network timeouts)
|
||||
tcp_keepalives_idle = 60 # seconds
|
||||
tcp_keepalives_interval = 10 # seconds
|
||||
tcp_keepalives_count = 6
|
||||
```
|
||||
|
||||
Reload PostgreSQL:
|
||||
```bash
|
||||
sudo systemctl reload postgresql
|
||||
# OR
|
||||
psql -U postgres -c "SELECT pg_reload_conf();"
|
||||
```
|
||||
|
||||
#### 3. Optimize Connection String
|
||||
|
||||
Add keepalive and timeout parameters to your connection string:
|
||||
|
||||
```xml
|
||||
<ConnectionString>Host=192.168.129.253;Port=5432;Database=jellyfin;Username=jellyfin;Password=yourpass;Pooling=True;Minimum Pool Size=0;Maximum Pool Size=50;Connection Idle Lifetime=300;Connection Pruning Interval=10;Timeout=30;Command Timeout=300;Keepalive=60</ConnectionString>
|
||||
```
|
||||
|
||||
**Key parameters**:
|
||||
- `Timeout=30` - Connection establishment timeout (30 seconds)
|
||||
- `Command Timeout=300` - Query execution timeout (5 minutes)
|
||||
- `Keepalive=60` - TCP keepalive interval (60 seconds)
|
||||
- `Connection Idle Lifetime=300` - Close idle connections after 5 minutes
|
||||
- `Connection Pruning Interval=10` - Check for stale connections every 10 seconds
|
||||
|
||||
#### 4. Check PostgreSQL Logs
|
||||
|
||||
On your PostgreSQL server:
|
||||
|
||||
```bash
|
||||
# View recent logs
|
||||
sudo tail -f /var/log/postgresql/postgresql-*.log
|
||||
|
||||
# Or check systemd journal
|
||||
sudo journalctl -u postgresql -n 100 --no-pager
|
||||
```
|
||||
|
||||
Look for:
|
||||
- `received fast shutdown request`
|
||||
- `terminating connection`
|
||||
- `too many connections`
|
||||
- `out of memory`
|
||||
|
||||
#### 5. Monitor Active Connections
|
||||
|
||||
```sql
|
||||
-- Check current connections
|
||||
SELECT
|
||||
datname,
|
||||
usename,
|
||||
application_name,
|
||||
state,
|
||||
state_change
|
||||
FROM pg_stat_activity
|
||||
WHERE datname = 'jellyfin'
|
||||
ORDER BY state_change DESC;
|
||||
|
||||
-- Check connection limits
|
||||
SELECT
|
||||
(SELECT setting::int FROM pg_settings WHERE name = 'max_connections') AS max_connections,
|
||||
(SELECT COUNT(*) FROM pg_stat_activity) AS current_connections,
|
||||
(SELECT COUNT(*) FROM pg_stat_activity WHERE datname = 'jellyfin') AS jellyfin_connections;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Building from Source
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/yourusername/jellyfin.git
|
||||
cd jellyfin
|
||||
|
||||
# Checkout the upgrade branch
|
||||
git checkout upgrade-to-NET11
|
||||
|
||||
# Build
|
||||
dotnet build
|
||||
|
||||
# Run
|
||||
cd Jellyfin.Server
|
||||
dotnet run
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
This is a development/preview branch. For production use, please use the official Jellyfin releases.
|
||||
|
||||
## License
|
||||
|
||||
GNU General Public License v2.0 (same as Jellyfin)
|
||||
|
||||
---
|
||||
|
||||
## Credits
|
||||
|
||||
Based on [Jellyfin](https://github.com/jellyfin/jellyfin) with PostgreSQL support and .NET 11 compatibility.
|
||||
@@ -0,0 +1,167 @@
|
||||
# WebSocket Authentication Guide
|
||||
|
||||
## Overview
|
||||
|
||||
WebSocket connections to Jellyfin servers require authentication. This guide explains how to properly authenticate WebSocket connections using API tokens.
|
||||
|
||||
## Authentication Methods
|
||||
|
||||
### 1. Query String Parameter (Recommended for WebSocket)
|
||||
|
||||
The simplest and most compatible method for WebSocket connections.
|
||||
|
||||
**URL Format:**
|
||||
```
|
||||
ws://jellyfin-server:8096/socket?api_key=YOUR_API_TOKEN
|
||||
wss://jellyfin-server:8096/socket?api_key=YOUR_API_TOKEN (for HTTPS)
|
||||
```
|
||||
|
||||
**JavaScript Example:**
|
||||
```javascript
|
||||
const token = "YOUR_API_KEY";
|
||||
const ws = new WebSocket(`ws://jellyfin-server:8096/socket?api_key=${token}`);
|
||||
|
||||
ws.onopen = function(event) {
|
||||
console.log("WebSocket connection established with authentication");
|
||||
};
|
||||
|
||||
ws.onerror = function(event) {
|
||||
console.error("WebSocket error:", event);
|
||||
};
|
||||
|
||||
ws.onmessage = function(event) {
|
||||
const message = JSON.parse(event.data);
|
||||
console.log("Received message:", message);
|
||||
};
|
||||
|
||||
ws.onclose = function(event) {
|
||||
console.log("WebSocket connection closed");
|
||||
};
|
||||
```
|
||||
|
||||
**Python Example:**
|
||||
```python
|
||||
import asyncio
|
||||
import websockets
|
||||
import json
|
||||
|
||||
async def connect_with_token():
|
||||
token = "YOUR_API_KEY"
|
||||
uri = f"ws://jellyfin-server:8096/socket?api_key={token}"
|
||||
|
||||
async with websockets.connect(uri) as websocket:
|
||||
print("Connected to Jellyfin WebSocket")
|
||||
|
||||
# Receive messages
|
||||
async for message in websocket:
|
||||
data = json.loads(message)
|
||||
print(f"Received: {data}")
|
||||
|
||||
asyncio.run(connect_with_token())
|
||||
```
|
||||
|
||||
**C# Example:**
|
||||
```csharp
|
||||
using System;
|
||||
using System.Net.WebSockets;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
public class JellyfinWebSocketClient
|
||||
{
|
||||
public async Task ConnectAsync(string serverUrl, string token)
|
||||
{
|
||||
var uri = new Uri($"ws://{serverUrl}:8096/socket?api_key={token}");
|
||||
|
||||
using (var client = new ClientWebSocket())
|
||||
{
|
||||
await client.ConnectAsync(uri, CancellationToken.None);
|
||||
|
||||
Console.WriteLine("Connected to Jellyfin WebSocket");
|
||||
|
||||
// Receive messages
|
||||
var buffer = new byte[1024 * 4];
|
||||
while (client.State == WebSocketState.Open)
|
||||
{
|
||||
var result = await client.ReceiveAsync(
|
||||
new ArraySegment<byte>(buffer),
|
||||
CancellationToken.None);
|
||||
|
||||
if (result.MessageType == WebSocketMessageType.Text)
|
||||
{
|
||||
var message = System.Text.Encoding.UTF8.GetString(
|
||||
buffer, 0, result.Count);
|
||||
Console.WriteLine($"Received: {message}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Authorization Header (Alternative)
|
||||
|
||||
For advanced use cases, you can also use the Authorization header.
|
||||
|
||||
**Header Format:**
|
||||
```
|
||||
Authorization: MediaBrowser Device="ClientName", DeviceId="unique-id", Version="1.0", Token="YOUR_API_TOKEN"
|
||||
```
|
||||
|
||||
**Note:** Some WebSocket implementations may not support custom headers during the upgrade handshake. Query parameters are recommended.
|
||||
|
||||
## Obtaining an API Token
|
||||
|
||||
### Via Server UI
|
||||
1. Navigate to your Jellyfin server dashboard
|
||||
2. Go to Settings → API Keys (or similar, depending on version)
|
||||
3. Create a new API key
|
||||
4. Copy the token to use in your WebSocket connection
|
||||
|
||||
### Programmatically
|
||||
Use the REST API to create API keys:
|
||||
```bash
|
||||
curl -X POST "http://jellyfin-server:8096/Auth/Keys" \
|
||||
-H "Authorization: MediaBrowser Token=existing_token" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"AppName": "My WebSocket Client"}'
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Connection Refused / 401 Unauthorized
|
||||
- Verify the API token is correct
|
||||
- Ensure the token hasn't expired
|
||||
- Check that the WebSocket endpoint path is correct (`/socket`)
|
||||
|
||||
### Token Not Found
|
||||
- Verify the query parameter is URL-encoded properly
|
||||
- Ensure the parameter name is correct: `api_key` (lowercase)
|
||||
- Check server logs for authentication errors
|
||||
|
||||
### WebSocket Connection Fails Immediately
|
||||
- Confirm the server is reachable
|
||||
- Check firewall rules allow WebSocket connections
|
||||
- Try with `wss://` (secure WebSocket) if using HTTPS
|
||||
|
||||
## Server Configuration
|
||||
|
||||
The server automatically extracts tokens from:
|
||||
1. Authorization header (MediaBrowser Token parameter)
|
||||
2. Query string `api_key` parameter
|
||||
3. Query string `ApiKey` parameter
|
||||
4. Legacy headers (if enabled in config)
|
||||
|
||||
No special server configuration is required for WebSocket authentication to work.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Always use `wss://` (secure WebSocket) when connecting over untrusted networks
|
||||
- Keep API tokens secure and rotate them periodically
|
||||
- Use separate tokens for different clients/applications
|
||||
- Consider implementing token expiration in your server configuration
|
||||
|
||||
## See Also
|
||||
|
||||
- [Jellyfin API Documentation](https://api.jellyfin.org/)
|
||||
- [WebSocket Protocol (RFC 6455)](https://tools.ietf.org/html/rfc6455)
|
||||
Reference in New Issue
Block a user