Commit Graph

71 Commits

Author SHA1 Message Date
wjones dce22a558f fix: remove unnecessary blank line before error logging in PostgresDatabaseProvider 2026-07-08 12:00:50 -04:00
wjones 1abf61a05f Add PostgresSubprocessException and refactor database initialization logic
- Introduced PostgresSubprocessException to handle errors during PostgreSQL subprocess execution.
- Refactored PostgresDatabaseProvider to improve schema initialization script discovery.
- Enhanced error handling for psql command execution, including logging of output and errors.
- Implemented methods to find the schema initialization script and the psql executable path.
2026-07-08 11:53:36 -04:00
wjones 421a8cc092 fix: implement exponential backoff strategy for concurrency exceptions in SaveChangesAsync 2026-07-07 11:31:47 -04:00
wjones 9bcb8501ab Add scripts for PostgreSQL migration and SQLite removal; update project references
- Introduced `publish-with-sql.ps1` to publish Jellyfin with SQL files.
- Added `remove-sqlite.ps1` to facilitate the removal of SQLite dependencies for PostgreSQL-only deployment.
- Created `rollback-to-net10.ps1` to revert .NET 11 changes back to .NET 10.
- Implemented `test_api.py` for testing API interactions.
- Added `verify-migration.ps1` to verify PostgreSQL migration steps.
- Updated various `.csproj` files to include `Microsoft.Kiota.Abstractions` package.
- Enhanced `JellyfinDbContext` to handle concurrency exceptions during save operations.
- Updated tests to include new package references and ensure compatibility with changes.
2026-07-07 10:59:40 -04:00
wjones 4ccd84342b feat: implement migration tracking and history management for PostgreSQL 2026-05-04 12:47:12 -04:00
wjones 3e5d29225a Refactor SQLite Database Provider
- Removed unused classes and files related to SQLite database provider, including SqliteDesignTimeJellyfinDbFactory, ModelBuilderExtensions, PragmaConnectionInterceptor, AssemblyInfo, SqliteDatabaseProvider, DateTimeKindValueConverter.
- Updated tests to remove dependencies on removed classes and adjusted mocking for configuration sections.
- Added Microsoft.EntityFrameworkCore.Sqlite package reference to test project.
- Improved string handling in tests for better consistency and clarity.
- Refactored logging methods in JellyfinApplicationFactory for better readability and maintainability.
2026-05-03 09:39:00 -04:00
wjones e1f7a4bee9 Refactor database configuration and migrations for PostgreSQL support
- Updated default database configuration to use PostgreSQL instead of SQLite when no database.xml exists.
- Removed obsolete migration routines: MigrateUserDb, RemoveDuplicateExtras, ReseedFolderFlag.
- Introduced new read-only view projections for PostgreSQL: ItemPersonView, ItemProviderView, LibrarySummaryView, MediaStreamInfoView, UserPlaybackHistoryView, VideoItemView.
- Registered view mappings in PostgresDatabaseProvider to ensure proper handling of read-only views.
- Cleaned up unused SQLite cache size configuration in ConfigurationExtensions.

Co-authored-by: Copilot <copilot@github.com>
2026-04-30 18:47:48 -04:00
wjones 9fc10226c9 feat: implement migration to split MediaStreamInfos into AudioStreamDetails and VideoStreamDetails 2026-04-30 17:48:49 -04:00
wjones 57d6342524 Optimize PostgreSQL dead tuple handling for media scans
Introduce bulk operation extensions and transaction helpers for efficient, batched EF Core operations. Refactor MediaStreamInfo schema by splitting audio/video details into dedicated tables, reducing table bloat and improving update performance. Add migration for data movement and schema changes. Update EF Core models and context for new structure. Enhance PostgreSQL provider configuration and add comprehensive documentation and monitoring guides. Includes minor middleware and code cleanups.
2026-04-30 15:52:37 -04:00
wjones e81c127514 Add JSON config, DB-backed library options, and docs
- Add JSON-based config loading with XML fallback for DB and library options
- Implement LibraryOptionsRepository with EF Core, migrations, and entity
- Update CollectionFolder to use DB-backed options with XML fallback/backfill
- Register repository in DI and initialize at startup
- Use EF execution strategy for transactional DB operations
- Suppress code analysis warnings in .csproj and test files
- Add DATABASE_MIGRATION.md, LIBRARY_OPTIONS_DB_DESIGN.md, and WEBSOCKET_AUTHENTICATION.md
- Add database.json.example and improve migration docs
- Add tests for JSON config loader and update test naming warnings
2026-04-30 12:25:24 -04:00
wjones 786c61ba09 Fix 12 code analysis errors in MediaEncoding projects - use LoggerMessage source generators and specific exception types 2026-04-29 08:24:34 -04:00
wjones dd7c38fb5d Add EnableRetryOnFailure to handle transient connection failures (57P01 terminating connection) 2026-04-29 08:12:44 -04:00
wjones e6e7c49888 Fix EnsureDatabaseExistsAsync to respect ConnectionString from configuration - was still using localhost default 2026-03-10 13:47:50 -04:00
wjones 7deb7c1432 Fix SA1513 StyleCop error - add blank line after closing brace 2026-03-10 13:01:12 -04:00
wjones 077f10a4af Fix connection configuration - respect ConnectionString and individual options from database.xml 2026-03-10 12:55:49 -04:00
wjones bb9329736f Fix password masking error - password not in args anyway 2026-03-08 11:19:28 -04:00
wjones d4426fe0f6 Use psql subprocess to execute schema script - handles psql meta-commands 2026-03-08 11:13:39 -04:00
wjones 7657caa960 Fix database initialization detection - check for TABLES not just schemas 2026-03-08 11:07:16 -04:00
wjones 2209f0e1c0 COMPLETE REWRITE: Remove ALL migration code, implement pure SQL script initialization 2026-03-07 13:29:23 -05:00
wjones 75acefe989 Add automatic SQL script execution for empty database initialization 2026-03-07 13:07:44 -05:00
wjones 2d6e10e163 CRITICAL: Also disable EnsureCreatedAsync - it was bypassing migration tracking and creating schema 2026-03-07 13:01:33 -05:00
wjones 091114657b Disable EF Core migrations for .NET 11 preview - use manual SQL scripts instead 2026-03-07 12:52:05 -05:00
wjones f2ddde1747 Remove fallback code for database index creation 2026-03-07 11:52:55 -05:00
wjones 7eb2b445cb Postgres perf, path migration, and shutdown race fixes
- Major query performance boost: replaced correlated subqueries with DistinctBy() in BaseItemRepository, added episode deduplication index, and increased default command timeout to 120s.
- Fixed LibraryMonitor shutdown race: added disposal checks and exception handling in FileRefresher to prevent ObjectDisposedException.
- Added PowerShell and SQL utilities for fixing Linux paths in config files and database; new scripts for WebDir and path migration.
- Auto-fix for WebDir in startup.json on load; new helper scripts and docs.
- Added automated weekly DB performance monitoring scripts and reporting.
- Expanded documentation and README for all fixes, scripts, and migration steps.
- Updated SQL scripts for index creation and diagnostics; improved output handling.
- Updated db-config defaults and added new diagnostic/report files.
2026-03-05 08:56:42 -05:00
wjones 667c3768a9 Enhanced logging & error handling for auth and pg backups
- Add detailed debug/warning logs to authentication flows (AuthService, AuthorizationContext, WebSocketManager) for better traceability of auth attempts and failures.
- Log WebSocket authentication attempts, including IP, path, and token preview.
- Introduce PostgresVersionMismatchException for pg_dump/pg_restore version mismatches; parse and log server/client versions, clean up failed backups, and provide upgrade guidance.
- Add helper for parsing version mismatch errors.
- Update using directives and minor code cleanups.
- Update .csproj.user with new publish profile path.
2026-03-04 12:37:39 -05:00
wjones c76853a442 PostgreSQL: Production fixes—UPSERT, remote backup, auth logs
- Fix: Atomic UPSERT for BaseItemProviders (resolves duplicate key errors during concurrent metadata refresh; clears navigation property to prevent EF Core tracking conflicts)
- Add: Remote PostgreSQL backup support (removes localhost-only restriction; works with pg_dump/pg_restore for both local and remote DBs)
- Add: Configurable backup disable option (`disable-backups`)
- Fix: Query timeout and performance (documented `command-timeout` config, added performance index scripts)
- Fix: Authentication errors now log as warnings with clear messages (ExceptionMiddleware), reducing log noise
- Fix: SyncPlay authorization handler validates user before lookup, logs warnings for unauthenticated/unknown users (returns 403/404)
- Fix: Database deadlock detection logs warnings and allows EF Core auto-retry
- Add: Configurable LibraryMonitorDelay (min 30s, default 60s)
- Fix: SQLite migration filtering—skip SQLite-only migrations on PostgreSQL
- Chore: Suppress StyleCop warnings (SA1137, etc.) for project consistency
- Docs: 21 documentation files added/updated (config, backup, performance, troubleshooting, session summary)
- All changes are backward compatible and production-ready
2026-03-03 16:35:27 -05:00
wjones 23ed81b6b1 Improve build reliability, query perf, and documentation
- Add build-error-resolution.md guide for CS0006 and IDE analyzer issues
- Add rebuild-solution.ps1 script for clean/reliable builds
- Suppress noisy IDE/StyleCop warnings in .editorconfig
- Optimize BaseItemRepository grouping queries to use Min(Id) instead of FirstOrDefault, avoiding correlated subqueries and greatly improving DB performance
- Add database-query-optimization.md explaining query changes and tuning
- Enable EF Core SQL query logging, sensitive data, and detailed errors when log level is Debug (ServiceCollectionExtensions, logging.json)
- Update readme with SQL logging instructions
- Clarify git commit/ignore guidance and improve inline documentation
2026-03-01 09:49:30 -05:00
wjones 5565dc3e05 Add automated PostgreSQL performance index management
- Add scripts and SQL for base and supplementary indexes (total 23)
- Integrate index checks and auto-creation into Jellyfin startup
- Update csproj to include SQL files in build/publish output
- Add diagnostics, publish/copy scripts, and troubleshooting docs
- Provide detailed guides for migration, verification, and merging
- Ensures robust, automated, and well-documented index setup
2026-02-28 15:13:04 -05:00
wjones 33d7c010fb Jellyfin 11.0.0-PostgreSQL PREVIEW: version bump & defaults
- Update version to 11.0.0-PostgreSQL-PREVIEW in assemblies, installer, and build scripts
- Default web directory is now "wwwroot" across all platforms
- Configuration manager now creates .example files if missing
- Set PostgreSQL as default database with sensible options
- Postgres provider ensures database exists before table checks
- Add docs for marker conflict fixes and startup.json path issues
- Version string in logs/UI uses informational version
- Installer output filename and wizard show PREVIEW suffix
- Documentation summarizes version bump and verification steps
2026-02-27 13:45:09 -05:00
wjones d623cd03a9 Enable EF Core SplitQuery for PostgreSQL provider
Configured QuerySplittingBehavior.SplitQuery in PostgresDatabaseProvider to optimize performance for queries with multiple Include() statements. This prevents cartesian explosion, eliminates EF Core warnings, and aligns with best practices for handling related collections in PostgreSQL.
2026-02-26 17:51:11 -05:00
wjones 5bdb7d53c3 Platform config templates & Postgres migration recovery
- Added Linux/Windows startup config templates and README
- Updated default paths to FHS-compliant locations
- Improved publish profiles for platform targeting
- Enhanced Postgres migration: auto-recover on 42P07 errors
- Added RecordMigrationAsAppliedAsync for manual migration history
- Improved migration logging and error handling
- Added detailed documentation for config and migration fixes
2026-02-26 13:44:03 -05:00
wjones 24ae417a39 PostgreSQL migration: schema fixes, docs, and tooling
- Assign all tables to correct PostgreSQL schemas in EF Core migrations
- Fix identifier quoting: replace SQL Server brackets with PostgreSQL double quotes
- Use InsertData for placeholder BaseItems row (no raw SQL)
- Remove obsolete/Windows publish profiles; set Linux as default
- Add detailed migration success and verification reports (Markdown)
- Add idempotent, production-ready SQL migration scripts
- Add PowerShell script for automated migration verification
- Ensures robust, production-ready PostgreSQL migration and documentation
2026-02-26 12:16:47 -05:00
wjones 69c213f13c Auto-generate startup.json and add database presets
Introduces automatic creation of a self-documenting startup.json file on first startup, with example path configurations for Linux, Windows, and portable setups. Adds preset database configurations to database.xml for easy switching between SQLite and PostgreSQL. Refactors database config classes for XML serialization compatibility and ensures database.xml is never empty. Comprehensive documentation and migration guides included.
2026-02-25 16:29:08 -05:00
wjones c38adfc0f7 Add file-based startup config & temp dir option
Added support for configuring all major Jellyfin paths (data, config, cache, log, temp, web) via a `startup.json` file, with priority after CLI and environment variables. Introduced a new `TempDir` option, configurable through CLI, env var, or config file, with backward-compatible defaults. Refactored path resolution logic to integrate file-based config and ensure directory creation. Updated constructors and CLI options to support temp directory. Improved EF Core migration error handling and logging. Added comprehensive documentation and example config files. All changes are backward compatible.
2026-02-25 15:18:23 -05:00
wjones dad5cbadf5 Stop tracking and ignore unwanted files 2026-02-25 09:17:02 -05:00
wjones 135b6a81b9 Update NuGet paths and add web client directory docs
Added README section describing the web client directory (`wwwroot`) and its configuration options. No code logic changes; only environment and documentation updates.
2026-02-24 12:21:55 -05:00
wjones ce2a060bd3 clean solution 2026-02-23 18:36:36 -05:00
wjones 88dba54087 Add user-configurable PostgreSQL backup system
Introduced backup/restore via external pg_dump/pg_restore tools, configurable in database.xml. Added DatabaseBackupOptions, PostgresBackupService, and example/config docs. Backup features are enabled only for local databases; remote DBs require manual management. Logging and documentation clarify external tool usage, error handling, and security. Updated provider logic and assembly files accordingly.
2026-02-23 18:31:27 -05:00
wjones 43cdbf9b35 Refactor startup user creation and add HomeSection config
Refactored user initialization to always ensure at least one user exists at startup. If no users are present, a default admin user (admin/jellyfin) is created automatically. Updated StartupController endpoints to handle cases where no user exists. Simplified UserManager.InitializeAsync logic. Added HomeSectionConfiguration for EF Core entity mapping. Includes minor project and assembly info updates.
2026-02-23 17:54:53 -05:00
wjones 534b0cde91 Ensure initial user exists before startup wizard completes
Add logic to initialize at least one user if the startup wizard is not completed, both during application startup and in the startup user API. After user creation, reload the user entity with all related navigation properties to ensure the in-memory user object is fully populated. Also update build and publish metadata files.
2026-02-23 15:42:19 -05:00
wjones f833f0ceeb BaseItemRepository: 100% async conversion complete
All 21 public methods in BaseItemRepository are now fully async, including complex query, retrieval, write, delete, and aggregation operations. Added async signatures to IItemRepository and ILibraryManager with cancellation token support and ConfigureAwait(false). Sync wrappers retained for backward compatibility. Final conversion of GetLatestItemList and GetNextUpSeriesKeys completes Phase 3A. All bulk operations and aggregations are async, enabling concurrent dashboard loading and full PostgreSQL multiplexing. Migration routines fixed for Npgsql "command in progress" errors. Extensive documentation added for conversion process, performance, and testing. Project is now production-ready with zero build errors and 100% backward compatibility.
2026-02-23 12:36:29 -05:00
wjones 86883cd5c6 Refactor PostgreSQL provider: multi-schema & async prep
- Refactor migrations and provider to use multiple PostgreSQL schemas, each matching a legacy SQLite database (activitylog, authentication, displaypreferences, library, users).
- All tables, foreign keys, and indexes are now schema-qualified; Down migration drops tables by schema.
- Provider ensures schemas exist before migrations; entities are mapped to correct schemas in OnModelCreating.
- Add support for max-pool-size, min-pool-size, and multiplexing connection options; update logging accordingly.
- VACUUM ANALYZE now runs per schema during scheduled optimization.
- TruncateAllTablesAsync now truncates tables with schema qualification.
- README updated with schema structure, new options, and multiplexing warnings.
- CacheDecorator now calls async repository methods using .GetAwaiter().GetResult(), with documentation.
- Lays groundwork for full async/await and multiplexing support in the database layer.
2026-02-23 09:38:22 -05:00
wjones ede6904433 Add Jellyfin.CodeAnalysis and analyzers to all projects
Integrated Jellyfin.CodeAnalysis as a project reference across core and test projects. Added centrally managed code analysis and style enforcement packages (IDisposableAnalyzers, Microsoft.CodeAnalysis.BannedApiAnalyzers, SerilogAnalyzer, SmartAnalyzers.MultithreadingAnalyzer, StyleCop.Analyzers) to all affected projects. Updated dependency graphs, asset files, and build scripts to ensure analyzers run during builds, enforcing consistent code quality and style rules throughout the codebase. Removed custom warning properties to rely on analyzer enforcement.
2026-02-23 07:03:54 -05:00
wjones 8d39eb77e6 Remove Jellyfin.CodeAnalysis and analyzers from test deps
Removed all references to the Jellyfin.CodeAnalysis project and related analyzer NuGet packages (IDisposableAnalyzers, SerilogAnalyzer, SmartAnalyzers.MultithreadingAnalyzer, StyleCop.Analyzers, Microsoft.CodeAnalysis.BannedApiAnalyzers) from test projects and their dependency graphs. Updated .dgspec.json, project.assets.json, and project.nuget.cache files accordingly. Also updated warningProperties to explicitly set warning behaviors. This streamlines the dependency tree and removes static analysis tooling from the test build process.
2026-02-22 19:03:29 -05:00
wjones ed8fe21055 Update migration filters for PostgreSQL compatibility
Changed unique index filters in migration to use PostgreSQL syntax ("UserId" IS NOT NULL) instead of SQL Server syntax. Updated assembly version metadata and rebuilt related binaries.
2026-02-22 19:00:38 -05:00
wjones a3eb4b1b57 Add automatic PostgreSQL database creation and migration
- Added Jellyfin.Database.Providers.Postgres project with full EF Core migration support for PostgreSQL.
- Implemented automatic database creation and privilege assignment on startup.
- Generated initial migration and model snapshot for PostgreSQL schema.
- Updated build, test, and dependency files to include PostgreSQL provider and Npgsql packages.
- Added PowerShell script for generating and testing PostgreSQL migrations.
- No changes to application logic outside database provider/migration infrastructure.
2026-02-22 18:51:24 -05:00
wjones 0efab2cfb6 Add JELLYFIN_WEB_DIR env and update publish settings
- Add JELLYFIN_WEB_DIR environment variable to launch profiles and test project for consistent web directory configuration.
- Conditionally add AssemblyMetadata for ASPNETCORE_ENVIRONMENT and JELLYFIN_WEB_DIR in DEBUG builds.
- Update FolderProfile.pubxml to delete existing files on publish and set additional properties (TargetFramework, ProjectGuid, etc.).
- Bump Jellyfin.CodeAnalysis assembly version and update related binaries.
- Refresh publish history in FolderProfile.pubxml.user.
2026-02-22 16:43:14 -05:00
wjones f8b7c62f9e Add PostgreSQL provider and code cleanup
Added Jellyfin.Database.Providers.Postgres project and integrated Npgsql packages for PostgreSQL support. Updated solution and test project references. Refactored code for clarity and performance, including improved string handling, log formatting, and plugin management. Enhanced .editorconfig rules and fixed minor test issues. Added publishing profiles and updated web server setup references. Cleaned up assembly info and cache files.
Now able to successfully build solution
2026-02-22 16:15:19 -05:00
wjones d1e6fbc122 Improve string comparison and code formatting
Updated string comparison logic to use explicit StringComparison parameters for correctness and performance. Moved using statements outside namespace in event consumer classes. Added minor formatting fixes and closing braces. Suppressed IDisposableAnalyzers and reduced StyleCop rule severity in .editorconfig. Updated project cache and assembly info files. No changes to business logic.
2026-02-22 12:44:55 -05:00
wjones 8c300533f9 Reformat .editorconfig and update analyzer suppressions
Reorganized .editorconfig for clarity and maintainability, adding and adjusting StyleCop and IDE analyzer suppressions. Rebuilt Jellyfin.CodeAnalysis assembly and updated project.nuget.cache files to reflect configuration changes. No source code changes included.
2026-02-22 12:02:16 -05:00