Commit Graph

38 Commits

Author SHA1 Message Date
wjones 8b0503f398 Fix EF Core DistinctBy query translation errors
Refactor all item query methods to avoid untranslatable DistinctBy
in IQueryable expressions. Replace ApplyGroupingFilter with a
two-phase pattern: load IDs and grouping keys to memory, apply
grouping and paging in-memory, then reload full entities by ID.
Add ApplyGroupingInMemory<T> and ApplyPagingToIds helpers.
Use AsEnumerable() before DistinctBy in GetItemValues to force
client-side evaluation. Resolves InvalidOperationException on
/Items, /Items/Filters, /Users/{id}/Items/Latest, and /Studios
endpoints across all supported databases. See
EF_CORE_QUERY_TRANSLATION_FIX.md for details.
2026-03-05 17:17:16 -05:00
wjones 7eb2b445cb Postgres perf, path migration, and shutdown race fixes
- Major query performance boost: replaced correlated subqueries with DistinctBy() in BaseItemRepository, added episode deduplication index, and increased default command timeout to 120s.
- Fixed LibraryMonitor shutdown race: added disposal checks and exception handling in FileRefresher to prevent ObjectDisposedException.
- Added PowerShell and SQL utilities for fixing Linux paths in config files and database; new scripts for WebDir and path migration.
- Auto-fix for WebDir in startup.json on load; new helper scripts and docs.
- Added automated weekly DB performance monitoring scripts and reporting.
- Expanded documentation and README for all fixes, scripts, and migration steps.
- Updated SQL scripts for index creation and diagnostics; improved output handling.
- Updated db-config defaults and added new diagnostic/report files.
2026-03-05 08:56:42 -05:00
wjones 667c3768a9 Enhanced logging & error handling for auth and pg backups
- Add detailed debug/warning logs to authentication flows (AuthService, AuthorizationContext, WebSocketManager) for better traceability of auth attempts and failures.
- Log WebSocket authentication attempts, including IP, path, and token preview.
- Introduce PostgresVersionMismatchException for pg_dump/pg_restore version mismatches; parse and log server/client versions, clean up failed backups, and provide upgrade guidance.
- Add helper for parsing version mismatch errors.
- Update using directives and minor code cleanups.
- Update .csproj.user with new publish profile path.
2026-03-04 12:37:39 -05:00
wjones c76853a442 PostgreSQL: Production fixes—UPSERT, remote backup, auth logs
- Fix: Atomic UPSERT for BaseItemProviders (resolves duplicate key errors during concurrent metadata refresh; clears navigation property to prevent EF Core tracking conflicts)
- Add: Remote PostgreSQL backup support (removes localhost-only restriction; works with pg_dump/pg_restore for both local and remote DBs)
- Add: Configurable backup disable option (`disable-backups`)
- Fix: Query timeout and performance (documented `command-timeout` config, added performance index scripts)
- Fix: Authentication errors now log as warnings with clear messages (ExceptionMiddleware), reducing log noise
- Fix: SyncPlay authorization handler validates user before lookup, logs warnings for unauthenticated/unknown users (returns 403/404)
- Fix: Database deadlock detection logs warnings and allows EF Core auto-retry
- Add: Configurable LibraryMonitorDelay (min 30s, default 60s)
- Fix: SQLite migration filtering—skip SQLite-only migrations on PostgreSQL
- Chore: Suppress StyleCop warnings (SA1137, etc.) for project consistency
- Docs: 21 documentation files added/updated (config, backup, performance, troubleshooting, session summary)
- All changes are backward compatible and production-ready
2026-03-03 16:35:27 -05:00
wjones 623af06e46 Optimize BaseItemRepository docs for grouping performance
Extensively document query grouping performance issues and optimization attempts in BaseItemRepository. Add guides on PostgreSQL UUID aggregation limitations, recommended indexes, increasing database timeouts, and monitoring query performance. Current code reverts to original (slow) implementation due to EF Core/Npgsql limitations, with detailed comments and references to new docs. Provides clear workarounds and upgrade path for future `.DistinctBy()` support.
2026-03-01 11:33:16 -05:00
wjones 23ed81b6b1 Improve build reliability, query perf, and documentation
- Add build-error-resolution.md guide for CS0006 and IDE analyzer issues
- Add rebuild-solution.ps1 script for clean/reliable builds
- Suppress noisy IDE/StyleCop warnings in .editorconfig
- Optimize BaseItemRepository grouping queries to use Min(Id) instead of FirstOrDefault, avoiding correlated subqueries and greatly improving DB performance
- Add database-query-optimization.md explaining query changes and tuning
- Enable EF Core SQL query logging, sensitive data, and detailed errors when log level is Debug (ServiceCollectionExtensions, logging.json)
- Update readme with SQL logging instructions
- Clarify git commit/ignore guidance and improve inline documentation
2026-03-01 09:49:30 -05:00
wjones 5565dc3e05 Add automated PostgreSQL performance index management
- Add scripts and SQL for base and supplementary indexes (total 23)
- Integrate index checks and auto-creation into Jellyfin startup
- Update csproj to include SQL files in build/publish output
- Add diagnostics, publish/copy scripts, and troubleshooting docs
- Provide detailed guides for migration, verification, and merging
- Ensures robust, automated, and well-documented index setup
2026-02-28 15:13:04 -05:00
wjones 78bc22db79 Update BaseItemProviders after deletion in repository
Ensure entity.Provider is re-added to context after deleting
existing BaseItemProviders, preserving updated provider info.
2026-02-27 17:09:06 -05:00
wjones 5197948fcc Remove SQLite runtime provider; keep for migration only
SQLite provider removed from service collection and runtime database options. Microsoft.Data.Sqlite package added to Jellyfin.Server for migration routines (SQLite → PostgreSQL). Migration service updated to always use PostgreSQL. SqliteExtensions restored for migration helpers. SQLite migration test deleted. Documentation added to explain migration-only support and deployment impact. No breaking changes for users migrating from SQLite; deployment size increases (~68 MB) for cross-platform SQLite DLLs.
2026-02-26 17:14:30 -05:00
wjones 27e4d11b15 Remove SQLite support for PostgreSQL-only deployment
- Removed SQLite provider project reference and NuGet package
- Added remove-sqlite.ps1 script for automated cleanup and verification
- Updated documentation and PR descriptions for PostgreSQL focus
- Provided migration guidance and detailed removal plan
- Reduced build output size and improved maintainability
2026-02-26 16:31:13 -05:00
wjones 69c213f13c Auto-generate startup.json and add database presets
Introduces automatic creation of a self-documenting startup.json file on first startup, with example path configurations for Linux, Windows, and portable setups. Adds preset database configurations to database.xml for easy switching between SQLite and PostgreSQL. Refactors database config classes for XML serialization compatibility and ensures database.xml is never empty. Comprehensive documentation and migration guides included.
2026-02-25 16:29:08 -05:00
wjones dad5cbadf5 Stop tracking and ignore unwanted files 2026-02-25 09:17:02 -05:00
wjones 135b6a81b9 Update NuGet paths and add web client directory docs
Added README section describing the web client directory (`wwwroot`) and its configuration options. No code logic changes; only environment and documentation updates.
2026-02-24 12:21:55 -05:00
wjones ce2a060bd3 clean solution 2026-02-23 18:36:36 -05:00
wjones 43cdbf9b35 Refactor startup user creation and add HomeSection config
Refactored user initialization to always ensure at least one user exists at startup. If no users are present, a default admin user (admin/jellyfin) is created automatically. Updated StartupController endpoints to handle cases where no user exists. Simplified UserManager.InitializeAsync logic. Added HomeSectionConfiguration for EF Core entity mapping. Includes minor project and assembly info updates.
2026-02-23 17:54:53 -05:00
wjones 534b0cde91 Ensure initial user exists before startup wizard completes
Add logic to initialize at least one user if the startup wizard is not completed, both during application startup and in the startup user API. After user creation, reload the user entity with all related navigation properties to ensure the in-memory user object is fully populated. Also update build and publish metadata files.
2026-02-23 15:42:19 -05:00
wjones f833f0ceeb BaseItemRepository: 100% async conversion complete
All 21 public methods in BaseItemRepository are now fully async, including complex query, retrieval, write, delete, and aggregation operations. Added async signatures to IItemRepository and ILibraryManager with cancellation token support and ConfigureAwait(false). Sync wrappers retained for backward compatibility. Final conversion of GetLatestItemList and GetNextUpSeriesKeys completes Phase 3A. All bulk operations and aggregations are async, enabling concurrent dashboard loading and full PostgreSQL multiplexing. Migration routines fixed for Npgsql "command in progress" errors. Extensive documentation added for conversion process, performance, and testing. Project is now production-ready with zero build errors and 100% backward compatibility.
2026-02-23 12:36:29 -05:00
wjones 86883cd5c6 Refactor PostgreSQL provider: multi-schema & async prep
- Refactor migrations and provider to use multiple PostgreSQL schemas, each matching a legacy SQLite database (activitylog, authentication, displaypreferences, library, users).
- All tables, foreign keys, and indexes are now schema-qualified; Down migration drops tables by schema.
- Provider ensures schemas exist before migrations; entities are mapped to correct schemas in OnModelCreating.
- Add support for max-pool-size, min-pool-size, and multiplexing connection options; update logging accordingly.
- VACUUM ANALYZE now runs per schema during scheduled optimization.
- TruncateAllTablesAsync now truncates tables with schema qualification.
- README updated with schema structure, new options, and multiplexing warnings.
- CacheDecorator now calls async repository methods using .GetAwaiter().GetResult(), with documentation.
- Lays groundwork for full async/await and multiplexing support in the database layer.
2026-02-23 09:38:22 -05:00
wjones ede6904433 Add Jellyfin.CodeAnalysis and analyzers to all projects
Integrated Jellyfin.CodeAnalysis as a project reference across core and test projects. Added centrally managed code analysis and style enforcement packages (IDisposableAnalyzers, Microsoft.CodeAnalysis.BannedApiAnalyzers, SerilogAnalyzer, SmartAnalyzers.MultithreadingAnalyzer, StyleCop.Analyzers) to all affected projects. Updated dependency graphs, asset files, and build scripts to ensure analyzers run during builds, enforcing consistent code quality and style rules throughout the codebase. Removed custom warning properties to rely on analyzer enforcement.
2026-02-23 07:03:54 -05:00
wjones 8d39eb77e6 Remove Jellyfin.CodeAnalysis and analyzers from test deps
Removed all references to the Jellyfin.CodeAnalysis project and related analyzer NuGet packages (IDisposableAnalyzers, SerilogAnalyzer, SmartAnalyzers.MultithreadingAnalyzer, StyleCop.Analyzers, Microsoft.CodeAnalysis.BannedApiAnalyzers) from test projects and their dependency graphs. Updated .dgspec.json, project.assets.json, and project.nuget.cache files accordingly. Also updated warningProperties to explicitly set warning behaviors. This streamlines the dependency tree and removes static analysis tooling from the test build process.
2026-02-22 19:03:29 -05:00
wjones a3eb4b1b57 Add automatic PostgreSQL database creation and migration
- Added Jellyfin.Database.Providers.Postgres project with full EF Core migration support for PostgreSQL.
- Implemented automatic database creation and privilege assignment on startup.
- Generated initial migration and model snapshot for PostgreSQL schema.
- Updated build, test, and dependency files to include PostgreSQL provider and Npgsql packages.
- Added PowerShell script for generating and testing PostgreSQL migrations.
- No changes to application logic outside database provider/migration infrastructure.
2026-02-22 18:51:24 -05:00
wjones f8b7c62f9e Add PostgreSQL provider and code cleanup
Added Jellyfin.Database.Providers.Postgres project and integrated Npgsql packages for PostgreSQL support. Updated solution and test project references. Refactored code for clarity and performance, including improved string handling, log formatting, and plugin management. Enhanced .editorconfig rules and fixed minor test issues. Added publishing profiles and updated web server setup references. Cleaned up assembly info and cache files.
Now able to successfully build solution
2026-02-22 16:15:19 -05:00
wjones d1e6fbc122 Improve string comparison and code formatting
Updated string comparison logic to use explicit StringComparison parameters for correctness and performance. Moved using statements outside namespace in event consumer classes. Added minor formatting fixes and closing braces. Suppressed IDisposableAnalyzers and reduced StyleCop rule severity in .editorconfig. Updated project cache and assembly info files. No changes to business logic.
2026-02-22 12:44:55 -05:00
wjones 8c300533f9 Reformat .editorconfig and update analyzer suppressions
Reorganized .editorconfig for clarity and maintainability, adding and adjusting StyleCop and IDE analyzer suppressions. Rebuilt Jellyfin.CodeAnalysis assembly and updated project.nuget.cache files to reflect configuration changes. No source code changes included.
2026-02-22 12:02:16 -05:00
wjones 48569427a5 Refactor for clarity; add docs and .editorconfig
Refactored code to consistently use `this.` for instance members, improving clarity and maintainability. Added XML documentation comments to multiple methods and properties for better API documentation. Streamlined `FfProbeKeyframeExtractor` logic and enhanced EBML parsing code with comments and style improvements. Introduced a new `.editorconfig` to disable legacy StyleCop and IDisposableAnalyzers rules. Updated binary files, assembly info, and NuGet cache files due to rebuilds and dependency changes. No functional changes were made.
2026-02-21 16:55:27 -05:00
wjones e887356385 Improve XML docs and formatting in tests and constants
Added and enhanced XML documentation comments across the codebase, including detailed summaries and parameter descriptions for methods, classes, and constants. Documented all MatroskaConstants. Improved test readability with comments and explicit variable declarations. Made minor formatting and consistency fixes in test files. No functional changes. Updated project cache and binary files to reflect documentation improvements.
2026-02-21 15:46:18 -05:00
wjones 8421e3ad5c Suppress non-critical warnings, refactor Emby.Naming
Expanded .editorconfig to silence non-critical IDE/StyleCop warnings and updated Emby.Naming.csproj to prevent warnings-as-errors in Debug. Refactored codebase for modern C# style, removed unused code and unnecessary usings, and fixed formatting and StringComparison issues. Added detailed documentation on code style and warning suppression. Project now builds cleanly with only critical issues surfaced.
2026-02-21 12:48:04 -05:00
wjones 477045704e Enforce NuGet warnings as errors; refactor field access
Added "allWarningsAsErrors": true to NuGet config files across the solution to treat all warnings as errors, increasing build strictness. Updated project cache hashes and assembly info files to reflect these changes. Refactored C# code to use explicit `this.` for field and property assignments, improving clarity. Note: some method signatures were incorrectly changed to use `this.` as a prefix, which is invalid C# syntax and must be corrected before compiling. Binary files updated due to build changes.
2026-02-21 11:07:54 -05:00
wjones 33e95f154a Upgrade from .net 10 to .net11 to support postgresql integration 2026-02-20 22:04:55 -05:00
wjones db3b19dbb0 Migrate all projects and tests to .NET 11.0
Upgraded the target framework for all main, test, and provider projects from .NET 10.0 (net10.0) to .NET 11.0 (net11.0). Updated all major NuGet dependencies to their .NET 11-compatible versions, including Microsoft.AspNetCore.*, Microsoft.EntityFrameworkCore.*, Microsoft.Extensions.*, Serilog, System.Text.Json, and Npgsql.EntityFrameworkCore.PostgreSQL (now 11.0.0-preview.1). Regenerated all project.assets.json, NuGet cache, and MSBuild files to reflect the new framework and package versions. Removed or updated incompatible dependencies and references. Set allWarningsAsErrors: true in NuGet spec files for stricter builds. No application logic changes were made; all updates are related to project configuration, dependency management, and build system modernization for .NET 11.0.
2026-02-20 17:55:22 -05:00
wjones fd03e1f564 Add PostgreSQL provider and EF Core 10 support
Introduce Jellyfin.Database.Providers.Postgres project, enabling PostgreSQL as a database backend using Entity Framework Core 10 and Npgsql. Add custom value converters and model builder extensions for PostgreSQL compatibility. Update solution-wide dependencies to Microsoft.EntityFrameworkCore 10.0.3 and Npgsql.EntityFrameworkCore.PostgreSQL 9.0.2 (as a temporary workaround until 10.x is released). Regenerate all relevant NuGet, build, and cache files. Add new and updated binaries, debug symbols, and XML documentation for affected projects. No application source code changes outside of the new provider and supporting infrastructure.
2026-02-20 16:52:36 -05:00
wjones af1152b001 Refactor: standardize namespace and using directive style
Refactored all C# test files to use explicit namespace declarations and moved all using directives inside the namespace block for consistency. Updated assembly info and cache files to reflect the new build. Adjusted MvcTestingAppManifest.json to use fully qualified assembly names. No functional changes; all updates are related to code style, organization, and project metadata.
2026-02-20 16:26:53 -05:00
wjones d5fdbec943 Updated code to correct build errors for Jelyfin.Data and Jellyfin.Database.Implementations 2026-02-19 15:51:56 -05:00
wjones f47555f2aa commit after checkout of development branch 2026-02-19 12:38:09 -05:00
wjones 036953f3ff Updating project to correct the following errors:
1. Using directive should appear within a namespace declaration
2. Use trailing comma in multi-line initializers
2026-02-19 11:06:43 -05:00
wjones 1c6730e2ad creating pgsql development branch 2026-02-19 09:20:41 -05:00
wjones ddf6ff69be create branch for pgsql development 2026-02-19 08:37:20 -05:00
wjones 460884fea3 repo creation with initial code after cloning public repo 2026-02-19 07:36:25 -05:00