pgsql-jellyfin/docs/PERFORMANCE_MONITORING_GUIDE.md

# Database Performance Monitoring - Quick Start Guide

**Generated**: March 5, 2026

## 📁 Files Created

### 1. **SQL Scripts**
- `sql/optimize_peoples_itemvalues_indexes.sql` - Add missing indexes for Peoples and ItemValues tables
- `sql/query-analysis.sql` - Identify and analyze slow queries (including 110-second ones)

### 2. **PowerShell Scripts**
- `scripts/Weekly-Performance-Monitor.ps1` - Automated weekly performance monitoring
- `scripts/Setup-Weekly-Monitor.ps1` - Set up automatic scheduling via Task Scheduler

---

## 🚀 Quick Start

### Step 1: Apply the Missing Indexes

Run this to add optimized indexes for Peoples and ItemValues tables:

```powershell
. .\scripts\db-config.ps1
Invoke-PSQL -File "sql\optimize_peoples_itemvalues_indexes.sql"
```

**Expected time**: 2-5 minutes (indexes created with CONCURRENTLY, no downtime)

**What it does**:
- ✅ Peoples table: 3 new indexes (name lookups, ID+Name combos, provider IDs)
- ✅ ItemValues table: 5 new indexes (CleanValue, Value, Type combos, common types)
- ✅ Updates table statistics (ANALYZE)
- ✅ Verifies new indexes were created

---

### Step 2: Analyze Slow Queries

Identify the 110-second queries and other performance issues:

```powershell
. .\scripts\db-config.ps1
Invoke-PSQL -File "sql\query-analysis.sql"
```

**What you'll see**:
- 🔥 Top 20 slowest queries (by max execution time)
- ⚡ Queries with highest average time
- 📊 Queries consuming most total database time
- 🎯 BaseItems-specific analysis (where 110s queries occur)
- 📈 Peoples and ItemValues query patterns
- ✅ Currently running queries (real-time)

**Save to file for review**:
```powershell
. .\scripts\db-config.ps1
Invoke-PSQL -File "sql\query-analysis.sql" -OutputFile "query_analysis_$(Get-Date -Format 'yyyy-MM-dd').txt"
```

---

### Step 3: Set Up Automated Weekly Monitoring

#### Option A: Manual Weekly Runs

Run the monitor script manually whenever needed:

```powershell
.\scripts\Weekly-Performance-Monitor.ps1
```

**Generates 5 reports in `reports/weekly/`**:
1. `diagnostics_<timestamp>.txt` - Full database diagnostics
2. `query_analysis_<timestamp>.txt` - Slow query analysis
3. `index_usage_<timestamp>.txt` - Index usage statistics
4. `table_sizes_<timestamp>.txt` - Table size and bloat
5. `summary_<timestamp>.txt` - Quick summary with recommendations

#### Option B: Automatic Weekly Runs (Recommended)

**Set up Task Scheduler** to run every Monday at 6 AM:

```powershell
# Must run PowerShell as Administrator
.\scripts\Setup-Weekly-Monitor.ps1
```

**Features**:
- ✅ Runs every Monday at 6 AM (customizable)
- ✅ Generates all 5 reports automatically
- ✅ Keeps last 12 weeks of history
- ✅ Auto-cleanup of old reports
- ✅ Compare week-over-week performance

**Customize schedule**:
```powershell
# Run on Sunday at 3 AM instead
.\scripts\Setup-Weekly-Monitor.ps1 -DayOfWeek Sunday -TaskTime "3am"
```

**Remove scheduled task**:
```powershell
.\scripts\Setup-Weekly-Monitor.ps1 -Uninstall
```

---

## 📊 Understanding the Results

### After Applying Indexes

**Wait 24-48 hours**, then check if sequential scans decreased:

```powershell
. .\scripts\db-config.ps1
Invoke-PSQL -Query @"
SELECT 
    tablename, 
    seq_scan, 
    idx_scan,
    ROUND(100.0 * idx_scan / NULLIF(seq_scan + idx_scan, 0), 2) AS index_usage_pct
FROM pg_stat_user_tables
WHERE tablename IN ('Peoples', 'ItemValues')
ORDER BY tablename;
"@
```

**Target metrics**:
- Peoples: Index usage should be >98% (currently 97.61%)
- ItemValues: Index usage should be >95% (currently 82.33%)
- Sequential scans should decrease by 50-80%

---

### Query Analysis Output

**Performance ratings in Section 5**:
- 🔥 **CRITICAL** (>100s) - Needs immediate attention
- ⚠️ **SLOW** (>10s) - Should be optimized
- ⚡ **MODERATE** (>1s) - Monitor but acceptable
- ✅ **FAST** (<1s) - Good performance

**What to look for**:
1. Queries marked 🔥 CRITICAL - top priority
2. High "calls" with slow avg_ms - big cumulative impact
3. Queries returning many rows slowly - missing indexes
4. BaseItems SELECT queries >10 seconds

---

## 🎯 Recommended Workflow

### Week 1 (Immediate):
1. ✅ Apply indexes: `Invoke-PSQL -File "sql\optimize_peoples_itemvalues_indexes.sql"`
2. ✅ Analyze queries: `Invoke-PSQL -File "sql\query-analysis.sql" -OutputFile "baseline_analysis.txt"`
3. ✅ Set up automation: `.\scripts\Setup-Weekly-Monitor.ps1`

### Week 2-4 (Monitoring):
1. 📊 Review weekly reports in `reports/weekly/`
2. 📈 Compare sequential scan counts
3. 🎯 Identify if 110-second queries still occur
4. ⚡ Check if new indexes are being used

### Week 4+ (Optimization):
1. 🗑️ Consider dropping unused indexes (>4 weeks with 0 usage)
2. 🔧 Fine-tune indexes based on actual query patterns
3. 📊 Review query performance trends
4. ✅ Document performance improvements

---

## 🛠️ Troubleshooting

### Problem: Indexes not being used

**Solution**: Update statistics and check query planner
```powershell
. .\scripts\db-config.ps1
Invoke-PSQL -Query "ANALYZE VERBOSE library.\"Peoples\", library.\"ItemValues\";"
```

### Problem: Still seeing high sequential scans

**Solution**: Get EXPLAIN ANALYZE for specific queries
```powershell
. .\scripts\db-config.ps1
Invoke-PSQL -Query "EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM library.\"Peoples\" WHERE \"Name\" = 'Tom Hanks';"
```

### Problem: 110-second queries persist

**Solution**: Run query analysis and review Section 2 and 5
```powershell
. .\scripts\db-config.ps1
Invoke-PSQL -File "sql\query-analysis.sql" | Select-String -Pattern "CRITICAL|110|max_ms.*:[1-9][0-9]{5}"
```

---

## 📞 Quick Reference Commands

```powershell
# Load database configuration
. .\scripts\db-config.ps1

# Apply indexes
Invoke-PSQL -File "sql\optimize_peoples_itemvalues_indexes.sql"

# Analyze slow queries
Invoke-PSQL -File "sql\query-analysis.sql"

# Run full diagnostics
Invoke-PSQL -File "sql\diagnostics.sql"

# Run weekly monitor (manual)
.\scripts\Weekly-Performance-Monitor.ps1

# Set up automation (as Admin)
.\scripts\Setup-Weekly-Monitor.ps1

# Check index usage
Invoke-PSQL -Query "SELECT tablename, indexname, idx_scan FROM pg_stat_user_indexes WHERE tablename IN ('Peoples', 'ItemValues') ORDER BY idx_scan DESC;"

# Reset statistics (after major changes)
Invoke-PSQL -Query "SELECT pg_stat_reset(); SELECT pg_stat_statements_reset();"
```

---

## 📈 Success Metrics

**After 1 week**:
- [ ] Peoples sequential scans reduced by >50%
- [ ] ItemValues index usage increased to >90%
- [ ] New indexes showing usage (idx_scan > 0)

**After 1 month**:
- [ ] No queries consistently taking >10 seconds
- [ ] Index usage >95% on all major tables
- [ ] Weekly reports show stable/improving performance
- [ ] 110-second queries identified and resolved

---

## 🔗 Related Files

- `/sql/diagnostics.sql` - Comprehensive diagnostics (already exists)
- `/sql/monitor-query-performance.sql` - Query performance monitoring (already exists)
- `/docs/DATABASE_ANALYSIS_REPORT.md` - Previous analysis (Feb 28, 2026)
- `/docs/database-query-optimization.md` - Query optimization guide

---

**Created**: March 5, 2026  
**Next Review**: March 12, 2026 (check weekly report)  
**Goal**: Eliminate 110-second queries, reduce sequential scans by 80%