Jellyfin: who is watching, and don't update a machine mid-film
New Jellyfin integration (read-only, every 2 minutes): version, restart wanted, streams and transcoding, failed scheduled tasks, item counts, failed sign-ins. Monitoring -> Jellyfin page with a peak-streams history, three alerts, /metrics. When Jellyfin runs on a managed server, an update run asks first if people are streaming (409 "watching" unless startAnyway), and a patch plan waits up to two hours for them before updating that server. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
2f082ca5c2
commit
0146eb8bca
38 files changed
+8136
-13
No files matched your search
+30
-2
@@ -23,8 +23,8 @@ what the app actually creates.
|
|||||||
|---|---|
|
|---|---|
|
||||||
| **File** | `DATABASE_PATH`, default `../data/homelab-manager.sqlite` (relative to `server/`; `/app/data/...` in Docker). The folder is created if missing. |
|
| **File** | `DATABASE_PATH`, default `../data/homelab-manager.sqlite` (relative to `server/`; `/app/data/...` in Docker). The folder is created if missing. |
|
||||||
| **Foreign keys** | Switched on for every connection (`PRAGMA foreign_keys = ON`), so the `ON DELETE` rules below are enforced. |
|
| **Foreign keys** | Switched on for every connection (`PRAGMA foreign_keys = ON`), so the `ON DELETE` rules below are enforced. |
|
||||||
| **Migrations** | SQL files in [`server/drizzle/`](server/drizzle) (`0000` … `0056`), applied automatically when the server starts. Which ones have run is recorded in the table `__drizzle_migrations`. |
|
| **Migrations** | SQL files in [`server/drizzle/`](server/drizzle) (`0000` … `0057`), applied automatically when the server starts. Which ones have run is recorded in the table `__drizzle_migrations`. |
|
||||||
| **Tables** | 68 application tables, plus drizzle's own `__drizzle_migrations`. |
|
| **Tables** | 70 application tables, plus drizzle's own `__drizzle_migrations`. |
|
||||||
|
|
||||||
**Not in the database, but beside it in the data folder:** the backups of the database itself (`backups/homelab-manager-….sqlite`, taken daily and on request; see [BACKUP.md](BACKUP.md)), and, only while a restore is waiting or has just happened, `restore-pending.sqlite` / `restore-pending.json` and `restore-result.json`.
|
**Not in the database, but beside it in the data folder:** the backups of the database itself (`backups/homelab-manager-….sqlite`, taken daily and on request; see [BACKUP.md](BACKUP.md)), and, only while a restore is waiting or has just happened, `restore-pending.sqlite` / `restore-pending.json` and `restore-result.json`.
|
||||||
|
|
||||||
@@ -68,6 +68,8 @@ erDiagram
|
|||||||
integrations ||--o{ ha_readings : "cascade"
|
integrations ||--o{ ha_readings : "cascade"
|
||||||
integrations ||--o| ups_state : "cascade"
|
integrations ||--o| ups_state : "cascade"
|
||||||
integrations ||--o{ ups_readings : "cascade"
|
integrations ||--o{ ups_readings : "cascade"
|
||||||
|
integrations ||--o| jellyfin_state : "cascade"
|
||||||
|
integrations ||--o{ jellyfin_readings : "cascade"
|
||||||
users ||--o{ todos : "assignee (set null)"
|
users ||--o{ todos : "assignee (set null)"
|
||||||
servers ||--o{ network_server_links : "cascade"
|
servers ||--o{ network_server_links : "cascade"
|
||||||
integration_credentials ||--o{ dns_providers : "credential (set null)"
|
integration_credentials ||--o{ dns_providers : "credential (set null)"
|
||||||
@@ -845,6 +847,31 @@ A UPS at a moment, for the charts: one every five minutes, and every minute whil
|
|||||||
| `runtime_seconds` | integer | |
|
| `runtime_seconds` | integer | |
|
||||||
| `on_battery` | bool NOT NULL, default 0 | |
|
| `on_battery` | bool NOT NULL, default 0 | |
|
||||||
|
|
||||||
|
#### `jellyfin_state`
|
||||||
|
|
||||||
|
How a Jellyfin (an integration of type `jellyfin`) is right now, one row each, replaced at every read (every two minutes). `integration_id` is the primary key and a foreign key with **cascade**.
|
||||||
|
|
||||||
|
| Column | Type | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `integration_id` | integer PK | |
|
||||||
|
| `server_name`, `version`, `os` | text | As Jellyfin says. |
|
||||||
|
| `pending_restart`, `update_available` | bool NOT NULL, default 0 | |
|
||||||
|
| `streams` | text NOT NULL, default `[]` | JSON: who is streaming what (user, client, device, title, transcoding, paused). |
|
||||||
|
| `tasks` | text NOT NULL, default `[]` | JSON: the scheduled tasks and how each last ended. |
|
||||||
|
| `movies`, `series`, `episodes`, `songs` | integer | Item counts; null when not said. |
|
||||||
|
| `failed_logins_24h` | integer | Null when the activity log could not be read. |
|
||||||
|
| `updated_at` | text NOT NULL | ISO. |
|
||||||
|
|
||||||
|
#### `jellyfin_readings`
|
||||||
|
|
||||||
|
How many were streaming, and how many of those were transcoded, at a moment: one every five minutes and at once when either number changes, deleted after 35 days. `integration_id` is a foreign key with **cascade**; (`integration_id`, `at`) is indexed.
|
||||||
|
|
||||||
|
| Column | Type | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `id` | integer PK | |
|
||||||
|
| `at` | text NOT NULL | ISO. |
|
||||||
|
| `streams`, `transcodes` | integer NOT NULL | |
|
||||||
|
|
||||||
#### `service_levels`
|
#### `service_levels`
|
||||||
|
|
||||||
A service-level target (Monitoring → Service levels): what a thing is held to, measured over calendar months. No foreign keys: the thing measured is a plain reference, and a target whose subject is removed is shown as having nothing to measure.
|
A service-level target (Monitoring → Service levels): what a thing is held to, measured over calendar months. No foreign keys: the thing measured is a plain reference, and a target whose subject is removed is shown as having nothing to measure.
|
||||||
@@ -1282,6 +1309,7 @@ are deleted), and `notification_queue` is emptied each time the quiet-hours dige
|
|||||||
| `storage_samples` | 400 days (a pool not reported for 30 days is deleted with its readings) |
|
| `storage_samples` | 400 days (a pool not reported for 30 days is deleted with its readings) |
|
||||||
| `ha_readings` | 35 days |
|
| `ha_readings` | 35 days |
|
||||||
| `ups_readings` | 35 days |
|
| `ups_readings` | 35 days |
|
||||||
|
| `jellyfin_readings` | 35 days |
|
||||||
| `duplicati_runs` | 400 days, and at most 400 per backup job |
|
| `duplicati_runs` | 400 days, and at most 400 per backup job |
|
||||||
| `heartbeat_pings` | 14 days |
|
| `heartbeat_pings` | 14 days |
|
||||||
| `alert_incidents`, `incident_notes` | 400 days after the incident resolved |
|
| `alert_incidents`, `incident_notes` | 400 days after the incident resolved |
|
||||||
|
|||||||
@@ -321,6 +321,20 @@ That works on every Duplicati version, needs no login handshake and leaves nothi
|
|||||||
- **Not verified against a real UPS** — NUT's protocol and the UPS-MIB are as published, and both readers are tried against pretend servers. Some UPSs leave out objects (the page then shows a dash), and some
|
- **Not verified against a real UPS** — NUT's protocol and the UPS-MIB are as published, and both readers are tried against pretend servers. Some UPSs leave out objects (the page then shows a dash), and some
|
||||||
network cards number the load or voltage tables differently; if a value is missing, the **Diagnostic Log** has what was asked and answered.
|
network cards number the load or voltage tables differently; if a value is missing, the **Diagnostic Log** has what was asked and answered.
|
||||||
|
|
||||||
|
### Jellyfin
|
||||||
|
|
||||||
|
- **Config fields:** the **address** of Jellyfin (`http://host:8096`), an **API key** (Jellyfin → Dashboard → API Keys → add one; stored encrypted) and, for an https address with a certificate you made yourself, *insecure*.
|
||||||
|
- **Required access:** read only. The API key is Jellyfin's own, which is administrator-level in Jellyfin; this app only ever sends GET requests for the server's information, its sessions, its scheduled tasks, the item counts and the activity log. **Nothing is ever changed or played.**
|
||||||
|
- **What it does:** every **two minutes** Jellyfin is read and stored — its version, whether it **wants a restart** or has an update, **who is streaming what** and whether it is being **transcoded**, how its **scheduled tasks** (a library scan) last ended, how many films, series, episodes and songs there are, and how many
|
||||||
|
**sign-ins failed** in the last day. **Monitoring → Jellyfin** shows it, with a chart of the most that were streaming at once (readings kept 35 days).
|
||||||
|
- **Alerts** (category *Monitoring*, source *Jellyfin*; held back by maintenance on the integration): **waiting to be restarted** (warning), **a scheduled task failed** (warning; the task's name and error), **10 or more sign-ins failed in a day** (warning).
|
||||||
|
- **Somebody is watching.** If the Jellyfin runs on a server this app knows (the Jellyfin's address is that server's address or host name), then **before an update run restarts that machine** the update drawer says who is streaming and asks again, and the server answers `409 watching` to a run started without saying it should go ahead anyway.
|
||||||
|
A **patch plan** that comes to that server while people are streaming **waits for them, up to two hours**, then goes ahead and says so in its message; the server's row in the run reads *Waiting: 2 people are streaming from Cinema (Ann: Dune, …)*. Only a reading from the last ten minutes counts: if Jellyfin cannot be read, nobody is assumed to be watching.
|
||||||
|
- If Jellyfin cannot be read for ten minutes its alerts are neither raised nor cleared, the Alerts page says it could not check, and the integration is raised as failing like any other.
|
||||||
|
- `/metrics` has `homelab_jellyfin_streams`, `homelab_jellyfin_transcodes` and `homelab_jellyfin_pending_restart` for each Jellyfin that was read lately.
|
||||||
|
- **Not verified against a live Jellyfin** — the endpoints (`/System/Info`, `/Sessions`, `/ScheduledTasks`, `/Items/Counts`, `/System/ActivityLog/Entries`) and their fields are as Jellyfin publishes them and the reader is tried against a pretend server. Every field is read defensively, and a part that cannot be read (older versions
|
||||||
|
differ) is left out without failing the rest; the **Diagnostic Log** has what was asked and answered.
|
||||||
|
|
||||||
## DNS providers
|
## DNS providers
|
||||||
|
|
||||||
### Cloudflare
|
### Cloudflare
|
||||||
|
|||||||
@@ -526,6 +526,9 @@ meant to be kept and how far back it actually reaches. A history that holds far
|
|||||||
address once, and paste three options into Duplicati's default options; no password for Duplicati is kept and the app never calls it. Every job then appears with when it last succeeded, how the last run went, how long it took, how big the source is and what changed, with a history; a failed run is a warning (critical from the
|
address once, and paste three options into Duplicati's default options; no password for Duplicati is kept and the app never calls it. Every job then appears with when it last succeeded, how the last run went, how long it took, how big the source is and what changed, with a history; a failed run is a warning (critical from the
|
||||||
second in a row), an overdue one (2 days by default, per job) a warning (critical at three times that), and maintenance on the machine's server silences them. See INTEGRATIONS.md and docs/MONITORING.md.
|
second in a row), an overdue one (2 days by default, per job) a warning (critical at three times that), and maintenance on the machine's server silences them. See INTEGRATIONS.md and docs/MONITORING.md.
|
||||||
|
|
||||||
|
**Jellyfin** (Monitoring → Jellyfin). Add a Jellyfin integration (its address and an API key) and the app shows whether it is up, **who is streaming what** and whether it is being transcoded, the scheduled tasks that failed, and a chart of how many watched at once; it warns when Jellyfin **wants a restart**, a
|
||||||
|
**task failed** or **sign-ins are failing**. And when the Jellyfin runs on a server the app manages, **an update run asks first if people are watching**, and a patch plan **waits for them (up to two hours)** instead of restarting the machine in the middle of a film. See INTEGRATIONS.md and docs/MONITORING.md.
|
||||||
|
|
||||||
**Database backups** (Settings → Data → Backup). A copy of Homelab Manager's own database is taken every day (03:30
|
**Database backups** (Settings → Data → Backup). A copy of Homelab Manager's own database is taken every day (03:30
|
||||||
in the notification time zone, the newest 14 kept; on unless you turn it off), checked as it is made, and
|
in the notification time zone, the newest 14 kept; on unless you turn it off), checked as it is made, and
|
||||||
kept in `data/backups/`. The page lists them with **Download**, **Check**, **Restore…** and **Delete**,
|
kept in `data/backups/`. The page lists them with **Download**, **Check**, **Restore…** and **Delete**,
|
||||||
|
|||||||
@@ -46,6 +46,7 @@ page's own top-level link.
|
|||||||
| Uptime Kuma | `/uptime-kuma` | ✅ | ✅ | ✅ |
|
| Uptime Kuma | `/uptime-kuma` | ✅ | ✅ | ✅ |
|
||||||
| Home Assistant | `/home-assistant` | ✅ | ✅ | ✅ |
|
| Home Assistant | `/home-assistant` | ✅ | ✅ | ✅ |
|
||||||
| UPS | `/ups` | ✅ | ✅ | ✅ |
|
| UPS | `/ups` | ✅ | ✅ | ✅ |
|
||||||
|
| Jellyfin | `/jellyfin` | ✅ | ✅ | ✅ |
|
||||||
| **Automation** | | | | |
|
| **Automation** | | | | |
|
||||||
| Semaphore | `/semaphore` | ✅ | ✅ | ✅ |
|
| Semaphore | `/semaphore` | ✅ | ✅ | ✅ |
|
||||||
| Gitea | `/gitea` | ✅ | ✅ | ✅ |
|
| Gitea | `/gitea` | ✅ | ✅ | ✅ |
|
||||||
@@ -132,6 +133,7 @@ even further, to admin only:
|
|||||||
- **UPS** (Monitoring → UPS): everyone signed in sees each UPS, its history and what is on it; adding one (its address and SNMP community) is an administrator's job, like any integration, and what it powers is written in the dependency map.
|
- **UPS** (Monitoring → UPS): everyone signed in sees each UPS, its history and what is on it; adding one (its address and SNMP community) is an administrator's job, like any integration, and what it powers is written in the dependency map.
|
||||||
- **Endpoints**: everyone sees the checks and how they are doing; adding, changing and removing one is administrator-only (each makes the server connect somewhere); that includes the value a check judges and the header (a token) it sends, whose value is never shown again to anyone;
|
- **Endpoints**: everyone sees the checks and how they are doing; adding, changing and removing one is administrator-only (each makes the server connect somewhere); that includes the value a check judges and the header (a token) it sends, whose value is never shown again to anyone;
|
||||||
*Check now* needs operator.
|
*Check now* needs operator.
|
||||||
|
- **Jellyfin** (Monitoring → Jellyfin): everyone signed in sees who is streaming and what is wrong; adding one (its address and API key) is an administrator's job, like any integration. Starting an update run on a server people are streaming from needs `startAnyway` — the update drawer asks (operators).
|
||||||
- **Duplicati** (Backups → Duplicati): everyone signed in sees every backup and how it went; setting how often one is expected, muting it and forgetting it need operator; adding a machine (which makes an address that can write to this app) and removing one are administrators'. Reports arrive on a secret address with no sign-in: the token is the access.
|
- **Duplicati** (Backups → Duplicati): everyone signed in sees every backup and how it went; setting how often one is expected, muting it and forgetting it need operator; adding a machine (which makes an address that can write to this app) and removing one are administrators'. Reports arrive on a secret address with no sign-in: the token is the access.
|
||||||
- **Restore tests**: everyone sees them on the Backup coverage page; recording one needs operator; removing one is administrator-only.
|
- **Restore tests**: everyone sees them on the Backup coverage page; recording one needs operator; removing one is administrator-only.
|
||||||
- **Change log**: operators and administrators (it is made from the audit log, which has the same readers).
|
- **Change log**: operators and administrators (it is made from the audit log, which has the same readers).
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ This page explains what Homelab Manager watches, how often, how a problem become
|
|||||||
- [Service levels](#service-levels)
|
- [Service levels](#service-levels)
|
||||||
- [Your own alert rules](#your-own-alert-rules)
|
- [Your own alert rules](#your-own-alert-rules)
|
||||||
- [Duplicati backups](#duplicati-backups)
|
- [Duplicati backups](#duplicati-backups)
|
||||||
|
- [Jellyfin](#jellyfin)
|
||||||
- [Docker containers that are broken](#docker-containers-that-are-broken)
|
- [Docker containers that are broken](#docker-containers-that-are-broken)
|
||||||
- [Endpoint checks and heartbeats](#endpoint-checks-and-heartbeats)
|
- [Endpoint checks and heartbeats](#endpoint-checks-and-heartbeats)
|
||||||
- [Integrations](#integrations)
|
- [Integrations](#integrations)
|
||||||
@@ -129,6 +130,7 @@ Every threshold here is a default; where it can be changed, the place is named.
|
|||||||
| **Restore test** | A restore test failed, or none has been done within the window | failure always; age off by default | Settings → Notifications |
|
| **Restore test** | A restore test failed, or none has been done within the window | failure always; age off by default | Settings → Notifications |
|
||||||
| **Expiry** | A secret, certificate, domain or Tailscale key is within its warning window or has expired | per item (domains 30 days) | the item; Settings → Notifications |
|
| **Expiry** | A secret, certificate, domain or Tailscale key is within its warning window or has expired | per item (domains 30 days) | the item; Settings → Notifications |
|
||||||
| **Docker updates** | A container image has a newer version | | Settings → Notifications |
|
| **Docker updates** | A container image has a newer version | | Settings → Notifications |
|
||||||
|
| **Jellyfin needs a look** | Jellyfin wants a restart, a scheduled task (a library scan) failed, or 10 or more sign-ins failed in a day | on, with the integration | Jellyfin page; [below](#jellyfin) |
|
||||||
| **Duplicati backup failed / overdue** | A backup's last run failed (critical from the second in a row), or it has not succeeded for longer than it should | per job (2 days by default) | Duplicati page; [below](#duplicati-backups) |
|
| **Duplicati backup failed / overdue** | A backup's last run failed (critical from the second in a row), or it has not succeeded for longer than it should | per job (2 days by default) | Duplicati page; [below](#duplicati-backups) |
|
||||||
| **Docker container broken** | A container keeps restarting (critical), is unhealthy, stopped unexpectedly, or was created and never started; or Dockhand cannot reach a Docker host | on | Settings → Notifications; [below](#docker-containers-that-are-broken) |
|
| **Docker container broken** | A container keeps restarting (critical), is unhealthy, stopped unexpectedly, or was created and never started; or Dockhand cannot reach a Docker host | on | Settings → Notifications; [below](#docker-containers-that-are-broken) |
|
||||||
| **Automation failed** | A Semaphore template's or Gitea repository's latest run failed (a run still going or cancelled does not count) | | not configurable |
|
| **Automation failed** | A Semaphore template's or Gitea repository's latest run failed (a run still going or cancelled does not count) | | not configurable |
|
||||||
@@ -211,6 +213,14 @@ and one that no longer exists can be **forgotten** (with its history; if Duplica
|
|||||||
What is kept: the last result of each job, and each report for 400 days or 400 reports. `/metrics` has `homelab_duplicati_backup_ok`, `homelab_duplicati_last_success_age_seconds` and `homelab_duplicati_source_size_bytes` for each job. A job is only learned from its first report, so a job that has
|
What is kept: the last result of each job, and each report for 400 days or 400 reports. `/metrics` has `homelab_duplicati_backup_ok`, `homelab_duplicati_last_success_age_seconds` and `homelab_duplicati_source_size_bytes` for each job. A job is only learned from its first report, so a job that has
|
||||||
*never* reported is not known to be missing; and because the reports are pushed, **the app being down when a backup finishes** loses that report (the backup shows as overdue only once it is).
|
*never* reported is not known to be missing; and because the reports are pushed, **the app being down when a backup finishes** loses that report (the backup shows as overdue only once it is).
|
||||||
|
|
||||||
|
## Jellyfin
|
||||||
|
|
||||||
|
An integration of type *Jellyfin* ([INTEGRATIONS.md](../INTEGRATIONS.md#jellyfin)) is read every two minutes, and **Monitoring → Jellyfin** shows whether it is up, who is **streaming** right now (and whether that is transcoded, which is what loads the machine), the scheduled tasks that failed, and a chart of the most that were streaming at once.
|
||||||
|
It raises three warnings: **a restart is wanted**, **a scheduled task failed**, **10 or more sign-ins failed in a day** (the last is the one to look at if the server is open to the internet). Maintenance on the integration holds them.
|
||||||
|
|
||||||
|
Its other use is **not cutting off a film**. When the Jellyfin runs on a server the app knows, an update run on that server asks first if people are streaming (the update drawer says who; *Update anyway* goes ahead), and a patch plan waits for them for up to two hours before it updates that server.
|
||||||
|
Only a recent reading counts, so a Jellyfin that cannot be read never holds an update up. The match is by address: the Jellyfin's address must be the server's IP address or host name as the agent reports it (a Jellyfin behind a proxy on another machine is not matched).
|
||||||
|
|
||||||
## Docker containers that are broken
|
## Docker containers that are broken
|
||||||
|
|
||||||
If you use Dockhand, the health pass (every 15 minutes) also reads each container's **state** and **status text** — the same text the Docker page shows — and raises an alert (category *Failing services*, source *Docker*) for:
|
If you use Dockhand, the health pass (every 15 minutes) also reads each container's **state** and **status text** — the same text the Docker page shows — and raises an alert (category *Failing services*, source *Docker*) for:
|
||||||
|
|||||||
@@ -296,6 +296,8 @@ All gauges. The same table is checked by a test against what `/metrics` really s
|
|||||||
| `homelab_service_level_availability_percent`, `homelab_service_level_error_budget_remaining_minutes` | `name`, `subject`, `type` | How available the thing has been so far this calendar month, and the downtime still allowed before the target is missed (negative once it is). Left out for a target with nothing recorded this month. |
|
| `homelab_service_level_availability_percent`, `homelab_service_level_error_budget_remaining_minutes` | `name`, `subject`, `type` | How available the thing has been so far this calendar month, and the downtime still allowed before the target is missed (negative once it is). Left out for a target with nothing recorded this month. |
|
||||||
| `homelab_duplicati_backup_ok` | `backup`, `source` | 1 if the Duplicati backup's last run worked and it is not overdue, else 0. |
|
| `homelab_duplicati_backup_ok` | `backup`, `source` | 1 if the Duplicati backup's last run worked and it is not overdue, else 0. |
|
||||||
| `homelab_duplicati_last_success_age_seconds`, `homelab_duplicati_source_size_bytes` | `backup`, `source` | Seconds since it last succeeded (a run with warnings counts), and the size of what its last report examined. |
|
| `homelab_duplicati_last_success_age_seconds`, `homelab_duplicati_source_size_bytes` | `backup`, `source` | Seconds since it last succeeded (a run with warnings counts), and the size of what its last report examined. |
|
||||||
|
| `homelab_jellyfin_streams`, `homelab_jellyfin_transcodes` | `name` | How many are streaming from Jellyfin right now, and how many of those it is transcoding; left out for a Jellyfin not read in the last ten minutes. |
|
||||||
|
| `homelab_jellyfin_pending_restart` | `name` | 1 if Jellyfin is waiting to be restarted, else 0. |
|
||||||
| `homelab_ups_on_battery` | `name` | 1 if the UPS has lost mains power and runs on its battery, else 0; left out for a UPS that has not been read in the last five minutes. |
|
| `homelab_ups_on_battery` | `name` | 1 if the UPS has lost mains power and runs on its battery, else 0; left out for a UPS that has not been read in the last five minutes. |
|
||||||
| `homelab_ups_battery_charge_percent`, `homelab_ups_runtime_seconds`, `homelab_ups_load_percent`, `homelab_ups_input_volts` | `name` | How full the battery is, how long it would last at the present load, how much of its capacity the load uses, and the mains voltage it sees — each left out when the UPS does not say. |
|
| `homelab_ups_battery_charge_percent`, `homelab_ups_runtime_seconds`, `homelab_ups_load_percent`, `homelab_ups_input_volts` | `name` | How full the battery is, how long it would last at the present load, how much of its capacity the load uses, and the mains voltage it sees — each left out when the UPS does not say. |
|
||||||
| `homelab_heartbeat_up` | `name` | 1 if the job has reported in time and not said it failed, else 0; switched-off heartbeats are left out. |
|
| `homelab_heartbeat_up` | `name` | 1 if the job has reported in time and not said it failed, else 0; switched-off heartbeats are left out. |
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
CREATE TABLE `jellyfin_readings` (
|
||||||
|
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
|
||||||
|
`integration_id` integer NOT NULL,
|
||||||
|
`at` text NOT NULL,
|
||||||
|
`streams` integer NOT NULL,
|
||||||
|
`transcodes` integer NOT NULL,
|
||||||
|
FOREIGN KEY (`integration_id`) REFERENCES `integrations`(`id`) ON UPDATE no action ON DELETE cascade
|
||||||
|
);
|
||||||
|
--> statement-breakpoint
|
||||||
|
CREATE INDEX `jellyfin_readings_idx` ON `jellyfin_readings` (`integration_id`,`at`);--> statement-breakpoint
|
||||||
|
CREATE TABLE `jellyfin_state` (
|
||||||
|
`integration_id` integer PRIMARY KEY NOT NULL,
|
||||||
|
`server_name` text,
|
||||||
|
`version` text,
|
||||||
|
`os` text,
|
||||||
|
`pending_restart` integer DEFAULT false NOT NULL,
|
||||||
|
`update_available` integer DEFAULT false NOT NULL,
|
||||||
|
`streams` text DEFAULT '[]' NOT NULL,
|
||||||
|
`tasks` text DEFAULT '[]' NOT NULL,
|
||||||
|
`movies` integer,
|
||||||
|
`series` integer,
|
||||||
|
`episodes` integer,
|
||||||
|
`songs` integer,
|
||||||
|
`failed_logins_24h` integer,
|
||||||
|
`updated_at` text NOT NULL,
|
||||||
|
FOREIGN KEY (`integration_id`) REFERENCES `integrations`(`id`) ON UPDATE no action ON DELETE cascade
|
||||||
|
);
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -400,6 +400,13 @@
|
|||||||
"when": 1792869961357,
|
"when": 1792869961357,
|
||||||
"tag": "0056_dizzy_carnage",
|
"tag": "0056_dizzy_carnage",
|
||||||
"breakpoints": true
|
"breakpoints": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"idx": 57,
|
||||||
|
"version": "6",
|
||||||
|
"when": 1792870755473,
|
||||||
|
"tag": "0057_lame_cloak",
|
||||||
|
"breakpoints": true
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
@@ -58,6 +58,7 @@ import { homeAssistantRouter } from "./routes/homeAssistant.js";
|
|||||||
import { upsRouter } from "./routes/ups.js";
|
import { upsRouter } from "./routes/ups.js";
|
||||||
import { awayRouter } from "./routes/away.js";
|
import { awayRouter } from "./routes/away.js";
|
||||||
import { duplicatiReportRouter, duplicatiRouter } from "./routes/duplicati.js";
|
import { duplicatiReportRouter, duplicatiRouter } from "./routes/duplicati.js";
|
||||||
|
import { jellyfinRouter } from "./routes/jellyfin.js";
|
||||||
import { serviceLevelsRouter } from "./routes/serviceLevels.js";
|
import { serviceLevelsRouter } from "./routes/serviceLevels.js";
|
||||||
import { publicApiRouter } from "./routes/publicApi.js";
|
import { publicApiRouter } from "./routes/publicApi.js";
|
||||||
import { generatorRouter } from "./routes/generator.js";
|
import { generatorRouter } from "./routes/generator.js";
|
||||||
@@ -193,6 +194,7 @@ export function createApp() {
|
|||||||
app.use("/api/ups", upsRouter);
|
app.use("/api/ups", upsRouter);
|
||||||
app.use("/api/away", awayRouter);
|
app.use("/api/away", awayRouter);
|
||||||
app.use("/api/duplicati", duplicatiRouter);
|
app.use("/api/duplicati", duplicatiRouter);
|
||||||
|
app.use("/api/jellyfin", jellyfinRouter);
|
||||||
app.use("/api/service-levels", serviceLevelsRouter);
|
app.use("/api/service-levels", serviceLevelsRouter);
|
||||||
app.use("/api/todos", todosRouter);
|
app.use("/api/todos", todosRouter);
|
||||||
app.use("/api/notes", notesRouter);
|
app.use("/api/notes", notesRouter);
|
||||||
|
|||||||
@@ -553,6 +553,7 @@ export const integrationTypes = [
|
|||||||
"npm",
|
"npm",
|
||||||
"homeassistant",
|
"homeassistant",
|
||||||
"ups",
|
"ups",
|
||||||
|
"jellyfin",
|
||||||
] as const;
|
] as const;
|
||||||
export type IntegrationType = (typeof integrationTypes)[number];
|
export type IntegrationType = (typeof integrationTypes)[number];
|
||||||
|
|
||||||
@@ -1159,6 +1160,44 @@ export const haReadings = sqliteTable(
|
|||||||
(t) => [index("ha_readings_lookup_idx").on(t.integrationId, t.entityId, t.at)],
|
(t) => [index("ha_readings_lookup_idx").on(t.integrationId, t.entityId, t.at)],
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// ─── Jellyfin ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// How a Jellyfin server (an integration of type "jellyfin") is right now, one row each, replaced at every read: its version, whether it wants a restart, who is streaming, how its
|
||||||
|
// scheduled tasks last ended, and how many sign-ins failed lately. Read by services/jellyfinSync.ts.
|
||||||
|
export const jellyfinState = sqliteTable("jellyfin_state", {
|
||||||
|
integrationId: integer("integration_id")
|
||||||
|
.primaryKey()
|
||||||
|
.references(() => integrations.id, { onDelete: "cascade" }),
|
||||||
|
serverName: text("server_name"),
|
||||||
|
version: text("version"),
|
||||||
|
os: text("os"),
|
||||||
|
pendingRestart: integer("pending_restart", { mode: "boolean" }).notNull().default(false),
|
||||||
|
updateAvailable: integer("update_available", { mode: "boolean" }).notNull().default(false),
|
||||||
|
streams: text("streams").notNull().default("[]"), // JSON: who is streaming what
|
||||||
|
tasks: text("tasks").notNull().default("[]"), // JSON: the scheduled tasks and how they last ended
|
||||||
|
movies: integer("movies"),
|
||||||
|
series: integer("series"),
|
||||||
|
episodes: integer("episodes"),
|
||||||
|
songs: integer("songs"),
|
||||||
|
failedLogins24h: integer("failed_logins_24h"),
|
||||||
|
updatedAt: text("updated_at").notNull(), // ISO
|
||||||
|
});
|
||||||
|
|
||||||
|
// How many were streaming (and how many of those were being transcoded) at a moment, for the chart: one every five minutes, and at once when it changes. Kept 35 days.
|
||||||
|
export const jellyfinReadings = sqliteTable(
|
||||||
|
"jellyfin_readings",
|
||||||
|
{
|
||||||
|
id: integer("id").primaryKey({ autoIncrement: true }),
|
||||||
|
integrationId: integer("integration_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => integrations.id, { onDelete: "cascade" }),
|
||||||
|
at: text("at").notNull(), // ISO
|
||||||
|
streams: integer("streams").notNull(),
|
||||||
|
transcodes: integer("transcodes").notNull(),
|
||||||
|
},
|
||||||
|
(t) => [index("jellyfin_readings_idx").on(t.integrationId, t.at)],
|
||||||
|
);
|
||||||
|
|
||||||
// ─── Duplicati ──────────────────────────────────────────────────────────────
|
// ─── Duplicati ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
// A Duplicati that reports to this app: one row per machine running it, with the secret token its reports carry in their address (only the hash is kept). Duplicati pushes a
|
// A Duplicati that reports to this app: one row per machine running it, with the secret token its reports carry in their address (only the hash is kept). Duplicati pushes a
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ import { initUnifiSync } from "./services/unifiSync.js";
|
|||||||
import { initHomeAssistantSync } from "./services/homeAssistantSync.js";
|
import { initHomeAssistantSync } from "./services/homeAssistantSync.js";
|
||||||
import { initUpsSync } from "./services/upsSync.js";
|
import { initUpsSync } from "./services/upsSync.js";
|
||||||
import { initAwayMonitor } from "./services/awayMonitor.js";
|
import { initAwayMonitor } from "./services/awayMonitor.js";
|
||||||
|
import { initJellyfinSync } from "./services/jellyfinSync.js";
|
||||||
import { backfillMetricRollups } from "./services/serverMetrics.js";
|
import { backfillMetricRollups } from "./services/serverMetrics.js";
|
||||||
import { backfillEndpointHourly } from "./services/endpointChecks.js";
|
import { backfillEndpointHourly } from "./services/endpointChecks.js";
|
||||||
import { initSelfBackupScheduler } from "./services/selfBackupScheduler.js";
|
import { initSelfBackupScheduler } from "./services/selfBackupScheduler.js";
|
||||||
@@ -79,6 +80,7 @@ initUnifiSync();
|
|||||||
initHomeAssistantSync();
|
initHomeAssistantSync();
|
||||||
initUpsSync();
|
initUpsSync();
|
||||||
initAwayMonitor();
|
initAwayMonitor();
|
||||||
|
initJellyfinSync();
|
||||||
// The hourly and daily figures are built from what is already kept, once, the first time after they were added (in the background: nothing waits for it).
|
// The hourly and daily figures are built from what is already kept, once, the first time after they were added (in the background: nothing waits for it).
|
||||||
void Promise.all([backfillMetricRollups(), backfillEndpointHourly()]).catch((err) => console.error("[history] couldn't build the hourly and daily figures:", err));
|
void Promise.all([backfillMetricRollups(), backfillEndpointHourly()]).catch((err) => console.error("[history] couldn't build the hourly and daily figures:", err));
|
||||||
|
|
||||||
|
|||||||
@@ -99,6 +99,11 @@ export const INTEGRATION_FIELDS: Partial<Record<IntegrationType, IntegrationFiel
|
|||||||
{ key: "token", label: "Long-lived access token", secret: true, type: "password", placeholder: "made on your profile page in Home Assistant" },
|
{ key: "token", label: "Long-lived access token", secret: true, type: "password", placeholder: "made on your profile page in Home Assistant" },
|
||||||
{ key: "insecure", label: "Allow self-signed certificate", secret: false, type: "checkbox" },
|
{ key: "insecure", label: "Allow self-signed certificate", secret: false, type: "checkbox" },
|
||||||
],
|
],
|
||||||
|
jellyfin: [
|
||||||
|
{ key: "url", label: "Jellyfin URL", secret: false, placeholder: "http://jellyfin.example.lan:8096" },
|
||||||
|
{ key: "apiKey", label: "API key", secret: true, type: "password", placeholder: "made under Dashboard → API Keys in Jellyfin" },
|
||||||
|
{ key: "insecure", label: "Allow self-signed certificate", secret: false, type: "checkbox" },
|
||||||
|
],
|
||||||
ups: [
|
ups: [
|
||||||
{ key: "host", label: "Host name or address", secret: false, placeholder: "the machine running NUT (upsd), or the UPS's network card" },
|
{ key: "host", label: "Host name or address", secret: false, placeholder: "the machine running NUT (upsd), or the UPS's network card" },
|
||||||
{ key: "snmp", label: "Read it over SNMP (version 2c) instead of NUT", secret: false, type: "checkbox" },
|
{ key: "snmp", label: "Read it over SNMP (version 2c) instead of NUT", secret: false, type: "checkbox" },
|
||||||
|
|||||||
@@ -0,0 +1,169 @@
|
|||||||
|
/**
|
||||||
|
* Jellyfin adapter. Config: url, apiKey (made in Jellyfin under Dashboard → API Keys); optional: insecure.
|
||||||
|
*
|
||||||
|
* Reading only: the server's own information (version, whether it wants a restart or an update), who is streaming right now and whether it is being transcoded, the scheduled tasks
|
||||||
|
* (a library scan that failed), how many items there are and how many sign-ins failed lately. Nothing is sent to Jellyfin but requests for these.
|
||||||
|
*
|
||||||
|
* Endpoints and field names are from Jellyfin's published API (`/System/Info`, `/Sessions`, `/ScheduledTasks`, `/Items/Counts`, `/System/ActivityLog/Entries`) and have not
|
||||||
|
* been checked against a live server here; every field is read defensively, and one that is missing is just left out.
|
||||||
|
*/
|
||||||
|
import { requestJson } from "../httpJson.js";
|
||||||
|
import { withDiagLogging } from "../../services/diagLog.js";
|
||||||
|
|
||||||
|
export interface JellyfinConfig {
|
||||||
|
url: string;
|
||||||
|
apiKey: string;
|
||||||
|
insecure?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface JellyfinStream {
|
||||||
|
user: string;
|
||||||
|
client: string | null;
|
||||||
|
device: string | null;
|
||||||
|
/** What is being watched or listened to, with the series it is of. */
|
||||||
|
title: string;
|
||||||
|
/** Converted on the fly, which is what loads the server. */
|
||||||
|
transcoding: boolean;
|
||||||
|
paused: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface JellyfinTask {
|
||||||
|
name: string;
|
||||||
|
category: string | null;
|
||||||
|
state: string;
|
||||||
|
/** Completed, Failed, Cancelled or Aborted; null when it has never run. */
|
||||||
|
lastStatus: string | null;
|
||||||
|
lastEndedAt: string | null;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface JellyfinRead {
|
||||||
|
name: string | null;
|
||||||
|
version: string | null;
|
||||||
|
os: string | null;
|
||||||
|
pendingRestart: boolean;
|
||||||
|
updateAvailable: boolean;
|
||||||
|
streams: JellyfinStream[];
|
||||||
|
tasks: JellyfinTask[];
|
||||||
|
counts: { movies: number | null; series: number | null; episodes: number | null; songs: number | null };
|
||||||
|
/** Failed sign-ins in the last day; null when the activity log could not be read. */
|
||||||
|
failedLogins24h: number | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface JellyfinAdapter {
|
||||||
|
ping(): Promise<{ ok: boolean; latencyMs?: number; error?: string }>;
|
||||||
|
read(now?: Date): Promise<JellyfinRead>;
|
||||||
|
}
|
||||||
|
|
||||||
|
const obj = (v: unknown): Record<string, unknown> => (v !== null && typeof v === "object" && !Array.isArray(v) ? (v as Record<string, unknown>) : {});
|
||||||
|
const str = (v: unknown): string | null => (typeof v === "string" && v.trim() !== "" ? v.trim() : null);
|
||||||
|
const num = (v: unknown): number | null => (typeof v === "number" && Number.isFinite(v) ? v : null);
|
||||||
|
const iso = (v: unknown): string | null => {
|
||||||
|
const s = str(v);
|
||||||
|
return s && Number.isFinite(Date.parse(s)) ? new Date(s).toISOString() : null;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Pure: what /System/Info says. */
|
||||||
|
export function parseSystemInfo(raw: unknown): Pick<JellyfinRead, "name" | "version" | "os" | "pendingRestart" | "updateAvailable"> {
|
||||||
|
const o = obj(raw);
|
||||||
|
return { name: str(o.ServerName), version: str(o.Version), os: str(o.OperatingSystemDisplayName) ?? str(o.OperatingSystem), pendingRestart: o.HasPendingRestart === true, updateAvailable: o.HasUpdateAvailable === true };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pure: the sessions that are playing something. A session with nothing playing is just somebody signed in. */
|
||||||
|
export function parseSessions(raw: unknown): JellyfinStream[] {
|
||||||
|
const out: JellyfinStream[] = [];
|
||||||
|
for (const item of Array.isArray(raw) ? raw : []) {
|
||||||
|
const s = obj(item);
|
||||||
|
const playing = obj(s.NowPlayingItem);
|
||||||
|
const name = str(playing.Name);
|
||||||
|
if (!name) continue;
|
||||||
|
const series = str(playing.SeriesName);
|
||||||
|
const state = obj(s.PlayState);
|
||||||
|
out.push({
|
||||||
|
user: str(s.UserName) ?? "someone",
|
||||||
|
client: str(s.Client),
|
||||||
|
device: str(s.DeviceName),
|
||||||
|
title: series ? `${series} — ${name}` : name,
|
||||||
|
transcoding: str(state.PlayMethod)?.toLowerCase() === "transcode" || (s.TranscodingInfo !== undefined && s.TranscodingInfo !== null && Object.keys(obj(s.TranscodingInfo)).length > 0),
|
||||||
|
paused: state.IsPaused === true,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pure: the scheduled tasks with how they last ended. */
|
||||||
|
export function parseTasks(raw: unknown): JellyfinTask[] {
|
||||||
|
const out: JellyfinTask[] = [];
|
||||||
|
for (const item of Array.isArray(raw) ? raw : []) {
|
||||||
|
const t = obj(item);
|
||||||
|
const name = str(t.Name);
|
||||||
|
if (!name || t.IsHidden === true) continue;
|
||||||
|
const last = obj(t.LastExecutionResult);
|
||||||
|
out.push({ name, category: str(t.Category), state: str(t.State) ?? "Idle", lastStatus: str(last.Status), lastEndedAt: iso(last.EndTimeUtc), error: str(last.ErrorMessage) });
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pure: how many of the activity log's entries from the last day are failed sign-ins. */
|
||||||
|
export function countFailedLogins(raw: unknown, now: Date): number | null {
|
||||||
|
const items = obj(raw).Items;
|
||||||
|
if (!Array.isArray(items)) return null;
|
||||||
|
const since = now.getTime() - 86_400_000;
|
||||||
|
return items.filter((i) => {
|
||||||
|
const e = obj(i);
|
||||||
|
const at = Date.parse(String(e.Date ?? ""));
|
||||||
|
return str(e.Type) === "AuthenticationFailed" && (!Number.isFinite(at) || at >= since);
|
||||||
|
}).length;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createJellyfinAdapter(config: JellyfinConfig): JellyfinAdapter {
|
||||||
|
const base = String(config.url ?? "").replace(/\/+$/, "");
|
||||||
|
const insecure = config.insecure === true;
|
||||||
|
const headers = { "X-Emby-Token": config.apiKey, Authorization: `MediaBrowser Token="${config.apiKey}"` };
|
||||||
|
|
||||||
|
async function get(path: string): Promise<unknown> {
|
||||||
|
const res = await requestJson(`${base}${path}`, { headers, insecure });
|
||||||
|
if (res.status === 401 || res.status === 403) throw new Error(`Jellyfin did not accept the API key (HTTP ${res.status})`);
|
||||||
|
if (res.status === 404) throw new Error("Jellyfin has no API at this address — check the address and port (8096)");
|
||||||
|
if (res.status < 200 || res.status >= 300) throw new Error(`Jellyfin error: HTTP ${res.status}`);
|
||||||
|
if (res.json === null) throw new Error("Jellyfin answered with something that isn't JSON — is the address its own?");
|
||||||
|
return res.json;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function read(now: Date = new Date()): Promise<JellyfinRead> {
|
||||||
|
const info = parseSystemInfo(await get("/System/Info"));
|
||||||
|
// The rest is worth having but not worth failing for: a Jellyfin that answers about itself and not about its sessions still says it is up.
|
||||||
|
const soft = async <T>(path: string, parse: (raw: unknown) => T, fallback: T): Promise<T> => {
|
||||||
|
try {
|
||||||
|
return parse(await get(path));
|
||||||
|
} catch {
|
||||||
|
return fallback;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const [streams, tasks, countsRaw, failed] = await Promise.all([
|
||||||
|
soft("/Sessions?ActiveWithinSeconds=300", parseSessions, [] as JellyfinStream[]),
|
||||||
|
soft("/ScheduledTasks?isHidden=false", parseTasks, [] as JellyfinTask[]),
|
||||||
|
soft("/Items/Counts", obj, {} as Record<string, unknown>),
|
||||||
|
soft(`/System/ActivityLog/Entries?startIndex=0&limit=100&minDate=${encodeURIComponent(new Date(now.getTime() - 86_400_000).toISOString())}`, (r) => countFailedLogins(r, now), null as number | null),
|
||||||
|
]);
|
||||||
|
return {
|
||||||
|
...info,
|
||||||
|
streams,
|
||||||
|
tasks,
|
||||||
|
counts: { movies: num(countsRaw.MovieCount), series: num(countsRaw.SeriesCount), episodes: num(countsRaw.EpisodeCount), songs: num(countsRaw.SongCount) },
|
||||||
|
failedLogins24h: failed,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function ping(): Promise<{ ok: boolean; latencyMs?: number; error?: string }> {
|
||||||
|
const start = Date.now();
|
||||||
|
try {
|
||||||
|
await get("/System/Info");
|
||||||
|
return { ok: true, latencyMs: Date.now() - start };
|
||||||
|
} catch (err) {
|
||||||
|
return { ok: false, error: err instanceof Error ? err.message : String(err) };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return withDiagLogging("jellyfin", { ping, read });
|
||||||
|
}
|
||||||
@@ -14,6 +14,7 @@ import { createUnifiAdapter } from "./unifi/adapter.js";
|
|||||||
import { createNpmAdapter } from "./npm/adapter.js";
|
import { createNpmAdapter } from "./npm/adapter.js";
|
||||||
import { createHomeAssistantAdapter } from "./homeassistant/adapter.js";
|
import { createHomeAssistantAdapter } from "./homeassistant/adapter.js";
|
||||||
import { createUpsAdapter } from "./ups/adapter.js";
|
import { createUpsAdapter } from "./ups/adapter.js";
|
||||||
|
import { createJellyfinAdapter } from "./jellyfin/adapter.js";
|
||||||
|
|
||||||
export interface PingableAdapter {
|
export interface PingableAdapter {
|
||||||
ping(): Promise<{ ok: boolean; latencyMs?: number; error?: string }>;
|
ping(): Promise<{ ok: boolean; latencyMs?: number; error?: string }>;
|
||||||
@@ -56,6 +57,8 @@ export function createIntegrationAdapter(type: IntegrationType, config: Integrat
|
|||||||
return createHomeAssistantAdapter(config as any);
|
return createHomeAssistantAdapter(config as any);
|
||||||
case "ups":
|
case "ups":
|
||||||
return createUpsAdapter(config as any);
|
return createUpsAdapter(config as any);
|
||||||
|
case "jellyfin":
|
||||||
|
return createJellyfinAdapter(config as any);
|
||||||
default:
|
default:
|
||||||
throw new Error(`Integration type "${type}" is not implemented yet`);
|
throw new Error(`Integration type "${type}" is not implemented yet`);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ import { getSourceStatus } from "../services/integrationHealthMonitor.js";
|
|||||||
import { removeIntegrationData, syncUnifiIntegration } from "../services/unifiSync.js";
|
import { removeIntegrationData, syncUnifiIntegration } from "../services/unifiSync.js";
|
||||||
import { removeHomeAssistantData, syncHomeAssistant } from "../services/homeAssistantSync.js";
|
import { removeHomeAssistantData, syncHomeAssistant } from "../services/homeAssistantSync.js";
|
||||||
import { forgetUps, syncUps } from "../services/upsSync.js";
|
import { forgetUps, syncUps } from "../services/upsSync.js";
|
||||||
|
import { forgetJellyfin, syncJellyfin } from "../services/jellyfinSync.js";
|
||||||
import { invalidateAlertsCache } from "../services/alerts.js";
|
import { invalidateAlertsCache } from "../services/alerts.js";
|
||||||
import { getSettings, updateSettings } from "../services/settingsStore.js";
|
import { getSettings, updateSettings } from "../services/settingsStore.js";
|
||||||
import { asyncHandler } from "../utils/asyncHandler.js";
|
import { asyncHandler } from "../utils/asyncHandler.js";
|
||||||
@@ -131,6 +132,7 @@ integrationsRouter.post("/", requireRole("admin"), asyncHandler(async (req, res)
|
|||||||
if (type === "unifi") void syncUnifiIntegration(created.id).then(() => invalidateAlertsCache());
|
if (type === "unifi") void syncUnifiIntegration(created.id).then(() => invalidateAlertsCache());
|
||||||
if (type === "homeassistant") void syncHomeAssistant(created.id).then(() => invalidateAlertsCache());
|
if (type === "homeassistant") void syncHomeAssistant(created.id).then(() => invalidateAlertsCache());
|
||||||
if (type === "ups") void syncUps(created.id).then(() => invalidateAlertsCache());
|
if (type === "ups") void syncUps(created.id).then(() => invalidateAlertsCache());
|
||||||
|
if (type === "jellyfin") void syncJellyfin(created.id).then(() => invalidateAlertsCache());
|
||||||
|
|
||||||
res.status(201).json({
|
res.status(201).json({
|
||||||
integration: {
|
integration: {
|
||||||
@@ -234,6 +236,7 @@ integrationsRouter.patch("/:id", requireRole("admin"), asyncHandler(async (req,
|
|||||||
if (updated.type === "unifi" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncUnifiIntegration(updated.id).then(() => invalidateAlertsCache());
|
if (updated.type === "unifi" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncUnifiIntegration(updated.id).then(() => invalidateAlertsCache());
|
||||||
if (updated.type === "homeassistant" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncHomeAssistant(updated.id).then(() => invalidateAlertsCache());
|
if (updated.type === "homeassistant" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncHomeAssistant(updated.id).then(() => invalidateAlertsCache());
|
||||||
if (updated.type === "ups" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncUps(updated.id).then(() => invalidateAlertsCache());
|
if (updated.type === "ups" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncUps(updated.id).then(() => invalidateAlertsCache());
|
||||||
|
if (updated.type === "jellyfin" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncJellyfin(updated.id).then(() => invalidateAlertsCache());
|
||||||
|
|
||||||
res.json({
|
res.json({
|
||||||
integration: {
|
integration: {
|
||||||
@@ -313,6 +316,7 @@ integrationsRouter.delete("/:id", requireRole("admin"), asyncHandler(async (req,
|
|||||||
if (existing.type === "unifi") await removeIntegrationData(id);
|
if (existing.type === "unifi") await removeIntegrationData(id);
|
||||||
if (existing.type === "homeassistant") await removeHomeAssistantData(id);
|
if (existing.type === "homeassistant") await removeHomeAssistantData(id);
|
||||||
if (existing.type === "ups") await forgetUps(id);
|
if (existing.type === "ups") await forgetUps(id);
|
||||||
|
if (existing.type === "jellyfin") await forgetJellyfin(id);
|
||||||
await db.delete(integrations).where(eq(integrations.id, id));
|
await db.delete(integrations).where(eq(integrations.id, id));
|
||||||
if (existing.credentialId) {
|
if (existing.credentialId) {
|
||||||
await db.delete(integrationCredentials).where(eq(integrationCredentials.id, existing.credentialId));
|
await db.delete(integrationCredentials).where(eq(integrationCredentials.id, existing.credentialId));
|
||||||
|
|||||||
@@ -0,0 +1,77 @@
|
|||||||
|
import { Router } from "express";
|
||||||
|
import { and, asc, eq, gte } from "drizzle-orm";
|
||||||
|
import { db } from "../db/client.js";
|
||||||
|
import { integrations, jellyfinReadings, jellyfinState, servers } from "../db/schema.js";
|
||||||
|
import { requireAuth } from "../auth/middleware.js";
|
||||||
|
import { hostOfAddress } from "../services/kumaTargets.js";
|
||||||
|
import { bucketPeaks, isJellyfinFresh, jellyfinProblems, jellyfinSyncStatuses, READING_KEEP_DAYS, streamsOf, tasksOf, watchingOnServer, watchingWords } from "../services/jellyfinSync.js";
|
||||||
|
import { matchServerForTarget } from "../services/uptimeKumaMatch.js";
|
||||||
|
import { asyncHandler } from "../utils/asyncHandler.js";
|
||||||
|
|
||||||
|
export const jellyfinRouter = Router();
|
||||||
|
jellyfinRouter.use(requireAuth);
|
||||||
|
|
||||||
|
const DAY_MS = 86_400_000;
|
||||||
|
const idOf = (v: string) => (/^\d+$/.test(v) ? Number(v) : NaN);
|
||||||
|
|
||||||
|
/** Every Jellyfin, how it is and who is streaming from it, from what the last read stored. Everyone signed in can see it, like the other pages made of stored readings. */
|
||||||
|
jellyfinRouter.get("/", asyncHandler(async (_req, res) => {
|
||||||
|
const [rows, states, statuses, serverRows] = await Promise.all([
|
||||||
|
db.select({ id: integrations.id, name: integrations.name, enabled: integrations.enabled, baseUrl: integrations.baseUrl }).from(integrations).where(eq(integrations.type, "jellyfin")).orderBy(integrations.name),
|
||||||
|
db.select().from(jellyfinState),
|
||||||
|
jellyfinSyncStatuses(),
|
||||||
|
db.select({ id: servers.id, name: servers.name, hostname: servers.hostname, ipAddresses: servers.ipAddresses }).from(servers),
|
||||||
|
]);
|
||||||
|
const matchable = serverRows.map((s) => {
|
||||||
|
let ips: string[] = [];
|
||||||
|
try {
|
||||||
|
const v = JSON.parse(s.ipAddresses ?? "[]");
|
||||||
|
ips = Array.isArray(v) ? v.filter((x): x is string => typeof x === "string") : [];
|
||||||
|
} catch {
|
||||||
|
// none
|
||||||
|
}
|
||||||
|
return { id: s.id, name: s.name, hostname: s.hostname, ips };
|
||||||
|
});
|
||||||
|
const byId = new Map(states.map((s) => [s.integrationId, s]));
|
||||||
|
const now = Date.now();
|
||||||
|
res.json({
|
||||||
|
jellyfins: rows.map((r) => {
|
||||||
|
const state = byId.get(r.id) ?? null;
|
||||||
|
const status = statuses[r.id] ?? null;
|
||||||
|
const fresh = isJellyfinFresh(status ?? undefined, now);
|
||||||
|
const machine = matchServerForTarget(hostOfAddress(r.baseUrl), matchable);
|
||||||
|
return {
|
||||||
|
id: r.id,
|
||||||
|
name: r.name,
|
||||||
|
enabled: r.enabled,
|
||||||
|
runsOn: machine ? { id: machine.id, name: machine.name } : null,
|
||||||
|
status,
|
||||||
|
fresh,
|
||||||
|
state: state ? { serverName: state.serverName, version: state.version, os: state.os, pendingRestart: state.pendingRestart, updateAvailable: state.updateAvailable, streams: streamsOf(state), tasks: tasksOf(state), counts: { movies: state.movies, series: state.series, episodes: state.episodes, songs: state.songs }, failedLogins24h: state.failedLogins24h, updatedAt: state.updatedAt } : null,
|
||||||
|
problems: state && fresh ? jellyfinProblems(r.id, r.name, state) : [],
|
||||||
|
};
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
}));
|
||||||
|
|
||||||
|
/** Who is streaming from the Jellyfin that runs on a server, for the warning before it is restarted. */
|
||||||
|
jellyfinRouter.get("/watching", asyncHandler(async (req, res) => {
|
||||||
|
const serverId = idOf(String(req.query.serverId ?? ""));
|
||||||
|
if (!Number.isInteger(serverId)) return res.status(400).json({ error: "invalid_id" });
|
||||||
|
const watching = await watchingOnServer(serverId);
|
||||||
|
res.json({ watching, words: watchingWords(watching) });
|
||||||
|
}));
|
||||||
|
|
||||||
|
/** How many were streaming over time, as peaks (readings are kept 35 days). */
|
||||||
|
jellyfinRouter.get("/:id/history", asyncHandler(async (req, res) => {
|
||||||
|
const id = idOf(req.params.id);
|
||||||
|
if (!Number.isInteger(id)) return res.status(400).json({ error: "invalid_id" });
|
||||||
|
const [row] = await db.select({ id: integrations.id, name: integrations.name, type: integrations.type }).from(integrations).where(eq(integrations.id, id)).limit(1);
|
||||||
|
if (!row || row.type !== "jellyfin") return res.status(404).json({ error: "not_found" });
|
||||||
|
const asked = Number(req.query.days);
|
||||||
|
const days = Number.isFinite(asked) ? Math.min(READING_KEEP_DAYS, Math.max(1, Math.round(asked))) : 7;
|
||||||
|
const to = Date.now();
|
||||||
|
const from = to - days * DAY_MS;
|
||||||
|
const rows = await db.select().from(jellyfinReadings).where(and(eq(jellyfinReadings.integrationId, id), gte(jellyfinReadings.at, new Date(from).toISOString()))).orderBy(asc(jellyfinReadings.at));
|
||||||
|
res.json({ jellyfin: { id: row.id, name: row.name }, days, points: bucketPeaks(rows, from, to, 200) });
|
||||||
|
}));
|
||||||
@@ -18,6 +18,8 @@ const startSchema = z.object({
|
|||||||
templateId: z.number().int().positive(),
|
templateId: z.number().int().positive(),
|
||||||
limit: z.string().trim().regex(/^[A-Za-z0-9][A-Za-z0-9._,:*\-]{0,199}$/, "Use the host name as it is in the Semaphore inventory, like web01"),
|
limit: z.string().trim().regex(/^[A-Za-z0-9][A-Za-z0-9._,:*\-]{0,199}$/, "Use the host name as it is in the Semaphore inventory, like web01"),
|
||||||
windowMinutes: z.number().int().min(15).max(480).default(90),
|
windowMinutes: z.number().int().min(15).max(480).default(90),
|
||||||
|
// Somebody is streaming from the machine and the person has seen the warning.
|
||||||
|
startAnyway: z.boolean().optional(),
|
||||||
});
|
});
|
||||||
|
|
||||||
const shape = (r: Awaited<ReturnType<typeof listUpdateRuns>>[number]) => ({
|
const shape = (r: Awaited<ReturnType<typeof listUpdateRuns>>[number]) => ({
|
||||||
|
|||||||
@@ -75,6 +75,7 @@ const SOURCES: Record<string, { label: string; link: string }> = {
|
|||||||
ups: { label: "UPS", link: "/ups" },
|
ups: { label: "UPS", link: "/ups" },
|
||||||
slo: { label: "Service level", link: "/service-levels" },
|
slo: { label: "Service level", link: "/service-levels" },
|
||||||
duplicati: { label: "Duplicati", link: "/duplicati" },
|
duplicati: { label: "Duplicati", link: "/duplicati" },
|
||||||
|
jellyfin: { label: "Jellyfin", link: "/jellyfin" },
|
||||||
secrets: { label: "Secrets", link: "/secrets" },
|
secrets: { label: "Secrets", link: "/secrets" },
|
||||||
domains: { label: "Domains", link: "/dns?view=domains" },
|
domains: { label: "Domains", link: "/dns?view=domains" },
|
||||||
endpoint: { label: "Endpoint", link: "/endpoints" },
|
endpoint: { label: "Endpoint", link: "/endpoints" },
|
||||||
@@ -113,6 +114,8 @@ function originOfConditionKey(key: string): { type: string; id: number | null; c
|
|||||||
if (m) return { type: "dockhand", id: Number(m[1]), category: "services" };
|
if (m) return { type: "dockhand", id: Number(m[1]), category: "services" };
|
||||||
m = /^duplicati-(?:failed|stale):(\d+)/.exec(key);
|
m = /^duplicati-(?:failed|stale):(\d+)/.exec(key);
|
||||||
if (m) return { type: "duplicati", id: Number(m[1]), category: "backup" };
|
if (m) return { type: "duplicati", id: Number(m[1]), category: "backup" };
|
||||||
|
m = /^jellyfin-(?:restart|task|logins):(\d+)/.exec(key);
|
||||||
|
if (m) return { type: "jellyfin", id: Number(m[1]), category: key.startsWith("jellyfin-task") ? "automation" : "monitoring" };
|
||||||
return { type: "server", id: null, category: "disk" };
|
return { type: "server", id: null, category: "disk" };
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -235,7 +238,7 @@ export async function collectAlerts(): Promise<AlertsReport> {
|
|||||||
type: origin.type,
|
type: origin.type,
|
||||||
message: c.message,
|
message: c.message,
|
||||||
key: c.key,
|
key: c.key,
|
||||||
link: origin.type === "server" && origin.id !== null ? `/servers/${origin.id}${/^(failed-units|smart|zfs):/.test(c.key) ? "?view=health" : c.key.startsWith("new-port:") ? "?view=network" : c.key.startsWith("rule:") || c.key.startsWith("baseline:") ? "?view=history" : ""}` : origin.type === "unifi" ? "/network/devices" : origin.type === "ups" ? "/ups" : origin.type === "slo" ? "/service-levels" : origin.type === "dockhand" ? "/docker" : origin.type === "duplicati" ? "/duplicati" : undefined,
|
link: origin.type === "server" && origin.id !== null ? `/servers/${origin.id}${/^(failed-units|smart|zfs):/.test(c.key) ? "?view=health" : c.key.startsWith("new-port:") ? "?view=network" : c.key.startsWith("rule:") || c.key.startsWith("baseline:") ? "?view=history" : ""}` : origin.type === "unifi" ? "/network/devices" : origin.type === "ups" ? "/ups" : origin.type === "slo" ? "/service-levels" : origin.type === "dockhand" ? "/docker" : origin.type === "duplicati" ? "/duplicati" : origin.type === "jellyfin" ? "/jellyfin" : undefined,
|
||||||
silenced: subject !== null && subjects.has(subject), target: targetOfSubject(subject),
|
silenced: subject !== null && subjects.has(subject), target: targetOfSubject(subject),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -245,7 +248,7 @@ export async function collectAlerts(): Promise<AlertsReport> {
|
|||||||
if (held.has("slo")) cannotCheck("Service-level targets", "couldn't be judged — see the server log", "Server and storage health");
|
if (held.has("slo")) cannotCheck("Service-level targets", "couldn't be judged — see the server log", "Server and storage health");
|
||||||
if (held.has("server:baseline")) cannotCheck("What is usual for the time", "couldn't be judged — see the server log", "Server and storage health");
|
if (held.has("server:baseline")) cannotCheck("What is usual for the time", "couldn't be judged — see the server log", "Server and storage health");
|
||||||
for (const h of held) {
|
for (const h of held) {
|
||||||
const m = /^(proxmox|synology|unifi|ups|dockhand):(\d+)(?::(.+))?$/.exec(h);
|
const m = /^(proxmox|synology|unifi|ups|dockhand|jellyfin):(\d+)(?::(.+))?$/.exec(h);
|
||||||
if (!m || (m[3] && held.has(`${m[1]}:${m[2]}`))) continue;
|
if (!m || (m[3] && held.has(`${m[1]}:${m[2]}`))) continue;
|
||||||
const row = intById.get(Number(m[2]));
|
const row = intById.get(Number(m[2]));
|
||||||
cannotCheck(`${SOURCES[m[1]].label} “${row?.name ?? `#${m[2]}`}”${m[3] ? ` (node ${m[3]})` : ""}`, "couldn't be read — see the Diagnostic Log", "Server and storage health");
|
cannotCheck(`${SOURCES[m[1]].label} “${row?.name ?? `#${m[2]}`}”${m[3] ? ` (node ${m[3]})` : ""}`, "couldn't be read — see the Diagnostic Log", "Server and storage health");
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import { and, eq } from "drizzle-orm";
|
import { and, eq } from "drizzle-orm";
|
||||||
import { db } from "../db/client.js";
|
import { db } from "../db/client.js";
|
||||||
import { integrations, networkDevices, serverPorts, servers, upsState } from "../db/schema.js";
|
import { integrations, jellyfinState, networkDevices, serverPorts, servers, upsState } from "../db/schema.js";
|
||||||
import { loadIntegrationConfig } from "../integrations/loadIntegration.js";
|
import { loadIntegrationConfig } from "../integrations/loadIntegration.js";
|
||||||
import { createProxmoxAdapter, type ProxmoxNodeStats } from "../integrations/proxmox/adapter.js";
|
import { createProxmoxAdapter, type ProxmoxNodeStats } from "../integrations/proxmox/adapter.js";
|
||||||
import { createSynologyAdapter, type SynologyStorageInfo } from "../integrations/synology/adapter.js";
|
import { createSynologyAdapter, type SynologyStorageInfo } from "../integrations/synology/adapter.js";
|
||||||
@@ -20,6 +20,7 @@ import { baselineKey, computeBaselineFindings, type BaselineFinding } from "./ba
|
|||||||
import { levelFindings, levelKey, type LevelFinding } from "./serviceLevels.js";
|
import { levelFindings, levelKey, type LevelFinding } from "./serviceLevels.js";
|
||||||
import { createDockhandAdapter } from "../integrations/dockhand/adapter.js";
|
import { createDockhandAdapter } from "../integrations/dockhand/adapter.js";
|
||||||
import { currentProblems as duplicatiProblemsNow, type DuplicatiProblem } from "./duplicati.js";
|
import { currentProblems as duplicatiProblemsNow, type DuplicatiProblem } from "./duplicati.js";
|
||||||
|
import { isJellyfinFresh, jellyfinProblems, jellyfinSyncStatuses, type JellyfinProblem } from "./jellyfinSync.js";
|
||||||
import { evaluateContainers, loadContainerMemory, saveContainerMemory, type ContainerMemory, type DockerSnapshot } from "./containerHealth.js";
|
import { evaluateContainers, loadContainerMemory, saveContainerMemory, type ContainerMemory, type DockerSnapshot } from "./containerHealth.js";
|
||||||
import { recordStorageSamples } from "./storageHistory.js";
|
import { recordStorageSamples } from "./storageHistory.js";
|
||||||
import { isUpsFresh, upsProblems, upsSyncStatuses, type UpsStateRow } from "./upsSync.js";
|
import { isUpsFresh, upsProblems, upsSyncStatuses, type UpsStateRow } from "./upsSync.js";
|
||||||
@@ -87,6 +88,8 @@ export interface HealthSnapshot {
|
|||||||
baselines?: BaselineFinding[];
|
baselines?: BaselineFinding[];
|
||||||
/** Service-level targets whose error budget for the month is mostly or wholly used (services/serviceLevels.ts). */
|
/** Service-level targets whose error budget for the month is mostly or wholly used (services/serviceLevels.ts). */
|
||||||
levels?: LevelFinding[];
|
levels?: LevelFinding[];
|
||||||
|
/** What is wrong with each Jellyfin, from what was stored, only while that reading is recent (services/jellyfinSync.ts). */
|
||||||
|
jellyfin?: { integrationId: number; problems: JellyfinProblem[] }[];
|
||||||
/** Duplicati backups that failed or have not succeeded in time (services/duplicati.ts). */
|
/** Duplicati backups that failed or have not succeeded in time (services/duplicati.ts). */
|
||||||
duplicati?: DuplicatiProblem[];
|
duplicati?: DuplicatiProblem[];
|
||||||
/** Each Docker integration's containers as last read, with what was remembered of them (services/containerHealth.ts). */
|
/** Each Docker integration's containers as last read, with what was remembered of them (services/containerHealth.ts). */
|
||||||
@@ -198,6 +201,9 @@ export function evaluateHealth(snapshot: HealthSnapshot, thresholds: Thresholds,
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A Jellyfin that wants a restart, a scheduled task that failed, a burst of failed sign-ins.
|
||||||
|
for (const j of snapshot.jellyfin ?? []) for (const p of j.problems) out.push({ key: p.key, source: `jellyfin:${j.integrationId}`, message: p.message, severity: p.severity });
|
||||||
|
|
||||||
// Duplicati backups that failed, or have not succeeded for longer than they should.
|
// Duplicati backups that failed, or have not succeeded for longer than they should.
|
||||||
for (const p of snapshot.duplicati ?? []) out.push({ key: p.key, source: "duplicati", message: p.message, severity: p.severity });
|
for (const p of snapshot.duplicati ?? []) out.push({ key: p.key, source: "duplicati", message: p.message, severity: p.severity });
|
||||||
|
|
||||||
@@ -428,6 +434,24 @@ export async function collectSnapshot(): Promise<{ snapshot: HealthSnapshot; hel
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Jellyfin: read by its own pass every couple of minutes; what is judged is what that pass stored, and only while it is recent.
|
||||||
|
try {
|
||||||
|
const rowsJ = await db.select({ id: integrations.id, name: integrations.name }).from(integrations).where(and(eq(integrations.type, "jellyfin"), eq(integrations.enabled, true)));
|
||||||
|
if (rowsJ.length > 0) {
|
||||||
|
const statuses = await jellyfinSyncStatuses();
|
||||||
|
for (const r of rowsJ) {
|
||||||
|
const [state] = await db.select().from(jellyfinState).where(eq(jellyfinState.integrationId, r.id)).limit(1);
|
||||||
|
if (!state || !isJellyfinFresh(statuses[r.id], Date.now())) {
|
||||||
|
held.add(`jellyfin:${r.id}`);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
(snapshot.jellyfin ??= []).push({ integrationId: r.id, problems: jellyfinProblems(r.id, r.name, state) });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
console.error("[health] couldn't judge Jellyfin:", err instanceof Error ? err.message : err);
|
||||||
|
}
|
||||||
|
|
||||||
// Duplicati: what its reports say, read from what is stored (nothing is asked of Duplicati). A failure here costs only this check.
|
// Duplicati: what its reports say, read from what is stored (nothing is asked of Duplicati). A failure here costs only this check.
|
||||||
try {
|
try {
|
||||||
snapshot.duplicati = await duplicatiProblemsNow(Date.now());
|
snapshot.duplicati = await duplicatiProblemsNow(Date.now());
|
||||||
|
|||||||
@@ -0,0 +1,229 @@
|
|||||||
|
/**
|
||||||
|
* Keeps what Jellyfin says up to date: every two minutes each Jellyfin integration is read (its version, who is streaming, how its scheduled tasks ended, how many sign-ins
|
||||||
|
* failed), the state is stored, and the number streaming is added to a history. Pages, charts and alerts are built from what is stored, so none of them talks to Jellyfin.
|
||||||
|
*
|
||||||
|
* Two things come of it besides a page. **Alerts**: a server that wants a restart, a scheduled task (a library scan) that failed, a burst of failed sign-ins. And **who is
|
||||||
|
* watching**: before an update run or a patch plan restarts the machine Jellyfin runs on, the app can say that people are streaming from it — and a patch plan waits for them to
|
||||||
|
* finish, for a while, instead of cutting a film off.
|
||||||
|
*/
|
||||||
|
import { and, desc, eq, lt } from "drizzle-orm";
|
||||||
|
import { db } from "../db/client.js";
|
||||||
|
import { integrations, jellyfinReadings, jellyfinState, servers } from "../db/schema.js";
|
||||||
|
import { createJellyfinAdapter, type JellyfinRead, type JellyfinStream, type JellyfinTask } from "../integrations/jellyfin/adapter.js";
|
||||||
|
import { loadIntegrationConfig } from "../integrations/loadIntegration.js";
|
||||||
|
import { hostOfAddress } from "./kumaTargets.js";
|
||||||
|
import { matchServerForTarget } from "./uptimeKumaMatch.js";
|
||||||
|
import { getInternalFlag, setInternalFlag } from "./settingsStore.js";
|
||||||
|
|
||||||
|
export const SYNC_INTERVAL_MS = 2 * 60_000;
|
||||||
|
/** After this long without a good read, what is stored is no longer believed. */
|
||||||
|
export const STALE_AFTER_MS = 10 * 60_000;
|
||||||
|
export const READING_KEEP_DAYS = 35;
|
||||||
|
const READING_GAP_MS = 5 * 60_000 - 20_000;
|
||||||
|
/** This many failed sign-ins in a day is a warning. */
|
||||||
|
export const FAILED_LOGINS_WARN = 10;
|
||||||
|
const STATUS_FLAG = "jellyfinSync";
|
||||||
|
|
||||||
|
export interface JellyfinSyncStatus {
|
||||||
|
/** The last read that worked (kept when a later one failed). */
|
||||||
|
at: string;
|
||||||
|
ok: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readStatuses(): Promise<Record<string, JellyfinSyncStatus>> {
|
||||||
|
try {
|
||||||
|
const raw = await getInternalFlag(STATUS_FLAG);
|
||||||
|
return raw ? (JSON.parse(raw) as Record<string, JellyfinSyncStatus>) : {};
|
||||||
|
} catch {
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function jellyfinSyncStatuses(): Promise<Record<number, JellyfinSyncStatus>> {
|
||||||
|
return Object.fromEntries(Object.entries(await readStatuses()).map(([k, v]) => [Number(k), v]));
|
||||||
|
}
|
||||||
|
|
||||||
|
export const isJellyfinFresh = (status: JellyfinSyncStatus | undefined, nowMs: number) => !!status && status.ok && nowMs - Date.parse(status.at) <= STALE_AFTER_MS;
|
||||||
|
|
||||||
|
export type StateRow = typeof jellyfinState.$inferSelect;
|
||||||
|
|
||||||
|
const parse = <T>(text: string): T[] => {
|
||||||
|
try {
|
||||||
|
const v = JSON.parse(text);
|
||||||
|
return Array.isArray(v) ? (v as T[]) : [];
|
||||||
|
} catch {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
};
|
||||||
|
export const streamsOf = (row: Pick<StateRow, "streams">) => parse<JellyfinStream>(row.streams);
|
||||||
|
export const tasksOf = (row: Pick<StateRow, "tasks">) => parse<JellyfinTask>(row.tasks);
|
||||||
|
|
||||||
|
/** Writes down one read: the state, and a point in the history when it is time (or when the number streaming has changed). */
|
||||||
|
export async function storeRead(integrationId: number, r: JellyfinRead, now: Date = new Date()): Promise<void> {
|
||||||
|
const nowIso = now.toISOString();
|
||||||
|
const values = {
|
||||||
|
serverName: r.name,
|
||||||
|
version: r.version,
|
||||||
|
os: r.os,
|
||||||
|
pendingRestart: r.pendingRestart,
|
||||||
|
updateAvailable: r.updateAvailable,
|
||||||
|
streams: JSON.stringify(r.streams),
|
||||||
|
tasks: JSON.stringify(r.tasks),
|
||||||
|
movies: r.counts.movies,
|
||||||
|
series: r.counts.series,
|
||||||
|
episodes: r.counts.episodes,
|
||||||
|
songs: r.counts.songs,
|
||||||
|
failedLogins24h: r.failedLogins24h,
|
||||||
|
updatedAt: nowIso,
|
||||||
|
};
|
||||||
|
await db.insert(jellyfinState).values({ integrationId, ...values }).onConflictDoUpdate({ target: jellyfinState.integrationId, set: values });
|
||||||
|
const transcodes = r.streams.filter((s) => s.transcoding).length;
|
||||||
|
const [newest] = await db.select({ at: jellyfinReadings.at, streams: jellyfinReadings.streams, transcodes: jellyfinReadings.transcodes }).from(jellyfinReadings).where(eq(jellyfinReadings.integrationId, integrationId)).orderBy(desc(jellyfinReadings.at)).limit(1);
|
||||||
|
const due = !newest || newest.streams !== r.streams.length || newest.transcodes !== transcodes || now.getTime() - Date.parse(newest.at) >= READING_GAP_MS;
|
||||||
|
if (due) {
|
||||||
|
await db.insert(jellyfinReadings).values({ integrationId, at: nowIso, streams: r.streams.length, transcodes });
|
||||||
|
}
|
||||||
|
await db.delete(jellyfinReadings).where(and(eq(jellyfinReadings.integrationId, integrationId), lt(jellyfinReadings.at, new Date(now.getTime() - READING_KEEP_DAYS * 86_400_000).toISOString())));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Reads one Jellyfin and stores it. Never throws: how it went is the answer, and what was stored is left alone when it failed. */
|
||||||
|
export async function syncJellyfin(integrationId: number, now: Date = new Date(), read?: (config: Record<string, unknown>) => Promise<JellyfinRead>): Promise<JellyfinSyncStatus> {
|
||||||
|
let status: JellyfinSyncStatus;
|
||||||
|
try {
|
||||||
|
const loaded = await loadIntegrationConfig(integrationId);
|
||||||
|
if (!loaded || loaded.integration.type !== "jellyfin") throw new Error("not a Jellyfin integration");
|
||||||
|
const result = await (read ? read(loaded.config) : createJellyfinAdapter(loaded.config as never).read(now));
|
||||||
|
await storeRead(integrationId, result, now);
|
||||||
|
status = { at: now.toISOString(), ok: true, error: null };
|
||||||
|
} catch (err) {
|
||||||
|
const previous = (await readStatuses())[String(integrationId)];
|
||||||
|
status = { at: previous?.at ?? now.toISOString(), ok: false, error: err instanceof Error ? err.message : String(err) };
|
||||||
|
console.error(`[jellyfin] couldn't read integration ${integrationId}:`, status.error);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const all = await readStatuses();
|
||||||
|
all[String(integrationId)] = status;
|
||||||
|
await setInternalFlag(STATUS_FLAG, JSON.stringify(all));
|
||||||
|
} catch (err) {
|
||||||
|
console.error("[jellyfin] couldn't record the result:", err);
|
||||||
|
}
|
||||||
|
return status;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function syncAllJellyfin(now: Date = new Date()): Promise<Record<number, JellyfinSyncStatus>> {
|
||||||
|
const rows = await db.select({ id: integrations.id }).from(integrations).where(and(eq(integrations.type, "jellyfin"), eq(integrations.enabled, true)));
|
||||||
|
const out: Record<number, JellyfinSyncStatus> = {};
|
||||||
|
for (const r of rows) out[r.id] = await syncJellyfin(r.id, now);
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function forgetJellyfin(integrationId: number): Promise<void> {
|
||||||
|
const all = await readStatuses();
|
||||||
|
if (String(integrationId) in all) {
|
||||||
|
delete all[String(integrationId)];
|
||||||
|
await setInternalFlag(STATUS_FLAG, JSON.stringify(all));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── What is wrong ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
export interface JellyfinProblem {
|
||||||
|
key: string;
|
||||||
|
message: string;
|
||||||
|
severity: "warning";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pure: what is wrong with one Jellyfin, from what is stored for it. */
|
||||||
|
export function jellyfinProblems(id: number, name: string, s: Pick<StateRow, "pendingRestart" | "tasks" | "failedLogins24h">): JellyfinProblem[] {
|
||||||
|
const out: JellyfinProblem[] = [];
|
||||||
|
if (s.pendingRestart) out.push({ key: `jellyfin-restart:${id}`, severity: "warning", message: `${name} is waiting to be restarted — a plugin or an update needs it` });
|
||||||
|
for (const t of tasksOf(s)) {
|
||||||
|
if (t.lastStatus === "Failed") out.push({ key: `jellyfin-task:${id}:${t.name}`, severity: "warning", message: `${name}: the scheduled task “${t.name}” failed${t.error ? ` — ${t.error}` : ""}` });
|
||||||
|
}
|
||||||
|
if (s.failedLogins24h !== null && s.failedLogins24h >= FAILED_LOGINS_WARN) out.push({ key: `jellyfin-logins:${id}`, severity: "warning", message: `${name}: ${s.failedLogins24h} sign-ins failed in the last day` });
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Who is watching ────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
export interface Watching {
|
||||||
|
integrationId: number;
|
||||||
|
integrationName: string;
|
||||||
|
streams: JellyfinStream[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pure: a sentence about who is watching, for a warning or a waiting message: "2 people are streaming from Jellyfin (Ann: Dune, Bo: The Office — S1E2)". */
|
||||||
|
export function watchingWords(w: Watching[]): string {
|
||||||
|
const streams = w.flatMap((x) => x.streams);
|
||||||
|
if (streams.length === 0) return "";
|
||||||
|
const names = w.map((x) => x.integrationName);
|
||||||
|
const who = streams.slice(0, 3).map((s) => `${s.user}: ${s.title}`).join(", ");
|
||||||
|
const rest = streams.length > 3 ? ` and ${streams.length - 3} more` : "";
|
||||||
|
const transcoding = streams.filter((s) => s.transcoding).length;
|
||||||
|
return `${streams.length === 1 ? "1 person is" : `${streams.length} people are`} streaming from ${[...new Set(names)].join(" and ")} (${who}${rest})${transcoding > 0 ? `, ${transcoding} being transcoded` : ""}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The Jellyfins that run on this server — the one whose address is the server's — and who is streaming from each, as last read and believed. Only those with somebody streaming are listed. */
|
||||||
|
export async function watchingOnServer(serverId: number, now: Date = new Date()): Promise<Watching[]> {
|
||||||
|
const rows = await db.select({ id: integrations.id, name: integrations.name, baseUrl: integrations.baseUrl }).from(integrations).where(and(eq(integrations.type, "jellyfin"), eq(integrations.enabled, true)));
|
||||||
|
if (rows.length === 0) return [];
|
||||||
|
const statuses = await jellyfinSyncStatuses();
|
||||||
|
const serverRows = await db.select({ id: servers.id, name: servers.name, hostname: servers.hostname, ipAddresses: servers.ipAddresses }).from(servers);
|
||||||
|
const matchable = serverRows.map((s) => {
|
||||||
|
let ips: string[] = [];
|
||||||
|
try {
|
||||||
|
const v = JSON.parse(s.ipAddresses ?? "[]");
|
||||||
|
ips = Array.isArray(v) ? v.filter((x): x is string => typeof x === "string") : [];
|
||||||
|
} catch {
|
||||||
|
// no addresses
|
||||||
|
}
|
||||||
|
return { id: s.id, name: s.name, hostname: s.hostname, ips };
|
||||||
|
});
|
||||||
|
const out: Watching[] = [];
|
||||||
|
for (const r of rows) {
|
||||||
|
if (!isJellyfinFresh(statuses[r.id], now.getTime())) continue;
|
||||||
|
if (matchServerForTarget(hostOfAddress(r.baseUrl), matchable)?.id !== serverId) continue;
|
||||||
|
const [state] = await db.select().from(jellyfinState).where(eq(jellyfinState.integrationId, r.id)).limit(1);
|
||||||
|
const streams = state ? streamsOf(state) : [];
|
||||||
|
if (streams.length > 0) out.push({ integrationId: r.id, integrationName: r.name, streams });
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PeakPoint {
|
||||||
|
/** ISO. */
|
||||||
|
t: string;
|
||||||
|
/** The most that were streaming, and the most transcoded, in this stretch. */
|
||||||
|
streams: number;
|
||||||
|
transcodes: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pure: readings into at most `maxPoints` equal slices of [from, to], each as its peak — an average would hide the evening somebody watched. */
|
||||||
|
export function bucketPeaks(rows: { at: string; streams: number; transcodes: number }[], from: number, to: number, maxPoints: number): PeakPoint[] {
|
||||||
|
if (rows.length === 0 || to <= from) return [];
|
||||||
|
const size = Math.max((to - from) / maxPoints, 1);
|
||||||
|
const buckets = new Map<number, { tSum: number; n: number; streams: number; transcodes: number }>();
|
||||||
|
for (const r of rows) {
|
||||||
|
const t = Date.parse(r.at);
|
||||||
|
if (t < from || t > to) continue;
|
||||||
|
const key = Math.min(maxPoints - 1, Math.floor((t - from) / size));
|
||||||
|
const b = buckets.get(key) ?? { tSum: 0, n: 0, streams: 0, transcodes: 0 };
|
||||||
|
b.tSum += t;
|
||||||
|
b.n += 1;
|
||||||
|
b.streams = Math.max(b.streams, r.streams);
|
||||||
|
b.transcodes = Math.max(b.transcodes, r.transcodes);
|
||||||
|
buckets.set(key, b);
|
||||||
|
}
|
||||||
|
return [...buckets.entries()].sort((a, b) => a[0] - b[0]).map(([, b]) => ({ t: new Date(b.tSum / b.n).toISOString(), streams: b.streams, transcodes: b.transcodes }));
|
||||||
|
}
|
||||||
|
|
||||||
|
let timer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
|
||||||
|
export function initJellyfinSync(): void {
|
||||||
|
if (timer) return;
|
||||||
|
const pass = () => void syncAllJellyfin().catch((err) => console.error("[jellyfin] pass failed:", err));
|
||||||
|
timer = setInterval(pass, SYNC_INTERVAL_MS);
|
||||||
|
setTimeout(pass, 11_000);
|
||||||
|
console.log(`[jellyfin] reading Jellyfin every ${SYNC_INTERVAL_MS / 60000} minutes`);
|
||||||
|
}
|
||||||
@@ -31,6 +31,8 @@ export function subjectOfConditionKey(key: string): string | null {
|
|||||||
if (m) return `server:${m[1]}`;
|
if (m) return `server:${m[1]}`;
|
||||||
m = /^container-(?:restarting|unhealthy|stopped|created|host):(\d+):/.exec(key);
|
m = /^container-(?:restarting|unhealthy|stopped|created|host):(\d+):/.exec(key);
|
||||||
if (m) return `integration:${m[1]}`;
|
if (m) return `integration:${m[1]}`;
|
||||||
|
m = /^jellyfin-(?:restart|task|logins):(\d+)/.exec(key);
|
||||||
|
if (m) return `integration:${m[1]}`;
|
||||||
m = /^duplicati-(?:failed|stale):\d+:server:(\d+)$/.exec(key);
|
m = /^duplicati-(?:failed|stale):\d+:server:(\d+)$/.exec(key);
|
||||||
if (m) return `server:${m[1]}`;
|
if (m) return `server:${m[1]}`;
|
||||||
m = /^(?:endpoint(?:-cert)?|heartbeat(?:-fail)?):\d+:server:(\d+)$/.exec(key);
|
m = /^(?:endpoint(?:-cert)?|heartbeat(?:-fail)?):\d+:server:(\d+)$/.exec(key);
|
||||||
|
|||||||
@@ -5,10 +5,11 @@
|
|||||||
*/
|
*/
|
||||||
import { eq } from "drizzle-orm";
|
import { eq } from "drizzle-orm";
|
||||||
import { db } from "../db/client.js";
|
import { db } from "../db/client.js";
|
||||||
import { assets, endpointChecks, heartbeats, integrations, maintenanceWindows, secrets, servers, upsState } from "../db/schema.js";
|
import { assets, endpointChecks, heartbeats, integrations, maintenanceWindows, secrets, servers, upsState, jellyfinState } from "../db/schema.js";
|
||||||
import { isUpsFresh, upsSyncStatuses } from "./upsSync.js";
|
import { isUpsFresh, upsSyncStatuses } from "./upsSync.js";
|
||||||
import { levelStatuses } from "./serviceLevels.js";
|
import { levelStatuses } from "./serviceLevels.js";
|
||||||
import { backupState, listAll as listDuplicati } from "./duplicati.js";
|
import { backupState, listAll as listDuplicati } from "./duplicati.js";
|
||||||
|
import { isJellyfinFresh, jellyfinSyncStatuses, streamsOf } from "./jellyfinSync.js";
|
||||||
import { latestTests } from "./restoreTests.js";
|
import { latestTests } from "./restoreTests.js";
|
||||||
import { heartbeatState, parseStamp } from "./heartbeats.js";
|
import { heartbeatState, parseStamp } from "./heartbeats.js";
|
||||||
import { env } from "../env.js";
|
import { env } from "../env.js";
|
||||||
@@ -60,6 +61,21 @@ class Exposition {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Each Jellyfin, for /metrics: how many are streaming, how many of those are transcoded, and whether it wants a restart. Only while the reading is recent. */
|
||||||
|
async function addJellyfinMetrics(out: Exposition, now: Date) {
|
||||||
|
const statuses = await jellyfinSyncStatuses();
|
||||||
|
const names = new Map((await db.select({ id: integrations.id, name: integrations.name, enabled: integrations.enabled }).from(integrations).where(eq(integrations.type, "jellyfin"))).map((r) => [r.id, r]));
|
||||||
|
for (const s of await db.select().from(jellyfinState)) {
|
||||||
|
const row = names.get(s.integrationId);
|
||||||
|
if (!row || !row.enabled || !isJellyfinFresh(statuses[s.integrationId], now.getTime())) continue;
|
||||||
|
const labels = { name: row.name };
|
||||||
|
const streams = streamsOf(s);
|
||||||
|
out.add("homelab_jellyfin_streams", "gauge", "How many are streaming from Jellyfin right now.", streams.length, labels);
|
||||||
|
out.add("homelab_jellyfin_transcodes", "gauge", "How many of those streams Jellyfin is transcoding.", streams.filter((x) => x.transcoding).length, labels);
|
||||||
|
out.add("homelab_jellyfin_pending_restart", "gauge", "1 if Jellyfin is waiting to be restarted, else 0.", s.pendingRestart ? 1 : 0, labels);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** Each Duplicati backup, for /metrics: whether it is in order, how long since it last succeeded, and how big what it backs up is. */
|
/** Each Duplicati backup, for /metrics: whether it is in order, how long since it last succeeded, and how big what it backs up is. */
|
||||||
async function addDuplicatiMetrics(out: Exposition, now: Date) {
|
async function addDuplicatiMetrics(out: Exposition, now: Date) {
|
||||||
const { sources, backups } = await listDuplicati();
|
const { sources, backups } = await listDuplicati();
|
||||||
@@ -267,6 +283,7 @@ export async function buildMetrics(now: Date = new Date()): Promise<string> {
|
|||||||
await addUpsMetrics(out, now);
|
await addUpsMetrics(out, now);
|
||||||
await addServiceLevelMetrics(out, now);
|
await addServiceLevelMetrics(out, now);
|
||||||
await addDuplicatiMetrics(out, now);
|
await addDuplicatiMetrics(out, now);
|
||||||
|
await addJellyfinMetrics(out, now);
|
||||||
await addHeartbeatMetrics(out, now);
|
await addHeartbeatMetrics(out, now);
|
||||||
return out.toString();
|
return out.toString();
|
||||||
}
|
}
|
||||||
@@ -17,6 +17,7 @@ import { invalidateAlertsCache } from "./alerts.js";
|
|||||||
import { recordAudit } from "./audit.js";
|
import { recordAudit } from "./audit.js";
|
||||||
import { impactOf } from "./dependencies.js";
|
import { impactOf } from "./dependencies.js";
|
||||||
import { startOrExtendWindow } from "./maintenance.js";
|
import { startOrExtendWindow } from "./maintenance.js";
|
||||||
|
import { watchingOnServer, watchingWords } from "./jellyfinSync.js";
|
||||||
import { occurrencesBetween, type Occurrence } from "./maintenanceSchedules.js";
|
import { occurrencesBetween, type Occurrence } from "./maintenanceSchedules.js";
|
||||||
import { notifyPatchPlanFinished } from "./notify.js";
|
import { notifyPatchPlanFinished } from "./notify.js";
|
||||||
import { parseTags } from "./serverTags.js";
|
import { parseTags } from "./serverTags.js";
|
||||||
@@ -36,8 +37,13 @@ export interface PlanItem {
|
|||||||
message: string | null;
|
message: string | null;
|
||||||
/** Windows made for what depends on this server, which end with it. */
|
/** Windows made for what depends on this server, which end with it. */
|
||||||
windowIds: number[];
|
windowIds: number[];
|
||||||
|
/** When its turn came and somebody was streaming from it, so it was left a while (ISO). */
|
||||||
|
waitingSince?: string | null;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** How long a server is left, when it is its turn and people are streaming from it, before it is updated anyway. */
|
||||||
|
export const WATCHING_WAIT_MS = 2 * 3_600_000;
|
||||||
|
|
||||||
/** How late a plan may still start, if the app was not running at its time. */
|
/** How late a plan may still start, if the app was not running at its time. */
|
||||||
export const START_GRACE_MS = 30 * 60_000;
|
export const START_GRACE_MS = 30 * 60_000;
|
||||||
export const POLL_INTERVAL_MS = 20_000;
|
export const POLL_INTERVAL_MS = 20_000;
|
||||||
@@ -217,6 +223,19 @@ async function advanceRun(run: PlanRunRow, now: Date): Promise<void> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
const actor = `Patch plan "${plan.name}"`;
|
const actor = `Patch plan "${plan.name}"`;
|
||||||
|
// People streaming from it (Jellyfin on this machine): its turn waits, up to two hours, rather than cutting a film off.
|
||||||
|
const watching = await watchingOnServer(server.id, now);
|
||||||
|
let waited = "";
|
||||||
|
if (watching.length > 0) {
|
||||||
|
const since = next.waitingSince ? Date.parse(next.waitingSince) : now.getTime();
|
||||||
|
if (!next.waitingSince) next.waitingSince = now.toISOString();
|
||||||
|
if (now.getTime() - since < WATCHING_WAIT_MS) {
|
||||||
|
next.message = `Waiting: ${watchingWords(watching)}.`;
|
||||||
|
await saveRun(run, items);
|
||||||
|
return; // looked at again at the next pass
|
||||||
|
}
|
||||||
|
waited = `Went ahead after waiting ${Math.round((now.getTime() - since) / 3_600_000)} h; ${watchingWords(watching)}. `;
|
||||||
|
}
|
||||||
try {
|
try {
|
||||||
const ur = await startUpdateRun(
|
const ur = await startUpdateRun(
|
||||||
{
|
{
|
||||||
@@ -227,10 +246,11 @@ async function advanceRun(run: PlanRunRow, now: Date): Promise<void> {
|
|||||||
limit: plan.limitFrom === "hostname" && server.hostname ? server.hostname : server.name,
|
limit: plan.limitFrom === "hostname" && server.hostname ? server.hostname : server.name,
|
||||||
windowMinutes: plan.windowMinutes,
|
windowMinutes: plan.windowMinutes,
|
||||||
actor,
|
actor,
|
||||||
|
startAnyway: true,
|
||||||
},
|
},
|
||||||
now,
|
now,
|
||||||
);
|
);
|
||||||
Object.assign(next, { status: "running", updateRunId: ur.id, message: "The update run has started.", windowIds: await silenceDependents(plan, server.id, server.name, actor, now) });
|
Object.assign(next, { status: "running", updateRunId: ur.id, message: `${waited}The update run has started.`, windowIds: await silenceDependents(plan, server.id, server.name, actor, now) });
|
||||||
await saveRun(run, items);
|
await saveRun(run, items);
|
||||||
return; // one at a time: this one is followed until it is over
|
return; // one at a time: this one is followed until it is over
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ import { createSemaphoreAdapter, type SemaphoreAdapter, type SemaphoreTask } fro
|
|||||||
import { invalidateAlertsCache } from "./alerts.js";
|
import { invalidateAlertsCache } from "./alerts.js";
|
||||||
import { recordAudit } from "./audit.js";
|
import { recordAudit } from "./audit.js";
|
||||||
import { startOrExtendWindow } from "./maintenance.js";
|
import { startOrExtendWindow } from "./maintenance.js";
|
||||||
|
import { watchingOnServer, watchingWords } from "./jellyfinSync.js";
|
||||||
import { notifyUpdateRunFinished } from "./notify.js";
|
import { notifyUpdateRunFinished } from "./notify.js";
|
||||||
import { parsePendingUpdates } from "./pendingUpdates.js";
|
import { parsePendingUpdates } from "./pendingUpdates.js";
|
||||||
|
|
||||||
@@ -68,6 +69,8 @@ export interface StartInput {
|
|||||||
/** How long the maintenance window lasts at most; it ends sooner when the run does. */
|
/** How long the maintenance window lasts at most; it ends sooner when the run does. */
|
||||||
windowMinutes: number;
|
windowMinutes: number;
|
||||||
actor: string;
|
actor: string;
|
||||||
|
/** Go ahead although somebody is streaming from the machine (a person who has seen the warning, or a plan that has waited long enough). */
|
||||||
|
startAnyway?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function startUpdateRun(input: StartInput, now: Date = new Date()): Promise<UpdateRunRow> {
|
export async function startUpdateRun(input: StartInput, now: Date = new Date()): Promise<UpdateRunRow> {
|
||||||
@@ -82,6 +85,12 @@ export async function startUpdateRun(input: StartInput, now: Date = new Date()):
|
|||||||
.limit(1);
|
.limit(1);
|
||||||
if (busy) throw new UpdateRunError(409, "already_running", `An update run for ${server.name} is already under way.`);
|
if (busy) throw new UpdateRunError(409, "already_running", `An update run for ${server.name} is already under way.`);
|
||||||
|
|
||||||
|
// Jellyfin on this machine, with people watching: the run may restart it. Said, and the person asked again, rather than cutting a film off.
|
||||||
|
if (!input.startAnyway) {
|
||||||
|
const watching = await watchingOnServer(server.id, now);
|
||||||
|
if (watching.length > 0) throw new UpdateRunError(409, "watching", `${watchingWords(watching)}. Wait until they have finished, or start it anyway.`);
|
||||||
|
}
|
||||||
|
|
||||||
const adapter = await semaphoreFor(input.integrationId);
|
const adapter = await semaphoreFor(input.integrationId);
|
||||||
if (!adapter) throw new UpdateRunError(400, "no_semaphore", "That Semaphore integration doesn't exist or isn't switched on.");
|
if (!adapter) throw new UpdateRunError(400, "no_semaphore", "That Semaphore integration doesn't exist or isn't switched on.");
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,337 @@
|
|||||||
|
/**
|
||||||
|
* Jellyfin: reading what its API says, what is stored and when a history point is added, what is wrong with it, who is watching (and the warning before an update run
|
||||||
|
* restarts the machine), the health pass, the page's routes and /metrics.
|
||||||
|
*
|
||||||
|
* Run with: npm test --workspace server
|
||||||
|
*/
|
||||||
|
import { test, describe, before, after } from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { createServer as createHttpServer, type Server } from "node:http";
|
||||||
|
import { mkdtempSync, rmSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import type { AddressInfo } from "node:net";
|
||||||
|
|
||||||
|
const dir = mkdtempSync(join(tmpdir(), "hm-jellyfin-"));
|
||||||
|
process.env.DATABASE_PATH = join(dir, "test.sqlite");
|
||||||
|
process.env.SESSION_DIR = join(dir, "sessions");
|
||||||
|
process.env.CREDENTIALS_ENCRYPTION_KEY = "ab".repeat(32);
|
||||||
|
process.env.HEALTH_STARTUP_GRACE_MS = "0";
|
||||||
|
|
||||||
|
const express = (await import("express")).default;
|
||||||
|
const { db } = await import("../src/db/client.js");
|
||||||
|
const { users, servers, integrations, jellyfinReadings, jellyfinState } = await import("../src/db/schema.js");
|
||||||
|
const { runMigrations } = await import("../src/db/migrate.js");
|
||||||
|
const adapter = await import("../src/integrations/jellyfin/adapter.js");
|
||||||
|
const jf = await import("../src/services/jellyfinSync.js");
|
||||||
|
const { evaluateHealth, collectSnapshot } = await import("../src/services/healthMonitor.js");
|
||||||
|
const { subjectOfConditionKey } = await import("../src/services/maintenance.js");
|
||||||
|
const { getAlerts, invalidateAlertsCache } = await import("../src/services/alerts.js");
|
||||||
|
const { startUpdateRun, UpdateRunError } = await import("../src/services/updateRuns.js");
|
||||||
|
const { jellyfinRouter } = await import("../src/routes/jellyfin.js");
|
||||||
|
const { buildMetrics } = await import("../src/services/metrics.js");
|
||||||
|
const { setInternalFlag } = await import("../src/services/settingsStore.js");
|
||||||
|
const { eq } = await import("drizzle-orm");
|
||||||
|
|
||||||
|
await runMigrations();
|
||||||
|
|
||||||
|
const MIN = 60_000;
|
||||||
|
const DAY = 86_400_000;
|
||||||
|
|
||||||
|
const SESSIONS = [
|
||||||
|
{ UserName: "Ann", Client: "Jellyfin Web", DeviceName: "Firefox", NowPlayingItem: { Name: "Dune" }, PlayState: { PlayMethod: "DirectPlay", IsPaused: false } },
|
||||||
|
{ UserName: "Bo", Client: "Android TV", DeviceName: "Shield", NowPlayingItem: { Name: "Pilot", SeriesName: "The Office" }, PlayState: { PlayMethod: "Transcode", IsPaused: true }, TranscodingInfo: { VideoCodec: "h264" } },
|
||||||
|
{ UserName: "Cy", Client: "Jellyfin Web", DeviceName: "Chrome" }, // signed in, playing nothing
|
||||||
|
];
|
||||||
|
|
||||||
|
const read = (extra: Partial<import("../src/integrations/jellyfin/adapter.js").JellyfinRead> = {}) => ({
|
||||||
|
name: "Den", version: "10.10.3", os: "Linux", pendingRestart: false, updateAvailable: false, streams: [] as import("../src/integrations/jellyfin/adapter.js").JellyfinStream[],
|
||||||
|
tasks: [] as import("../src/integrations/jellyfin/adapter.js").JellyfinTask[], counts: { movies: 120, series: 30, episodes: 900, songs: null }, failedLogins24h: 0, ...extra,
|
||||||
|
});
|
||||||
|
const stream = (user: string, extra: Partial<import("../src/integrations/jellyfin/adapter.js").JellyfinStream> = {}) => ({ user, client: "Web", device: "Firefox", title: "Dune", transcoding: false, paused: false, ...extra });
|
||||||
|
|
||||||
|
describe("what Jellyfin's API says", () => {
|
||||||
|
test("the server's own information", () => {
|
||||||
|
const i = adapter.parseSystemInfo({ ServerName: "Den", Version: "10.10.3", OperatingSystemDisplayName: "Debian", HasPendingRestart: true, HasUpdateAvailable: false });
|
||||||
|
assert.deepEqual(i, { name: "Den", version: "10.10.3", os: "Debian", pendingRestart: true, updateAvailable: false });
|
||||||
|
assert.deepEqual(adapter.parseSystemInfo(null), { name: null, version: null, os: null, pendingRestart: false, updateAvailable: false });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("only the sessions playing something are streams, with the series in the title and transcoding told from the play method", () => {
|
||||||
|
const s = adapter.parseSessions(SESSIONS);
|
||||||
|
assert.deepEqual(s.map((x) => [x.user, x.title, x.transcoding, x.paused]), [["Ann", "Dune", false, false], ["Bo", "The Office — Pilot", true, true]]);
|
||||||
|
assert.deepEqual(adapter.parseSessions("nonsense"), []);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the scheduled tasks, hidden ones left out, with how they last ended", () => {
|
||||||
|
const t = adapter.parseTasks([
|
||||||
|
{ Name: "Scan media library", Category: "Library", State: "Idle", LastExecutionResult: { Status: "Failed", EndTimeUtc: "2026-10-20T03:00:00Z", ErrorMessage: "disk full" } },
|
||||||
|
{ Name: "Hidden", IsHidden: true },
|
||||||
|
{ Name: "Never run" },
|
||||||
|
]);
|
||||||
|
assert.equal(t.length, 2);
|
||||||
|
assert.deepEqual([t[0].lastStatus, t[0].error, t[0].lastEndedAt], ["Failed", "disk full", "2026-10-20T03:00:00.000Z"]);
|
||||||
|
assert.equal(t[1].lastStatus, null);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("failed sign-ins in the last day, from the activity log", () => {
|
||||||
|
const now = new Date("2026-10-20T12:00:00Z");
|
||||||
|
const raw = { Items: [{ Type: "AuthenticationFailed", Date: "2026-10-20T11:00:00Z" }, { Type: "AuthenticationFailed", Date: "2026-10-18T11:00:00Z" }, { Type: "UserLoggedIn", Date: "2026-10-20T11:30:00Z" }] };
|
||||||
|
assert.equal(adapter.countFailedLogins(raw, now), 1);
|
||||||
|
assert.equal(adapter.countFailedLogins({}, now), null);
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("the adapter against a Jellyfin that answers", () => {
|
||||||
|
let server: Server;
|
||||||
|
let url = "";
|
||||||
|
const seen: { path: string; token: string | undefined }[] = [];
|
||||||
|
before(async () => {
|
||||||
|
server = createHttpServer((req, res) => {
|
||||||
|
seen.push({ path: req.url ?? "", token: req.headers["x-emby-token"] as string | undefined });
|
||||||
|
const send = (code: number, body: unknown) => {
|
||||||
|
res.writeHead(code, { "content-type": "application/json" });
|
||||||
|
res.end(JSON.stringify(body));
|
||||||
|
};
|
||||||
|
if (req.headers["x-emby-token"] !== "key") return send(401, {});
|
||||||
|
const path = (req.url ?? "").split("?")[0];
|
||||||
|
if (path === "/System/Info") return send(200, { ServerName: "Den", Version: "10.10.3", HasPendingRestart: false });
|
||||||
|
if (path === "/Sessions") return send(200, SESSIONS);
|
||||||
|
if (path === "/ScheduledTasks") return send(500, {}); // the rest is worth having, not worth failing for
|
||||||
|
if (path === "/Items/Counts") return send(200, { MovieCount: 120, SeriesCount: 30 });
|
||||||
|
send(404, {});
|
||||||
|
});
|
||||||
|
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
|
||||||
|
url = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
|
||||||
|
});
|
||||||
|
after(() => server.close());
|
||||||
|
|
||||||
|
test("it reads, sends the key, and a part that cannot be read does not fail the whole", async () => {
|
||||||
|
const r = await adapter.createJellyfinAdapter({ url: `${url}/`, apiKey: "key" }).read(new Date());
|
||||||
|
assert.equal(r.name, "Den");
|
||||||
|
assert.equal(r.streams.length, 2);
|
||||||
|
assert.deepEqual(r.tasks, [], "its tasks could not be read");
|
||||||
|
assert.deepEqual([r.counts.movies, r.counts.songs, r.failedLogins24h], [120, null, null]);
|
||||||
|
assert.ok(seen.every((s) => s.token === "key"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a refused key and a dead address are said plainly", async () => {
|
||||||
|
const bad = await adapter.createJellyfinAdapter({ url, apiKey: "nope" }).ping();
|
||||||
|
assert.equal(bad.ok, false);
|
||||||
|
assert.match(bad.error ?? "", /did not accept the API key/);
|
||||||
|
const good = await adapter.createJellyfinAdapter({ url, apiKey: "key" }).ping();
|
||||||
|
assert.equal(good.ok, true);
|
||||||
|
const dead = await adapter.createJellyfinAdapter({ url: "http://127.0.0.1:1", apiKey: "key" }).ping();
|
||||||
|
assert.equal(dead.ok, false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("what is stored", () => {
|
||||||
|
let id = 0;
|
||||||
|
before(async () => {
|
||||||
|
id = (await db.insert(integrations).values({ type: "jellyfin", name: "Den", baseUrl: "http://10.9.9.8:8096", config: JSON.stringify({ url: "http://10.9.9.8:8096", apiKey: "key" }) }).returning({ id: integrations.id }))[0].id;
|
||||||
|
});
|
||||||
|
|
||||||
|
const points = async () => (await db.select().from(jellyfinReadings).where(eq(jellyfinReadings.integrationId, id))).sort((a, b) => a.at.localeCompare(b.at));
|
||||||
|
|
||||||
|
test("the state is replaced at each read; a history point is added when it is time or when the number streaming changes", async () => {
|
||||||
|
const t0 = new Date(Date.now() - 60 * MIN);
|
||||||
|
await jf.storeRead(id, read({ streams: [stream("Ann")] }), t0);
|
||||||
|
await jf.storeRead(id, read({ streams: [stream("Ann")] }), new Date(t0.getTime() + 1 * MIN));
|
||||||
|
assert.equal((await points()).length, 1, "nothing changed, and it is too soon");
|
||||||
|
await jf.storeRead(id, read({ streams: [stream("Ann"), stream("Bo", { transcoding: true })] }), new Date(t0.getTime() + 2 * MIN));
|
||||||
|
assert.deepEqual((await points()).map((p) => [p.streams, p.transcodes]), [[1, 0], [2, 1]], "a change is a point at once");
|
||||||
|
await jf.storeRead(id, read({ streams: [stream("Ann"), stream("Bo", { transcoding: true })] }), new Date(t0.getTime() + 8 * MIN));
|
||||||
|
assert.equal((await points()).length, 3, "and so is the passing of five minutes");
|
||||||
|
const [state] = await db.select().from(jellyfinState).where(eq(jellyfinState.integrationId, id));
|
||||||
|
assert.equal(jf.streamsOf(state).length, 2);
|
||||||
|
assert.equal(state.movies, 120);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("readings older than 35 days are dropped", async () => {
|
||||||
|
await db.insert(jellyfinReadings).values({ integrationId: id, at: new Date(Date.now() - 40 * DAY).toISOString(), streams: 5, transcodes: 0 });
|
||||||
|
await jf.storeRead(id, read(), new Date());
|
||||||
|
assert.ok((await points()).every((p) => Date.parse(p.at) > Date.now() - 36 * DAY));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a read that failed is said, and what was stored is left alone", async () => {
|
||||||
|
const ok = await jf.syncJellyfin(id, new Date(), async () => read({ name: "Den", version: "10.10.4" }));
|
||||||
|
assert.equal(ok.ok, true);
|
||||||
|
const failed = await jf.syncJellyfin(id, new Date(), async () => {
|
||||||
|
throw new Error("connection refused");
|
||||||
|
});
|
||||||
|
assert.deepEqual([failed.ok, failed.error], [false, "connection refused"]);
|
||||||
|
assert.equal(failed.at, ok.at, "the time of the last good read is kept");
|
||||||
|
const [state] = await db.select().from(jellyfinState).where(eq(jellyfinState.integrationId, id));
|
||||||
|
assert.equal(state.version, "10.10.4");
|
||||||
|
assert.equal((await jf.jellyfinSyncStatuses())[id].ok, false);
|
||||||
|
assert.equal(jf.isJellyfinFresh((await jf.jellyfinSyncStatuses())[id], Date.now()), false, "a failed read is not believed");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("only a good read recent enough is fresh", () => {
|
||||||
|
const now = Date.now();
|
||||||
|
assert.equal(jf.isJellyfinFresh({ at: new Date(now - 3 * MIN).toISOString(), ok: true, error: null }, now), true);
|
||||||
|
assert.equal(jf.isJellyfinFresh({ at: new Date(now - 11 * MIN).toISOString(), ok: true, error: null }, now), false);
|
||||||
|
assert.equal(jf.isJellyfinFresh(undefined, now), false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("what is wrong, and the history in peaks", () => {
|
||||||
|
const row = (extra: Partial<Parameters<typeof jf.jellyfinProblems>[2]> = {}) => ({ pendingRestart: false, tasks: "[]", failedLogins24h: 0, ...extra });
|
||||||
|
|
||||||
|
test("a restart that is wanted, a task that failed, a burst of failed sign-ins", () => {
|
||||||
|
assert.deepEqual(jf.jellyfinProblems(1, "Den", row()), []);
|
||||||
|
const p = jf.jellyfinProblems(
|
||||||
|
1,
|
||||||
|
"Den",
|
||||||
|
row({ pendingRestart: true, failedLogins24h: jf.FAILED_LOGINS_WARN, tasks: JSON.stringify([{ name: "Scan", category: null, state: "Idle", lastStatus: "Failed", lastEndedAt: null, error: "disk full" }, { name: "Clean", category: null, state: "Idle", lastStatus: "Completed", lastEndedAt: null, error: null }]) }),
|
||||||
|
);
|
||||||
|
assert.deepEqual(p.map((x) => x.key), ["jellyfin-restart:1", "jellyfin-task:1:Scan", "jellyfin-logins:1"]);
|
||||||
|
assert.match(p[1].message, /“Scan” failed — disk full/);
|
||||||
|
assert.equal(jf.jellyfinProblems(1, "Den", row({ failedLogins24h: jf.FAILED_LOGINS_WARN - 1 })).length, 0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("history is bucketed to the peak of each stretch", () => {
|
||||||
|
const from = Date.parse("2026-10-20T00:00:00Z");
|
||||||
|
const to = from + 4 * 3_600_000;
|
||||||
|
const rows = [
|
||||||
|
{ at: new Date(from + 5 * MIN).toISOString(), streams: 1, transcodes: 0 },
|
||||||
|
{ at: new Date(from + 20 * MIN).toISOString(), streams: 3, transcodes: 2 },
|
||||||
|
{ at: new Date(from + 3 * 3_600_000 + 10 * MIN).toISOString(), streams: 2, transcodes: 0 },
|
||||||
|
{ at: new Date(from - MIN).toISOString(), streams: 9, transcodes: 9 }, // before the range
|
||||||
|
];
|
||||||
|
const b = jf.bucketPeaks(rows, from, to, 4);
|
||||||
|
assert.deepEqual(b.map((p) => [p.streams, p.transcodes]), [[3, 2], [2, 0]], "an average would have hidden the evening");
|
||||||
|
assert.deepEqual(jf.bucketPeaks([], from, to, 4), []);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("who is watching, the warning before an update, the health pass, the page and /metrics", () => {
|
||||||
|
let jfId = 0;
|
||||||
|
let serverId = 0;
|
||||||
|
let otherId = 0;
|
||||||
|
let app: Server;
|
||||||
|
let base = "";
|
||||||
|
const fresh = () => setInternalFlag("jellyfinSync", JSON.stringify({ [jfId]: { at: new Date().toISOString(), ok: true, error: null } }));
|
||||||
|
|
||||||
|
before(async () => {
|
||||||
|
await db.insert(users).values([{ oidcSub: "viewer", name: "Vic", role: "viewer" }]);
|
||||||
|
jfId = (await db.insert(integrations).values({ type: "jellyfin", name: "Cinema", baseUrl: "http://10.9.9.9:8096", config: JSON.stringify({ url: "http://10.9.9.9:8096", apiKey: "key" }) }).returning({ id: integrations.id }))[0].id;
|
||||||
|
const mk = async (name: string, ip: string) =>
|
||||||
|
(await db.insert(servers).values({ name, apiTokenHash: name, apiTokenPrefix: name, lastSeenAt: new Date().toISOString(), ipAddresses: JSON.stringify([ip]) }).returning({ id: servers.id }))[0].id;
|
||||||
|
serverId = await mk("media", "10.9.9.9");
|
||||||
|
otherId = await mk("other", "10.9.9.10");
|
||||||
|
await jf.storeRead(jfId, read({ name: "Cinema", pendingRestart: true, streams: [stream("Ann", { title: "Dune" }), stream("Bo", { title: "The Office — Pilot", transcoding: true })] }));
|
||||||
|
await fresh();
|
||||||
|
|
||||||
|
const a = express();
|
||||||
|
a.use((req, _res, next) => {
|
||||||
|
(req as unknown as { session: unknown }).session = { user: { sub: "viewer" } };
|
||||||
|
next();
|
||||||
|
});
|
||||||
|
a.use("/api/jellyfin", jellyfinRouter);
|
||||||
|
app = await new Promise<Server>((resolve) => {
|
||||||
|
const s = a.listen(0, "127.0.0.1", () => resolve(s));
|
||||||
|
});
|
||||||
|
base = `http://127.0.0.1:${(app.address() as AddressInfo).port}/api/jellyfin`;
|
||||||
|
});
|
||||||
|
after(() => {
|
||||||
|
app.close();
|
||||||
|
try {
|
||||||
|
rmSync(dir, { recursive: true, force: true });
|
||||||
|
} catch {
|
||||||
|
// Windows keeps the database file locked until the process exits.
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the people streaming from the Jellyfin that runs on a server, in words; none on another, none when the read is old", async () => {
|
||||||
|
const w = await jf.watchingOnServer(serverId);
|
||||||
|
assert.equal(w.length, 1);
|
||||||
|
assert.equal(jf.watchingWords(w), "2 people are streaming from Cinema (Ann: Dune, Bo: The Office — Pilot), 1 being transcoded");
|
||||||
|
assert.deepEqual(await jf.watchingOnServer(otherId), []);
|
||||||
|
await setInternalFlag("jellyfinSync", JSON.stringify({ [jfId]: { at: new Date(Date.now() - 30 * MIN).toISOString(), ok: true, error: null } }));
|
||||||
|
assert.deepEqual(await jf.watchingOnServer(serverId), [], "an old reading is not believed");
|
||||||
|
await fresh();
|
||||||
|
assert.equal(jf.watchingWords([]), "");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("an update run on that machine is held back with 409 'watching' unless it is told to go ahead", async () => {
|
||||||
|
const input = { serverId, integrationId: 999_999, projectId: 1, templateId: 1, limit: "media", windowMinutes: 90, actor: "test" };
|
||||||
|
await assert.rejects(
|
||||||
|
() => startUpdateRun(input),
|
||||||
|
(err: unknown) => err instanceof UpdateRunError && err.status === 409 && err.code === "watching" && /2 people are streaming from Cinema/.test(err.message),
|
||||||
|
);
|
||||||
|
// Past the warning it carries on to the next check, which is the Semaphore that doesn't exist here.
|
||||||
|
await assert.rejects(
|
||||||
|
() => startUpdateRun({ ...input, startAnyway: true }),
|
||||||
|
(err: unknown) => err instanceof UpdateRunError && err.code === "no_semaphore",
|
||||||
|
);
|
||||||
|
// A machine nobody is streaming from is not held back.
|
||||||
|
await assert.rejects(
|
||||||
|
() => startUpdateRun({ ...input, serverId: otherId }),
|
||||||
|
(err: unknown) => err instanceof UpdateRunError && err.code === "no_semaphore",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the health pass judges it from what is stored, and holds it when the last read is old", async () => {
|
||||||
|
let { snapshot, held } = await collectSnapshot();
|
||||||
|
assert.deepEqual(snapshot.jellyfin?.map((j) => j.integrationId), [jfId]);
|
||||||
|
assert.equal(held.has(`jellyfin:${jfId}`), false);
|
||||||
|
const list = evaluateHealth(snapshot, { serverOfflineMinutes: 60, diskUsagePercent: 90 }, Date.now(), { skipServers: true });
|
||||||
|
const c = list.find((x) => x.key === `jellyfin-restart:${jfId}`)!;
|
||||||
|
assert.deepEqual([c.severity, c.source], ["warning", `jellyfin:${jfId}`]);
|
||||||
|
assert.equal(subjectOfConditionKey(c.key), `integration:${jfId}`, "maintenance on it silences its alerts");
|
||||||
|
await setInternalFlag("jellyfinSync", JSON.stringify({ [jfId]: { at: new Date(Date.now() - 30 * MIN).toISOString(), ok: true, error: null } }));
|
||||||
|
({ snapshot, held } = await collectSnapshot());
|
||||||
|
assert.deepEqual([snapshot.jellyfin ?? [], held.has(`jellyfin:${jfId}`)], [[], true]);
|
||||||
|
await fresh();
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the Alerts page shows it, with where to look", async () => {
|
||||||
|
invalidateAlertsCache();
|
||||||
|
const report = await getAlerts(true);
|
||||||
|
const a = report.alerts.find((x) => x.id === `monitoring:jellyfin-restart:${jfId}`)!;
|
||||||
|
assert.ok(a, "the restart that is wanted is an alert");
|
||||||
|
assert.deepEqual([a.severity, a.category, a.source, a.link, a.target?.type, a.target?.id], ["warning", "monitoring", "Jellyfin", "/jellyfin", "integration", jfId]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("anyone signed in sees each Jellyfin, who is streaming and what is wrong", async () => {
|
||||||
|
const res = await fetch(base);
|
||||||
|
assert.equal(res.status, 200);
|
||||||
|
const body = (await res.json()) as { jellyfins: { id: number; fresh: boolean; runsOn: { name: string } | null; state: { streams: unknown[] } | null; problems: { key: string }[] }[] };
|
||||||
|
const j = body.jellyfins.find((x) => x.id === jfId)!;
|
||||||
|
assert.equal(j.fresh, true);
|
||||||
|
assert.equal(j.runsOn?.name, "media");
|
||||||
|
assert.equal(j.state?.streams.length, 2);
|
||||||
|
assert.ok(j.problems.some((p) => p.key === `jellyfin-restart:${jfId}`));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("who is watching on a server is asked for by its id, and a wrong id is refused", async () => {
|
||||||
|
const r = (await (await fetch(`${base}/watching?serverId=${serverId}`)).json()) as { watching: unknown[]; words: string };
|
||||||
|
assert.equal(r.watching.length, 1);
|
||||||
|
assert.match(r.words, /2 people are streaming/);
|
||||||
|
const none = (await (await fetch(`${base}/watching?serverId=${otherId}`)).json()) as { watching: unknown[]; words: string };
|
||||||
|
assert.deepEqual([none.watching.length, none.words], [0, ""]);
|
||||||
|
assert.equal((await fetch(`${base}/watching?serverId=abc`)).status, 400);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the history is the peaks of the last days; a missing Jellyfin is not found", async () => {
|
||||||
|
const res = await fetch(`${base}/${jfId}/history?days=1`);
|
||||||
|
assert.equal(res.status, 200);
|
||||||
|
const body = (await res.json()) as { days: number; points: { streams: number; transcodes: number }[] };
|
||||||
|
assert.equal(body.days, 1);
|
||||||
|
assert.ok(body.points.some((p) => p.streams === 2 && p.transcodes === 1));
|
||||||
|
assert.equal((await (await fetch(`${base}/${jfId}/history?days=400`)).json() as { days: number }).days, jf.READING_KEEP_DAYS, "no further back than is kept");
|
||||||
|
assert.equal((await fetch(`${base}/999999/history`)).status, 404);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("/metrics has how many are streaming, how many are transcoded and whether a restart is wanted — only while the reading is recent", async () => {
|
||||||
|
let text = await buildMetrics(new Date());
|
||||||
|
assert.match(text, /homelab_jellyfin_streams\{name="Cinema"\} 2/);
|
||||||
|
assert.match(text, /homelab_jellyfin_transcodes\{name="Cinema"\} 1/);
|
||||||
|
assert.match(text, /homelab_jellyfin_pending_restart\{name="Cinema"\} 1/);
|
||||||
|
await setInternalFlag("jellyfinSync", JSON.stringify({ [jfId]: { at: new Date(Date.now() - 30 * MIN).toISOString(), ok: true, error: null } }));
|
||||||
|
text = await buildMetrics(new Date());
|
||||||
|
assert.doesNotMatch(text, /homelab_jellyfin_streams/);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -20,14 +20,15 @@ process.env.HEALTH_STARTUP_GRACE_MS = "0";
|
|||||||
|
|
||||||
const express = (await import("express")).default;
|
const express = (await import("express")).default;
|
||||||
const { db } = await import("../src/db/client.js");
|
const { db } = await import("../src/db/client.js");
|
||||||
const { users, servers, patchPlans, patchPlanRuns, updateRuns, maintenanceWindows } = await import("../src/db/schema.js");
|
const { users, servers, patchPlans, patchPlanRuns, updateRuns, maintenanceWindows, integrations } = await import("../src/db/schema.js");
|
||||||
const { runMigrations } = await import("../src/db/migrate.js");
|
const { runMigrations } = await import("../src/db/migrate.js");
|
||||||
const { planServers, dueOccurrence, summariseItems, startPlanRun, advancePlanRuns, startDuePlans, cancelPlanRun, parseItems, PlanError } = await import("../src/services/patchPlans.js");
|
const { planServers, dueOccurrence, summariseItems, startPlanRun, advancePlanRuns, startDuePlans, cancelPlanRun, parseItems, PlanError } = await import("../src/services/patchPlans.js");
|
||||||
const { advanceUpdateRuns, setSemaphoreForTests } = await import("../src/services/updateRuns.js");
|
const { advanceUpdateRuns, setSemaphoreForTests } = await import("../src/services/updateRuns.js");
|
||||||
const { listActiveWindows } = await import("../src/services/maintenance.js");
|
const { listActiveWindows } = await import("../src/services/maintenance.js");
|
||||||
const { agendaFromData } = await import("../src/services/agenda.js");
|
const { agendaFromData } = await import("../src/services/agenda.js");
|
||||||
const { patchPlansRouter } = await import("../src/routes/patchPlans.js");
|
const { patchPlansRouter } = await import("../src/routes/patchPlans.js");
|
||||||
const { updateSettings } = await import("../src/services/settingsStore.js");
|
const { updateSettings, setInternalFlag } = await import("../src/services/settingsStore.js");
|
||||||
|
const { storeRead } = await import("../src/services/jellyfinSync.js");
|
||||||
const { eq } = await import("drizzle-orm");
|
const { eq } = await import("drizzle-orm");
|
||||||
|
|
||||||
await runMigrations();
|
await runMigrations();
|
||||||
@@ -196,6 +197,39 @@ describe("a run, from start to end", () => {
|
|||||||
assert.match(last, /✓ web/);
|
assert.match(last, /✓ web/);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("a server people are streaming from waits its turn for up to two hours, then is updated anyway", async () => {
|
||||||
|
await reset({ tags: "[]", serverIds: JSON.stringify([ids.api]) });
|
||||||
|
await db.update(servers).set({ ipAddresses: JSON.stringify(["10.7.7.7"]) }).where(eq(servers.id, ids.api));
|
||||||
|
const jfId = (await db.insert(integrations).values({ type: "jellyfin", name: "Cinema", baseUrl: "http://10.7.7.7:8096", config: "{}" }).returning({ id: integrations.id }))[0].id;
|
||||||
|
const seen = (m: number) => setInternalFlag("jellyfinSync", JSON.stringify({ [jfId]: { at: at(m).toISOString(), ok: true, error: null } }));
|
||||||
|
try {
|
||||||
|
await storeRead(jfId, { name: "Cinema", version: null, os: null, pendingRestart: false, updateAvailable: false, streams: [{ user: "Ann", client: null, device: null, title: "Dune", transcoding: false, paused: false }], tasks: [], counts: { movies: null, series: null, episodes: null, songs: null }, failedLogins24h: null }, T0);
|
||||||
|
const run = await startPlanRun(planId, { trigger: "manual", actor: "Olle", now: T0 });
|
||||||
|
|
||||||
|
await seen(0);
|
||||||
|
await advancePlanRuns(T0);
|
||||||
|
let [item] = await itemsOf(run.id);
|
||||||
|
assert.deepEqual([item.status, fake.runs.length], ["pending", 0], "somebody is watching, so it is not its turn yet");
|
||||||
|
assert.match(item.message ?? "", /^Waiting: 1 person is streaming from Cinema \(Ann: Dune\)/);
|
||||||
|
assert.equal(item.waitingSince, T0.toISOString());
|
||||||
|
|
||||||
|
await seen(60);
|
||||||
|
await advancePlanRuns(at(60));
|
||||||
|
[item] = await itemsOf(run.id);
|
||||||
|
assert.deepEqual([item.status, fake.runs.length, item.waitingSince], ["pending", 0, T0.toISOString()], "an hour on, still waiting, and the wait is counted from the first time");
|
||||||
|
|
||||||
|
await seen(121);
|
||||||
|
await advancePlanRuns(at(121));
|
||||||
|
[item] = await itemsOf(run.id);
|
||||||
|
assert.deepEqual([item.status, fake.runs.map((r) => r.limit)], ["running", ["api"]], "after two hours it goes ahead");
|
||||||
|
assert.match(item.message ?? "", /^Went ahead after waiting 2 h; 1 person is streaming/);
|
||||||
|
} finally {
|
||||||
|
await db.delete(integrations).where(eq(integrations.id, jfId));
|
||||||
|
await setInternalFlag("jellyfinSync", "{}");
|
||||||
|
await db.update(servers).set({ ipAddresses: null }).where(eq(servers.id, ids.api));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
test("what depends on a server is silenced while it is updated, and let go when it is done", async () => {
|
test("what depends on a server is silenced while it is updated, and let go when it is done", async () => {
|
||||||
await reset({ tags: "[]", serverIds: JSON.stringify([ids.web]) });
|
await reset({ tags: "[]", serverIds: JSON.stringify([ids.web]) });
|
||||||
const run = await startPlanRun(planId, { trigger: "manual", actor: "Olle", now: T0 });
|
const run = await startPlanRun(planId, { trigger: "manual", actor: "Olle", now: T0 });
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ import ProxyHosts from "./pages/ProxyHosts";
|
|||||||
import AlertRules from "./pages/AlertRules";
|
import AlertRules from "./pages/AlertRules";
|
||||||
import HomeAssistant from "./pages/HomeAssistant";
|
import HomeAssistant from "./pages/HomeAssistant";
|
||||||
import Ups from "./pages/Ups";
|
import Ups from "./pages/Ups";
|
||||||
|
import Jellyfin from "./pages/Jellyfin";
|
||||||
import ServiceLevels from "./pages/ServiceLevels";
|
import ServiceLevels from "./pages/ServiceLevels";
|
||||||
import Duplicati from "./pages/Duplicati";
|
import Duplicati from "./pages/Duplicati";
|
||||||
import MyWorkPage from "./pages/MyWork";
|
import MyWorkPage from "./pages/MyWork";
|
||||||
@@ -158,6 +159,7 @@ function Shelled({ user }: { user: CurrentUser }) {
|
|||||||
<Route path="/alert-rules" element={<AlertRules user={user} />} />
|
<Route path="/alert-rules" element={<AlertRules user={user} />} />
|
||||||
<Route path="/home-assistant" element={<HomeAssistant user={user} />} />
|
<Route path="/home-assistant" element={<HomeAssistant user={user} />} />
|
||||||
<Route path="/ups" element={<Ups user={user} />} />
|
<Route path="/ups" element={<Ups user={user} />} />
|
||||||
|
<Route path="/jellyfin" element={<Jellyfin />} />
|
||||||
<Route path="/service-levels" element={<ServiceLevels user={user} />} />
|
<Route path="/service-levels" element={<ServiceLevels user={user} />} />
|
||||||
<Route path="/duplicati" element={<Duplicati user={user} />} />
|
<Route path="/duplicati" element={<Duplicati user={user} />} />
|
||||||
<Route path="/ports" element={<Ports user={user} />} />
|
<Route path="/ports" element={<Ports user={user} />} />
|
||||||
|
|||||||
+46
-1
@@ -766,6 +766,8 @@ export interface UpdateRunInput {
|
|||||||
templateId: number;
|
templateId: number;
|
||||||
limit: string;
|
limit: string;
|
||||||
windowMinutes: number;
|
windowMinutes: number;
|
||||||
|
/** Somebody is streaming from the machine and the person has seen the warning. */
|
||||||
|
startAnyway?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface UpdatesServer {
|
export interface UpdatesServer {
|
||||||
@@ -1194,6 +1196,44 @@ export interface MonthlyReport {
|
|||||||
renewals: { kind: string; subject: string; at: string; fromDate: string; toDate: string }[];
|
renewals: { kind: string; subject: string; at: string; fromDate: string; toDate: string }[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** One Jellyfin, as its page shows it. */
|
||||||
|
export interface JellyfinStream {
|
||||||
|
user: string;
|
||||||
|
client: string | null;
|
||||||
|
device: string | null;
|
||||||
|
title: string;
|
||||||
|
transcoding: boolean;
|
||||||
|
paused: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface JellyfinInfo {
|
||||||
|
id: number;
|
||||||
|
name: string;
|
||||||
|
enabled: boolean;
|
||||||
|
runsOn: { id: number; name: string } | null;
|
||||||
|
status: { at: string; ok: boolean; error: string | null } | null;
|
||||||
|
fresh: boolean;
|
||||||
|
state: {
|
||||||
|
serverName: string | null;
|
||||||
|
version: string | null;
|
||||||
|
os: string | null;
|
||||||
|
pendingRestart: boolean;
|
||||||
|
updateAvailable: boolean;
|
||||||
|
streams: JellyfinStream[];
|
||||||
|
tasks: { name: string; category: string | null; state: string; lastStatus: string | null; lastEndedAt: string | null; error: string | null }[];
|
||||||
|
counts: { movies: number | null; series: number | null; episodes: number | null; songs: number | null };
|
||||||
|
failedLogins24h: number | null;
|
||||||
|
updatedAt: string;
|
||||||
|
} | null;
|
||||||
|
problems: { key: string; message: string; severity: "warning" | "critical" }[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface JellyfinHistory {
|
||||||
|
jellyfin: { id: number; name: string };
|
||||||
|
days: number;
|
||||||
|
points: { t: string; streams: number; transcodes: number }[];
|
||||||
|
}
|
||||||
|
|
||||||
/** A check the app could set up for you, from what it already knows. */
|
/** A check the app could set up for you, from what it already knows. */
|
||||||
/** A backup job Duplicati reported, as the page shows it. */
|
/** A backup job Duplicati reported, as the page shows it. */
|
||||||
export interface DuplicatiBackup {
|
export interface DuplicatiBackup {
|
||||||
@@ -2108,7 +2148,7 @@ export interface ManualTaskInput {
|
|||||||
enabled?: boolean;
|
enabled?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type IntegrationType = "proxmox" | "synology" | "semaphore" | "tailscale" | "gitea" | "dockhand" | "uptimekuma" | "phpipam" | "pbs" | "osticket" | "unifi" | "npm" | "homeassistant" | "ups";
|
export type IntegrationType = "proxmox" | "synology" | "semaphore" | "tailscale" | "gitea" | "dockhand" | "uptimekuma" | "phpipam" | "pbs" | "osticket" | "unifi" | "npm" | "homeassistant" | "ups" | "jellyfin";
|
||||||
|
|
||||||
export interface IntegrationField {
|
export interface IntegrationField {
|
||||||
key: string;
|
key: string;
|
||||||
@@ -2655,6 +2695,11 @@ export const api = {
|
|||||||
update: (id: number, data: ServiceLevelInput) => request<{ ok: boolean }>(`/api/service-levels/${id}`, { method: "PUT", body: JSON.stringify(data) }),
|
update: (id: number, data: ServiceLevelInput) => request<{ ok: boolean }>(`/api/service-levels/${id}`, { method: "PUT", body: JSON.stringify(data) }),
|
||||||
remove: (id: number) => request<void>(`/api/service-levels/${id}`, { method: "DELETE" }),
|
remove: (id: number) => request<void>(`/api/service-levels/${id}`, { method: "DELETE" }),
|
||||||
},
|
},
|
||||||
|
jellyfin: {
|
||||||
|
list: () => request<{ jellyfins: JellyfinInfo[] }>("/api/jellyfin"),
|
||||||
|
history: (id: number, days: number) => request<JellyfinHistory>(`/api/jellyfin/${id}/history?days=${days}`),
|
||||||
|
watching: (serverId: number) => request<{ watching: { integrationId: number; integrationName: string; streams: JellyfinStream[] }[]; words: string }>(`/api/jellyfin/watching?serverId=${serverId}`),
|
||||||
|
},
|
||||||
ups: {
|
ups: {
|
||||||
list: () => request<{ ups: UpsInfo[] }>("/api/ups"),
|
list: () => request<{ ups: UpsInfo[] }>("/api/ups"),
|
||||||
history: (id: number, days: number) => request<UpsHistory>(`/api/ups/${id}/history?days=${days}`),
|
history: (id: number, days: number) => request<UpsHistory>(`/api/ups/${id}/history?days=${days}`),
|
||||||
|
|||||||
@@ -47,6 +47,7 @@ const INTEGRATION_TYPE_LABELS: Record<IntegrationType, string> = {
|
|||||||
npm: "Nginx Proxy Manager",
|
npm: "Nginx Proxy Manager",
|
||||||
homeassistant: "Home Assistant",
|
homeassistant: "Home Assistant",
|
||||||
ups: "UPS",
|
ups: "UPS",
|
||||||
|
jellyfin: "Jellyfin",
|
||||||
};
|
};
|
||||||
|
|
||||||
const DNS_PROVIDER_LABELS: Record<DnsProviderType, string> = {
|
const DNS_PROVIDER_LABELS: Record<DnsProviderType, string> = {
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ const TYPE_LABELS: Record<IntegrationType, string> = {
|
|||||||
npm: "Nginx Proxy Manager",
|
npm: "Nginx Proxy Manager",
|
||||||
homeassistant: "Home Assistant",
|
homeassistant: "Home Assistant",
|
||||||
ups: "UPS",
|
ups: "UPS",
|
||||||
|
jellyfin: "Jellyfin",
|
||||||
};
|
};
|
||||||
|
|
||||||
export default function IntegrationEditForm({
|
export default function IntegrationEditForm({
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ const TYPE_LABELS: Record<IntegrationType, string> = {
|
|||||||
npm: "Nginx Proxy Manager",
|
npm: "Nginx Proxy Manager",
|
||||||
homeassistant: "Home Assistant",
|
homeassistant: "Home Assistant",
|
||||||
ups: "UPS",
|
ups: "UPS",
|
||||||
|
jellyfin: "Jellyfin",
|
||||||
};
|
};
|
||||||
|
|
||||||
export default function IntegrationForm({ onCreated, onCancel }: { onCreated: () => void; onCancel: () => void }) {
|
export default function IntegrationForm({ onCreated, onCancel }: { onCreated: () => void; onCancel: () => void }) {
|
||||||
|
|||||||
@@ -42,6 +42,19 @@ export default function UpdateRunDrawer({ server, onClose, onStarted }: { server
|
|||||||
const [limit, setLimit] = useState(server.name);
|
const [limit, setLimit] = useState(server.name);
|
||||||
const [minutes, setMinutes] = useState(90);
|
const [minutes, setMinutes] = useState(90);
|
||||||
const [busy, setBusy] = useState(false);
|
const [busy, setBusy] = useState(false);
|
||||||
|
// People streaming from a Jellyfin on this machine: the update may restart it, so they are named first and the person asked.
|
||||||
|
const [watching, setWatching] = useState("");
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
let alive = true;
|
||||||
|
api.jellyfin
|
||||||
|
.watching(server.id)
|
||||||
|
.then((r) => alive && setWatching(r.words))
|
||||||
|
.catch(() => undefined); // no Jellyfin, or it can't be read: nothing to warn about
|
||||||
|
return () => {
|
||||||
|
alive = false;
|
||||||
|
};
|
||||||
|
}, [server.id]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
let alive = true;
|
let alive = true;
|
||||||
@@ -68,13 +81,13 @@ export default function UpdateRunDrawer({ server, onClose, onStarted }: { server
|
|||||||
if (!picked) return;
|
if (!picked) return;
|
||||||
const ok = await confirmDialog({
|
const ok = await confirmDialog({
|
||||||
title: `Update ${server.name} now?`,
|
title: `Update ${server.name} now?`,
|
||||||
message: `This silences alerts for ${server.name} for up to ${WINDOWS.find((w) => w.minutes === minutes)?.label ?? `${minutes} minutes`}, runs "${picked.template.name}" in ${picked.integration.name} limited to ${limit.trim()}, and waits for the server to report again. The playbook may restart it.`,
|
message: `${watching ? `${watching[0].toUpperCase()}${watching.slice(1)}, and the update may restart the machine. ` : ""}This silences alerts for ${server.name} for up to ${WINDOWS.find((w) => w.minutes === minutes)?.label ?? `${minutes} minutes`}, runs "${picked.template.name}" in ${picked.integration.name} limited to ${limit.trim()}, and waits for the server to report again. The playbook may restart it.`,
|
||||||
confirmLabel: "Start the update",
|
confirmLabel: watching ? "Update anyway" : "Start the update",
|
||||||
});
|
});
|
||||||
if (!ok) return;
|
if (!ok) return;
|
||||||
setBusy(true);
|
setBusy(true);
|
||||||
try {
|
try {
|
||||||
await api.updateRuns.start({ serverId: server.id, integrationId: picked.integration.id, projectId: picked.template.projectId, templateId: picked.template.id, limit: limit.trim(), windowMinutes: minutes });
|
await api.updateRuns.start({ serverId: server.id, integrationId: picked.integration.id, projectId: picked.template.projectId, templateId: picked.template.id, limit: limit.trim(), windowMinutes: minutes, startAnyway: watching !== "" });
|
||||||
try {
|
try {
|
||||||
localStorage.setItem(LAST_KEY, picked.key);
|
localStorage.setItem(LAST_KEY, picked.key);
|
||||||
} catch {
|
} catch {
|
||||||
@@ -112,6 +125,12 @@ export default function UpdateRunDrawer({ server, onClose, onStarted }: { server
|
|||||||
<div className="mb-3">
|
<div className="mb-3">
|
||||||
<ImpactNote targets={[{ type: "server", id: server.id }]} verb="may look down while it restarts" />
|
<ImpactNote targets={[{ type: "server", id: server.id }]} verb="may look down while it restarts" />
|
||||||
</div>
|
</div>
|
||||||
|
{watching && (
|
||||||
|
<div className="alert alert-warning" role="status">
|
||||||
|
<strong>Somebody is watching.</strong> {watching[0].toUpperCase()}
|
||||||
|
{watching.slice(1)}. The update may restart this machine; you will be asked once more.
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
<ol className="text-secondary small ps-3 mb-3">
|
<ol className="text-secondary small ps-3 mb-3">
|
||||||
<li>Alerts for this server are silenced (a maintenance window).</li>
|
<li>Alerts for this server are silenced (a maintenance window).</li>
|
||||||
<li>The playbook runs in Semaphore, for this server only.</li>
|
<li>The playbook runs in Semaphore, for this server only.</li>
|
||||||
|
|||||||
@@ -19,6 +19,7 @@ const TYPE_LABELS: Record<string, string> = {
|
|||||||
npm: "Nginx Proxy Manager",
|
npm: "Nginx Proxy Manager",
|
||||||
homeassistant: "Home Assistant",
|
homeassistant: "Home Assistant",
|
||||||
ups: "UPS",
|
ups: "UPS",
|
||||||
|
jellyfin: "Jellyfin",
|
||||||
};
|
};
|
||||||
|
|
||||||
export type WallLevel = "critical" | "warning" | "ok" | "unsure";
|
export type WallLevel = "critical" | "warning" | "ok" | "unsure";
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ import {
|
|||||||
IconPlugConnected,
|
IconPlugConnected,
|
||||||
IconBrandDocker,
|
IconBrandDocker,
|
||||||
IconAffiliate,
|
IconAffiliate,
|
||||||
|
IconMovie,
|
||||||
IconPlayerPlay,
|
IconPlayerPlay,
|
||||||
IconBrandGit,
|
IconBrandGit,
|
||||||
IconServerCog,
|
IconServerCog,
|
||||||
@@ -121,6 +122,7 @@ export const NAV: NavEntry[] = [
|
|||||||
{ to: "/uptime-kuma", label: "Uptime Kuma", icon: <IconActivityHeartbeat size={20} /> },
|
{ to: "/uptime-kuma", label: "Uptime Kuma", icon: <IconActivityHeartbeat size={20} /> },
|
||||||
{ to: "/home-assistant", label: "Home Assistant", icon: <IconHome2 size={20} /> },
|
{ to: "/home-assistant", label: "Home Assistant", icon: <IconHome2 size={20} /> },
|
||||||
{ to: "/ups", label: "UPS", icon: <IconBatteryCharging size={20} /> },
|
{ to: "/ups", label: "UPS", icon: <IconBatteryCharging size={20} /> },
|
||||||
|
{ to: "/jellyfin", label: "Jellyfin", icon: <IconMovie size={20} /> },
|
||||||
{ to: "/service-levels", label: "Service levels", icon: <IconTargetArrow size={20} /> },
|
{ to: "/service-levels", label: "Service levels", icon: <IconTargetArrow size={20} /> },
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ const TYPE_LABELS: Record<IntegrationType, string> = {
|
|||||||
npm: "Nginx Proxy Manager",
|
npm: "Nginx Proxy Manager",
|
||||||
homeassistant: "Home Assistant",
|
homeassistant: "Home Assistant",
|
||||||
ups: "UPS",
|
ups: "UPS",
|
||||||
|
jellyfin: "Jellyfin",
|
||||||
};
|
};
|
||||||
|
|
||||||
/** The page each kind of integration gets once it is connected. */
|
/** The page each kind of integration gets once it is connected. */
|
||||||
@@ -47,6 +48,7 @@ const TYPE_PAGES: Record<IntegrationType, string> = {
|
|||||||
npm: "/proxy-hosts",
|
npm: "/proxy-hosts",
|
||||||
homeassistant: "/home-assistant",
|
homeassistant: "/home-assistant",
|
||||||
ups: "/ups",
|
ups: "/ups",
|
||||||
|
jellyfin: "/jellyfin",
|
||||||
};
|
};
|
||||||
|
|
||||||
/** "pve.example.lan" out of "https://pve.example.lan:8006/", or the text itself when it isn't an address. */
|
/** "pve.example.lan" out of "https://pve.example.lan:8006/", or the text itself when it isn't an address. */
|
||||||
|
|||||||
@@ -0,0 +1,199 @@
|
|||||||
|
import { useEffect, useMemo, useState } from "react";
|
||||||
|
import { Link } from "react-router-dom";
|
||||||
|
import { api, type JellyfinHistory, type JellyfinInfo } from "../api/client";
|
||||||
|
import EmptyState from "../components/EmptyState";
|
||||||
|
import RangeSelect from "../components/RangeSelect";
|
||||||
|
import TimeAgo from "../components/TimeAgo";
|
||||||
|
import TimeChart from "../components/TimeChart";
|
||||||
|
import { BlockSkeleton } from "../components/Skeleton";
|
||||||
|
import { ErrorNotice } from "../components/Notice";
|
||||||
|
import { useChartRange } from "../utils/chartRanges";
|
||||||
|
import { readableError } from "../utils/errors";
|
||||||
|
|
||||||
|
function Badge({ j }: { j: JellyfinInfo }) {
|
||||||
|
if (!j.enabled) return <span className="badge bg-secondary-lt text-secondary">Off</span>;
|
||||||
|
if (!j.state) return <span className="badge bg-secondary-lt text-secondary">Not read yet</span>;
|
||||||
|
if (!j.fresh) return <span className="badge bg-yellow-lt text-yellow">Can’t be read</span>;
|
||||||
|
if (j.problems.some((p) => p.severity === "critical")) return <span className="badge bg-red-lt text-red">Needs a look</span>;
|
||||||
|
if (j.problems.length > 0) return <span className="badge bg-yellow-lt text-yellow">Needs a look</span>;
|
||||||
|
return <span className="badge bg-green-lt text-green">Up</span>;
|
||||||
|
}
|
||||||
|
|
||||||
|
const count = (n: number | null, one: string, many: string) => (n === null ? null : `${n.toLocaleString()} ${n === 1 ? one : many}`);
|
||||||
|
|
||||||
|
function History({ id, name }: { id: number; name: string }) {
|
||||||
|
const [days, setDays] = useChartRange(7);
|
||||||
|
const shown = days > 30 ? 30 : days;
|
||||||
|
const [history, setHistory] = useState<JellyfinHistory | null>(null);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
const [to, setTo] = useState(() => Date.now());
|
||||||
|
useEffect(() => {
|
||||||
|
let alive = true;
|
||||||
|
api.jellyfin
|
||||||
|
.history(id, shown)
|
||||||
|
.then((res) => {
|
||||||
|
if (!alive) return;
|
||||||
|
setHistory(res);
|
||||||
|
setTo(Date.now());
|
||||||
|
setError(null);
|
||||||
|
})
|
||||||
|
.catch((err) => alive && setError(readableError(err)));
|
||||||
|
return () => {
|
||||||
|
alive = false;
|
||||||
|
};
|
||||||
|
}, [id, shown]);
|
||||||
|
const from = to - shown * 86_400_000;
|
||||||
|
const streams = useMemo(
|
||||||
|
() => [
|
||||||
|
{ name: "Streaming", color: "#4299e1", points: (history?.points ?? []).map((p) => ({ t: p.t, v: p.streams })) },
|
||||||
|
{ name: "Transcoded", color: "#f76707", dash: "4 3", points: (history?.points ?? []).map((p) => ({ t: p.t, v: p.transcodes })) },
|
||||||
|
],
|
||||||
|
[history],
|
||||||
|
);
|
||||||
|
return (
|
||||||
|
<div>
|
||||||
|
<div className="d-flex align-items-center mb-2">
|
||||||
|
<h4 className="mb-0">Who was watching</h4>
|
||||||
|
<div className="ms-auto">
|
||||||
|
<RangeSelect value={shown} onChange={setDays} only={[1, 7, 30]} />
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<ErrorNotice error={error} />
|
||||||
|
{!history && !error && <BlockSkeleton lines={3} />}
|
||||||
|
{history && <TimeChart series={streams} from={from} to={to} unit="value" compact label={`${name}: streams at the busiest`} empty="Nothing read in this stretch yet." format={(v) => `${Math.round(v)}`} />}
|
||||||
|
<div className="text-secondary small mt-1">The most that were streaming at once in each stretch — an average would hide the evening somebody watched.</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function Card({ j }: { j: JellyfinInfo }) {
|
||||||
|
const s = j.state;
|
||||||
|
const failed = (s?.tasks ?? []).filter((t) => t.lastStatus === "Failed");
|
||||||
|
const libraries = s ? [count(s.counts.movies, "film", "films"), count(s.counts.series, "series", "series"), count(s.counts.episodes, "episode", "episodes"), count(s.counts.songs, "song", "songs")].filter(Boolean) : [];
|
||||||
|
return (
|
||||||
|
<div className="card mb-3">
|
||||||
|
<div className="card-header">
|
||||||
|
<h3 className="card-title">
|
||||||
|
{j.name} <Badge j={j} />
|
||||||
|
</h3>
|
||||||
|
<div className="card-actions text-secondary small">
|
||||||
|
{j.runsOn && (
|
||||||
|
<>
|
||||||
|
runs on <Link to={`/servers/${j.runsOn.id}`}>{j.runsOn.name}</Link> ·{" "}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
{j.status ? <>read <TimeAgo value={j.status.at} /></> : "not read yet"}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="card-body">
|
||||||
|
{j.status && !j.status.ok && <div className="text-yellow mb-2">The last attempt to read it failed: {j.status.error ?? "no answer"}.</div>}
|
||||||
|
{j.problems.length > 0 && (
|
||||||
|
<ul className="mb-3">
|
||||||
|
{j.problems.map((p) => (
|
||||||
|
<li key={p.key} className={p.severity === "critical" ? "text-red" : ""}>
|
||||||
|
{p.message}
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
)}
|
||||||
|
{s && (
|
||||||
|
<>
|
||||||
|
<div className="text-secondary small mb-3">
|
||||||
|
{[s.serverName && s.serverName !== j.name ? s.serverName : null, s.version ? `version ${s.version}` : null, s.os, s.updateAvailable ? "an update is available" : null].filter(Boolean).join(" · ")}
|
||||||
|
{libraries.length > 0 && <div>{libraries.join(" · ")}</div>}
|
||||||
|
</div>
|
||||||
|
<h4>Streaming now</h4>
|
||||||
|
{s.streams.length === 0 ? (
|
||||||
|
<div className="text-secondary mb-3">Nobody.</div>
|
||||||
|
) : (
|
||||||
|
<div className="table-responsive mb-3">
|
||||||
|
<table className="table table-vcenter table-sm">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Who</th>
|
||||||
|
<th>Watching</th>
|
||||||
|
<th>On</th>
|
||||||
|
<th>How</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{s.streams.map((x, i) => (
|
||||||
|
<tr key={`${x.user}-${i}`}>
|
||||||
|
<td>{x.user}</td>
|
||||||
|
<td>
|
||||||
|
{x.title}
|
||||||
|
{x.paused && <span className="text-secondary"> (paused)</span>}
|
||||||
|
</td>
|
||||||
|
<td className="text-secondary">{[x.client, x.device].filter(Boolean).join(" · ") || "—"}</td>
|
||||||
|
<td>{x.transcoding ? <span className="badge bg-orange-lt text-orange">Transcoded</span> : <span className="badge bg-green-lt text-green">Direct</span>}</td>
|
||||||
|
</tr>
|
||||||
|
))}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{failed.length > 0 && (
|
||||||
|
<>
|
||||||
|
<h4>Scheduled tasks that failed</h4>
|
||||||
|
<ul className="mb-3">
|
||||||
|
{failed.map((t) => (
|
||||||
|
<li key={t.name}>
|
||||||
|
<strong>{t.name}</strong>
|
||||||
|
{t.lastEndedAt && (
|
||||||
|
<>
|
||||||
|
{" "}
|
||||||
|
— <TimeAgo value={t.lastEndedAt} />
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
{t.error && <div className="text-secondary small">{t.error}</div>}
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
{s.failedLogins24h !== null && s.failedLogins24h > 0 && <div className="text-secondary small mb-3">{s.failedLogins24h} failed sign-in{s.failedLogins24h === 1 ? "" : "s"} in the last day.</div>}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
{j.enabled && s && <History id={j.id} name={j.name} />}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Jellyfin: whether it is up, who is streaming from it right now and whether that is being transcoded, and the things that need a look. */
|
||||||
|
export default function Jellyfin() {
|
||||||
|
const [data, setData] = useState<JellyfinInfo[] | null>(null);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
useEffect(() => {
|
||||||
|
const load = () =>
|
||||||
|
api.jellyfin
|
||||||
|
.list()
|
||||||
|
.then((r) => {
|
||||||
|
setData(r.jellyfins);
|
||||||
|
setError(null);
|
||||||
|
})
|
||||||
|
.catch((err) => setError(readableError(err)));
|
||||||
|
void load();
|
||||||
|
const t = setInterval(load, 30_000);
|
||||||
|
return () => clearInterval(t);
|
||||||
|
}, []);
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<h2 className="page-title">Jellyfin</h2>
|
||||||
|
<p className="text-secondary">
|
||||||
|
Whether Jellyfin is up and who is watching. It is read every couple of minutes with an API key (see <Link to="/integrations">Integrations</Link>). A Jellyfin that <strong>wants a restart</strong>, a <strong>scheduled task that failed</strong> or a burst of{" "}
|
||||||
|
<strong>failed sign-ins</strong> is an alert; and an update run on the machine it lives on <strong>asks first when somebody is streaming</strong>.
|
||||||
|
</p>
|
||||||
|
<ErrorNotice error={error} />
|
||||||
|
{!data && !error && <BlockSkeleton lines={4} />}
|
||||||
|
{data && data.length === 0 && (
|
||||||
|
<div className="card">
|
||||||
|
<div className="card-body">
|
||||||
|
<EmptyState title="No Jellyfin connected" hint="Add one under Integrations with its address and an API key (Jellyfin → Dashboard → API Keys)." />
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{data?.map((j) => <Card key={j.id} j={j} />)}
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -27,6 +27,7 @@ const INTEGRATION_NAMES: Record<string, string> = {
|
|||||||
npm: "Nginx Proxy Manager",
|
npm: "Nginx Proxy Manager",
|
||||||
homeassistant: "Home Assistant",
|
homeassistant: "Home Assistant",
|
||||||
ups: "UPS",
|
ups: "UPS",
|
||||||
|
jellyfin: "Jellyfin",
|
||||||
};
|
};
|
||||||
|
|
||||||
function ColorList({
|
function ColorList({
|
||||||
|
|||||||
Reference in new issue
Block a user