Lidarr/Readarr, integration inspector, app versions, grouped alerts

Lidarr and Readarr join the shared *arr reader. Integrations get an Inspect button that
runs the reader once and shows what each request returned as shapes (never values) and
where the result is empty. Operations -> App versions compares each application's version
with its newest GitHub release (lookup off until an admin switches it on; optional alert).
The Alerts page can group by source, kind of problem or type of alert. The shared HTTP
helper no longer reuses connections (stale-socket resets). Migration 0078; docs and tests
included.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
bobbanandClaude Sonnet 5.5 committed 2026-10-29 20:07:36 +01:00
1 parent 551eb1597a
commit 93aaf5476d
45 files changed
+10033 -65

No files matched your search

+18 -4
View File
@@ -23,8 +23,8 @@ what the app actually creates.
|---|---| |---|---|
| **File** | `DATABASE_PATH`, default `../data/homelab-manager.sqlite` (relative to `server/`; `/app/data/...` in Docker). The folder is created if missing. | | **File** | `DATABASE_PATH`, default `../data/homelab-manager.sqlite` (relative to `server/`; `/app/data/...` in Docker). The folder is created if missing. |
| **Foreign keys** | Switched on for every connection (`PRAGMA foreign_keys = ON`), so the `ON DELETE` rules below are enforced. | | **Foreign keys** | Switched on for every connection (`PRAGMA foreign_keys = ON`), so the `ON DELETE` rules below are enforced. |
| **Migrations** | SQL files in [`server/drizzle/`](server/drizzle) (`0000` … `0077`), applied automatically when the server starts. Which ones have run is recorded in the table `__drizzle_migrations`. | | **Migrations** | SQL files in [`server/drizzle/`](server/drizzle) (`0000` … `0078`), applied automatically when the server starts. Which ones have run is recorded in the table `__drizzle_migrations`. |
| **Tables** | 87 application tables, plus drizzle's own `__drizzle_migrations`. | | **Tables** | 88 application tables, plus drizzle's own `__drizzle_migrations`. |
**Not in the database, but beside it in the data folder:** the backups of the database itself (`backups/homelab-manager-….sqlite`, taken daily and on request; see [BACKUP.md](BACKUP.md)), and, only while a restore is waiting or has just happened, `restore-pending.sqlite` / `restore-pending.json` and `restore-result.json`. **Not in the database, but beside it in the data folder:** the backups of the database itself (`backups/homelab-manager-….sqlite`, taken daily and on request; see [BACKUP.md](BACKUP.md)), and, only while a restore is waiting or has just happened, `restore-pending.sqlite` / `restore-pending.json` and `restore-result.json`.
@@ -107,10 +107,10 @@ erDiagram
event_webhooks ||--o{ webhook_deliveries : "cascade" event_webhooks ||--o{ webhook_deliveries : "cascade"
``` ```
These 22 tables are without a foreign key to another table, and — but for `integration_credentials`, which `integrations` and `dns_providers` point at — with none from another table to them; where they refer to something else it is by id or name, see These 23 tables are without a foreign key to another table, and — but for `integration_credentials`, which `integrations` and `dns_providers` point at — with none from another table to them; where they refer to something else it is by id or name, see
[soft references](#soft-references-not-foreign-keys): `settings`, `secrets`, `ipam_entries`, `domains`, `diag_log`, `notification_queue`, `consistency_ignores`, `tag_definitions`, [soft references](#soft-references-not-foreign-keys): `settings`, `secrets`, `ipam_entries`, `domains`, `diag_log`, `notification_queue`, `consistency_ignores`, `tag_definitions`,
`maintenance_windows`, `maintenance_aliases`, `maintenance_schedules`, `metrics_tokens`, `restore_tests`, `dependencies`, `runbooks`, `renewals`, `assets`, `notes`, `maintenance_windows`, `maintenance_aliases`, `maintenance_schedules`, `metrics_tokens`, `restore_tests`, `dependencies`, `runbooks`, `renewals`, `assets`, `notes`,
`alert_rules`, `alert_snoozes`, `alert_kind_runbooks` and `integration_credentials`. `alert_rules`, `alert_snoozes`, `alert_kind_runbooks`, `release_cache` and `integration_credentials`.
## Tables ## Tables
@@ -1045,6 +1045,19 @@ How a Grafana (an integration of type `grafana`) is right now, one row each, rep
How the last read went is kept in `settings`, key `_internal:grafanaSync`. How the last read went is kept in `settings`, key `_internal:grafanaSync`.
#### `release_cache`
The newest release of each application there is a connected integration for, as GitHub last said — only while Operations → App versions is switched on. One row for each repository (`owner/name`); no foreign keys.
| Column | Type | Notes |
|---|---|---|
| `repo` | text PK | |
| `tag` | text NOT NULL | The release's tag as published (`v4.0.9.2244`); empty when none has been found. |
| `published_at` | text | ISO. |
| `url` | text | The release's page. |
| `checked_at` | text NOT NULL | ISO: when it was last looked up. |
| `error` | text | Why the last lookup failed; null when it worked. The tag it had is kept. |
#### `n8n_state` #### `n8n_state`
How an n8n (an integration of type `n8n`) is right now, one row each, replaced at every read (every five minutes). `integration_id` is the primary key and a foreign key with **cascade**. How an n8n (an integration of type `n8n`) is right now, one row each, replaced at every read (every five minutes). `integration_id` is the primary key and a foreign key with **cascade**.
@@ -1494,6 +1507,7 @@ that were never changed aren't stored; the app fills in defaults when reading.
| `digest` | The weekly digest: `enabled` (false), `day` (0 = Sunday … 6; default 1, Monday) and `time` (`HH:MM`, default `08:00`), in the notification time zone. | | `digest` | The weekly digest: `enabled` (false), `day` (0 = Sunday … 6; default 1, Monday) and `time` (`HH:MM`, default `08:00`), in the notification time zone. |
| `healthChecks` | `restoreTestDays` (0 = off — how long a guest's last passing restore test counts; a failed one is always raised), `serverOfflineMinutes` (60), `diskUsagePercent` (90), `domainWarnDays` (30), `diskForecastDays` (14; 0 = off — how far ahead a filling disk is flagged on the Alerts page), `backupStaleHours` (48; 0 = off — how old a Proxmox guest's newest backup may be), `newPortAlerts` (true — warn when a server starts listening on a TCP port that is not in its baseline and not noted), `flapRaises` (3; 0 = off — raises of one alert within an hour that make it "flapping" so its messages are held; see NOTIFICATIONS.md), `deviceOfflineMinutes` (10 — how long a UniFi device may be offline before it is raised), `osSupportWarnDays` (90; 0 = off), `baselineAlerts` (true — warn when CPU or memory over the last hour is far above what the server's own history says is normal for this hour of the week; see docs/MONITORING.md), `containerAlerts` (true — warn when a Docker container is broken), `vulnerabilityAlerts` (true — warn when Dockhand's scans found critical vulnerabilities; an older stored setting without it counts as on) and `containerIgnore` (a list of container names or `host/name`, `*` allowed, to leave alone). The baselines themselves are not stored: they are worked out from `server_metrics_rollup` (hourly rows, `grain = 'h'`) when needed. A server can override each of these (see `servers.thresholds`). | | `healthChecks` | `restoreTestDays` (0 = off — how long a guest's last passing restore test counts; a failed one is always raised), `serverOfflineMinutes` (60), `diskUsagePercent` (90), `domainWarnDays` (30), `diskForecastDays` (14; 0 = off — how far ahead a filling disk is flagged on the Alerts page), `backupStaleHours` (48; 0 = off — how old a Proxmox guest's newest backup may be), `newPortAlerts` (true — warn when a server starts listening on a TCP port that is not in its baseline and not noted), `flapRaises` (3; 0 = off — raises of one alert within an hour that make it "flapping" so its messages are held; see NOTIFICATIONS.md), `deviceOfflineMinutes` (10 — how long a UniFi device may be offline before it is raised), `osSupportWarnDays` (90; 0 = off), `baselineAlerts` (true — warn when CPU or memory over the last hour is far above what the server's own history says is normal for this hour of the week; see docs/MONITORING.md), `containerAlerts` (true — warn when a Docker container is broken), `vulnerabilityAlerts` (true — warn when Dockhand's scans found critical vulnerabilities; an older stored setting without it counts as on) and `containerIgnore` (a list of container names or `host/name`, `*` allowed, to leave alone). The baselines themselves are not stored: they are worked out from `server_metrics_rollup` (hourly rows, `grain = 'h'`) when needed. A server can override each of these (see `servers.thresholds`). |
| `logRetention` | `enabled`, `retentionDays` (90), `intervalHours` (24). | | `logRetention` | `enabled`, `retentionDays` (90), `intervalHours` (24). |
| `versionChecks` | `enabled` (false) — whether the newest release of each connected application is looked up on GitHub — and `alertAfterDays` (0 = never): after how many days a newer release is a warning. |
| `display` | `dateFormat`, `timeFormat`, `pageSize`. | | `display` | `dateFormat`, `timeFormat`, `pageSize`. |
| `providerColors`, `integrationColors` | Badge colours, by provider / integration type. | | `providerColors`, `integrationColors` | Badge colours, by provider / integration type. |
| `consistency` | `excludedRanges` — addresses the Consistency and IP Addresses pages ignore. Default `["172.16.0.0/12"]` (Docker's networks). | | `consistency` | `excludedRanges` — addresses the Consistency and IP Addresses pages ignore. Default `["172.16.0.0/12"]` (Docker's networks). |
+11
View File
@@ -13,6 +13,16 @@ the dashboard views themselves load fine. (UniFi, Nginx Proxy Manager, Home Assi
## Integrations ## Integrations
### Checking what an integration can read
On the Integrations page an administrator has an **Inspect** button on each integration that reads through the shared HTTP helper (Authentik, Immich, Jellyfin, NetBox, Nextcloud, Paperless-ngx, Prometheus, Hetzner Cloud, Speedtest Tracker, Fragnet, Sonarr, Radarr, Prowlarr, Lidarr, Readarr, AdGuard Home and Grafana). It runs the integration's reader once and shows:
- **what was asked** — each request, its path (with the *names* of the query parameters, not their values), the status, how long it took and how many bytes;
- **what came back** — the *shape* of each JSON answer to three levels: the names of the fields and the kind of each (`string`, `number`, `array(12) of { … }`), with a `?` on a field that only some items of a list have. Never a value, never a token, so the report can be copied into a message;
- **what the reader made of it**, in the same way, and **where that is empty** — a field that is null, an empty text, an empty list, or null in every item of a list. A reader that expects a field the system names differently ends up here.
It is for the readers written from documentation (see "Not verified" on each integration): put what came back beside what is empty and a mismatch is plain. Something being empty is often right — no containers, nothing wrong — so it is a list to look at, not a verdict. Each inspection is in the audit log; nothing is stored. Only an administrator can run it, and it times out after 90 seconds.
### Tailscale ### Tailscale
- **Config fields:** Tailnet, API key - **Config fields:** Tailnet, API key
@@ -396,6 +406,7 @@ osTicket's API cannot close a ticket, so this is the one place in the app that *
### Sonarr, Radarr and Prowlarr ### Sonarr, Radarr and Prowlarr
- **Config fields:** the application's **URL** and its **API key** (in the application: Settings → General → Security → API Key; stored encrypted), and *Allow self-signed certificate* if it is on HTTPS with its own certificate. Add each application as its own integration, as the kind it is — the *Test* button says if the address turns out to be another one. The key is sent as `X-Api-Key`; the app only ever sends GET requests. - **Config fields:** the application's **URL** and its **API key** (in the application: Settings → General → Security → API Key; stored encrypted), and *Allow self-signed certificate* if it is on HTTPS with its own certificate. Add each application as its own integration, as the kind it is — the *Test* button says if the address turns out to be another one. The key is sent as `X-Api-Key`; the app only ever sends GET requests.
- **Lidarr and Readarr** (music and books) are the same: add them as their own kinds. They answer under `/api/v1` (as Prowlarr does), and ask for their own unknown items in the queue (`includeUnknownArtistItems`, `includeUnknownAuthorItems`). The *Test* button says if an address is another one of the family. Readarr's own project has been retired; the integration reads whatever install you still run.
- **Required access:** the application's address reachable from the app. Sonarr and Radarr answer under `/api/v3`, Prowlarr under `/api/v1`. A read is five to six requests, every five minutes. - **Required access:** the application's address reachable from the app. Sonarr and Radarr answer under `/api/v3`, Prowlarr under `/api/v1`. A read is five to six requests, every five minutes.
- **What it does:** every **five minutes** each application is read — its version, what its own checks found (`/health`), its **download queue** (`/queue`: the first 100, and how many there are), its **disks** (`/diskspace`), for Sonarr and Radarr how many **wanted** items have no file (`/wanted/missing`), and for Prowlarr its **indexers** and which it has switched off after failures (`/indexer`, `/indexerstatus`). **Apps → Media managers** shows them. Nothing is searched, grabbed, removed or changed. - **What it does:** every **five minutes** each application is read — its version, what its own checks found (`/health`), its **download queue** (`/queue`: the first 100, and how many there are), its **disks** (`/diskspace`), for Sonarr and Radarr how many **wanted** items have no file (`/wanted/missing`), and for Prowlarr its **indexers** and which it has switched off after failures (`/indexer`, `/indexerstatus`). **Apps → Media managers** shows them. Nothing is searched, grabbed, removed or changed.
- **Alerts** (sources *Sonarr*, *Radarr*, *Prowlarr*; held back by maintenance on the integration): what the application **says is wrong** with itself (a finding of type warning or error — a download client that cannot be reached, a root folder gone, indexers unavailable; a *notice*, such as an update being available, is shown but not raised); **downloads that are stuck** (in the queue for two hours or more and judged warning or error, failed, or blocked from importing — one alert for the lot, naming the first three and why); a **disk nearly full** (under 10% free, critical under 5%; folders on one disk count once, and a disk under 2 GB is not a media disk). See [MONITORING.md](docs/MONITORING.md#media-managers-sonarr-radarr-prowlarr). - **Alerts** (sources *Sonarr*, *Radarr*, *Prowlarr*; held back by maintenance on the integration): what the application **says is wrong** with itself (a finding of type warning or error — a download client that cannot be reached, a root folder gone, indexers unavailable; a *notice*, such as an update being available, is shown but not raised); **downloads that are stuck** (in the queue for two hours or more and judged warning or error, failed, or blocked from importing — one alert for the lot, naming the first three and why); a **disk nearly full** (under 10% free, critical under 5%; folders on one disk count once, and a disk under 2 GB is not a media disk). See [MONITORING.md](docs/MONITORING.md#media-managers-sonarr-radarr-prowlarr).
+7 -1
View File
@@ -539,12 +539,18 @@ The page also shows the users, the files and the app updates waiting. Whether it
**Game servers** (Infrastructure → Game servers, for Fragnet's game panel). Add an account's API key and the app lists its game servers: the game, the IP addresses and ports (click one to copy it), whether each is running, its memory and disk against the plan, its backups, schedules and databases. It warns when a server's **newest backup is late** (7 days by default, or none at all — a server can be left out by name) and when **disk or memory is nearly full** (90% by default); a server that is merely stopped is not an alert, since game servers are often stopped on purpose. Operators can **start, stop, restart and kill** a server from the page (asked first, written to the audit log); nothing else at Fragnet is changed. Fragnet's panel is a fork of Pterodactyl, so the same integration reads any Pterodactyl panel (give its address). See INTEGRATIONS.md and docs/MONITORING.md. **Game servers** (Infrastructure → Game servers, for Fragnet's game panel). Add an account's API key and the app lists its game servers: the game, the IP addresses and ports (click one to copy it), whether each is running, its memory and disk against the plan, its backups, schedules and databases. It warns when a server's **newest backup is late** (7 days by default, or none at all — a server can be left out by name) and when **disk or memory is nearly full** (90% by default); a server that is merely stopped is not an alert, since game servers are often stopped on purpose. Operators can **start, stop, restart and kill** a server from the page (asked first, written to the audit log); nothing else at Fragnet is changed. Fragnet's panel is a fork of Pterodactyl, so the same integration reads any Pterodactyl panel (give its address). See INTEGRATIONS.md and docs/MONITORING.md.
**Media managers** (Apps → Media managers). Add Sonarr, Radarr and Prowlarr with their address and API key and the app shows, for each, what the application says is wrong with itself, its download queue (what is stuck or failed, and why), the disks it fills, how many wanted items have no file and, for Prowlarr, which indexers have failed. It warns when an application reports a **warning or error** (a download client that cannot be reached, indexers switched off), when **downloads have been stuck for two hours**, and when a **disk is nearly full**. Nothing is searched, grabbed or changed. See INTEGRATIONS.md and docs/MONITORING.md. **Media managers** (Apps → Media managers). Add Sonarr, Radarr, Prowlarr, Lidarr and Readarr with their address and API key and the app shows, for each, what the application says is wrong with itself, its download queue (what is stuck or failed, and why), the disks it fills, how many wanted items have no file and, for Prowlarr, which indexers have failed. It warns when an application reports a **warning or error** (a download client that cannot be reached, indexers switched off), when **downloads have been stuck for two hours**, and when a **disk is nearly full**. Nothing is searched, grabbed or changed. See INTEGRATIONS.md and docs/MONITORING.md.
**AdGuard Home** (Network → AdGuard Home). Add its address (and sign-in, if it has one) and the app reads it every two minutes — and each time asks its DNS server a real question over UDP, because the web interface can be up while DNS is not. It shows whether DNS answers (and how fast), whether protection is on, the queries and how many were blocked, the names blocked most and the clients asking most, and its filter lists with when each was updated. It raises **DNS that has stopped answering** (ten minutes of it, critical), **a DNS server that is not running** (critical), **protection left off for an hour** and **filter lists that have not updated for a week**. Nothing is changed. See INTEGRATIONS.md and docs/MONITORING.md. **AdGuard Home** (Network → AdGuard Home). Add its address (and sign-in, if it has one) and the app reads it every two minutes — and each time asks its DNS server a real question over UDP, because the web interface can be up while DNS is not. It shows whether DNS answers (and how fast), whether protection is on, the queries and how many were blocked, the names blocked most and the clients asking most, and its filter lists with when each was updated. It raises **DNS that has stopped answering** (ten minutes of it, critical), **a DNS server that is not running** (critical), **protection left off for an hour** and **filter lists that have not updated for a week**. Nothing is changed. See INTEGRATIONS.md and docs/MONITORING.md.
**Grafana** (Monitoring → Grafana). Add its address and a service account token (the Viewer role is enough) and the app shows which alert rules are firing or pending, which cannot be evaluated, and which data sources fail Grafana's own health check. It raises **a rule that is firing** (critical when it is labelled critical or page), **a rule that cannot be evaluated** and **a data source that fails its check**. Nothing is silenced or changed. See INTEGRATIONS.md and docs/MONITORING.md. **Grafana** (Monitoring → Grafana). Add its address and a service account token (the Viewer role is enough) and the app shows which alert rules are firing or pending, which cannot be evaluated, and which data sources fail Grafana's own health check. It raises **a rule that is firing** (critical when it is labelled critical or page), **a rule that cannot be evaluated** and **a data source that fails its check**. Nothing is silenced or changed. See INTEGRATIONS.md and docs/MONITORING.md.
**App versions** (Operations → App versions). The version each connected application runs — as its integration last read it — against the newest release of it, with how far behind it is (a patch, a minor update, a major version) and how long the newer release has been out. It looks the newest releases up on GitHub once a day, and **that is the only thing this app asks of an address that is not yours, so it is switched off until an administrator switches it on**. A number of days (0 = never) makes a newer release that has been out that long a warning. Sonarr, Radarr, Prowlarr, Lidarr, Readarr, AdGuard Home, Grafana, Jellyfin, Immich, Paperless-ngx, Nextcloud, NetBox, Authentik and Prometheus are compared. See docs/MONITORING.md.
**Inspect an integration** (Integrations → *Inspect*, administrators). Runs the integration's reader once, now, and shows what each request asked and got back — the **shape** of every answer (the names and kinds of its fields, never the values) — beside what the reader made of it and the fields that came out empty. It is how an integration written from documentation is checked against the real system: an answer with a field the reader did not expect shows up as an empty field. *Copy the report* gives plain text that can be pasted into a message. Each inspection is written to the audit log; nothing is stored or changed.
**Grouping alerts.** On the Alerts page a *Group by* choice puts the alerts under a header for each source (Hetzner, Sonarr …), kind of problem (Disk, Backups …) or type of alert (a Hetzner certificate about to end …) — worst group first, each header showing how many and how serious, each group folding away, and a tick on the header picking everything in it for a bulk action. Remembered per browser; not grouped is the default.
**Speedtest** (Monitoring → Speedtest). Add a Speedtest Tracker integration (its address and an API token with the *results:read* ability) and the app charts how fast the internet has been — download, upload and ping over up to 90 days — and, if you tell it what the line is sold at, warns when **the middle of the last three tests is under 60%** of it, and when **the tests keep failing**. One slow test is not an alert. The app never starts a test. On a status page the integration reads as the internet. See INTEGRATIONS.md and docs/MONITORING.md. **Speedtest** (Monitoring → Speedtest). Add a Speedtest Tracker integration (its address and an API token with the *results:read* ability) and the app charts how fast the internet has been — download, upload and ping over up to 90 days — and, if you tell it what the line is sold at, warns when **the middle of the last three tests is under 60%** of it, and when **the tests keep failing**. One slow test is not an alert. The app never starts a test. On a status page the integration reads as the internet. See INTEGRATIONS.md and docs/MONITORING.md.
**NetBox** (Monitoring → NetBox). Add a NetBox integration (its address and a read-only token): a warning when **no background worker** is running (they are a separate container in the Docker image, and can be down while the pages load), and a list on the page of **the servers you run that have no address in NetBox**. That list is **NetBox** (Monitoring → NetBox). Add a NetBox integration (its address and a read-only token): a warning when **no background worker** is running (they are a separate container in the Docker image, and can be down while the pages load), and a list on the page of **the servers you run that have no address in NetBox**. That list is
+4
View File
@@ -68,6 +68,7 @@ page's own top-level link.
| **Operations** | | | | | | **Operations** | | | | |
| Maintenance | `/maintenance` | ✅ | ✅ | ✅ | | Maintenance | `/maintenance` | ✅ | ✅ | ✅ |
| Updates (and Patch plans) | `/updates` | ✅ | ✅ | ✅ | | Updates (and Patch plans) | `/updates` | ✅ | ✅ | ✅ |
| App versions | `/versions` | ✅ | ✅ | ✅ |
| Coming up | `/coming-up` | ✅ | ✅ | ✅ | | Coming up | `/coming-up` | ✅ | ✅ | ✅ |
| To do | `/todo` | ✅ | ✅ | ✅ | | To do | `/todo` | ✅ | ✅ | ✅ |
| osTicket | `/osticket` | ✅ | ✅ | ✅ | | osTicket | `/osticket` | ✅ | ✅ | ✅ |
@@ -163,6 +164,9 @@ even further, to admin only:
- **Authentik** (Apps → Authentik): everyone signed in sees the outposts, what runs out and the failed-sign-in count (the names of tokens and certificates, not their values); adding one (its address and API token) is an administrator's job, like any integration. - **Authentik** (Apps → Authentik): everyone signed in sees the outposts, what runs out and the failed-sign-in count (the names of tokens and certificates, not their values); adding one (its address and API token) is an administrator's job, like any integration.
- **Jellyfin** (Apps → Jellyfin): everyone signed in sees who is streaming and what is wrong; adding one (its address and API key) is an administrator's job, like any integration. Starting an update run on a server people are streaming from needs `startAnyway` — the update drawer asks (operators). - **Jellyfin** (Apps → Jellyfin): everyone signed in sees who is streaming and what is wrong; adding one (its address and API key) is an administrator's job, like any integration. Starting an update run on a server people are streaming from needs `startAnyway` — the update drawer asks (operators).
- **Media managers** (Apps → Media managers, for Sonarr, Radarr and Prowlarr): everyone signed in sees what each says is wrong, its queue, disks and indexers; adding one (its API key) is an administrator's job, like any integration. Nothing on the page changes anything in them. - **Media managers** (Apps → Media managers, for Sonarr, Radarr and Prowlarr): everyone signed in sees what each says is wrong, its queue, disks and indexers; adding one (its API key) is an administrator's job, like any integration. Nothing on the page changes anything in them.
- **App versions** (Operations → App versions): everyone signed in sees what each application runs against the newest release; switching the lookup of releases on and off, and the number of days after which a newer release is a warning, are an administrator's.
- **Inspecting an integration** (Integrations → Inspect): administrators only — it makes the integration's requests, here and now (and says what came back, as shapes without values); each one is in the audit log.
- **Grouping the alerts** is each person's own view, remembered in their browser; it needs no role.
- **AdGuard Home** (Network → AdGuard Home) and **Grafana** (Monitoring → Grafana): everyone signed in sees what each says is wrong, the statistics, lists, rules and data sources; adding one (its sign-in or token) is an administrator's job, like any integration. Nothing on the pages changes anything in them. - **AdGuard Home** (Network → AdGuard Home) and **Grafana** (Monitoring → Grafana): everyone signed in sees what each says is wrong, the statistics, lists, rules and data sources; adding one (its sign-in or token) is an administrator's job, like any integration. Nothing on the pages changes anything in them.
- **Arranging a server's page**: folding the cards of a server's overview and putting them in order is each person's own (kept on their account, the same for every server) — it needs no role. - **Arranging a server's page**: folding the cards of a server's overview and putting them in order is each person's own (kept on their account, the same for every server) — it needs no role.
- **Duplicati** (Backups → Duplicati): everyone signed in sees every backup and how it went; setting how often one is expected, muting it and forgetting it need operator; adding a machine (which makes an address that can write to this app) and removing one are administrators'. Reports arrive on a secret address with no sign-in: the token is the access. - **Duplicati** (Backups → Duplicati): everyone signed in sees every backup and how it went; setting how often one is expected, muting it and forgetting it need operator; adding a machine (which makes an address that can write to this app) and removing one are administrators'. Reports arrive on a secret address with no sign-in: the token is the access.
+15 -1
View File
@@ -17,6 +17,7 @@ This page explains what Homelab Manager watches, how often, how a problem become
- [Hetzner Cloud](#hetzner-cloud) - [Hetzner Cloud](#hetzner-cloud)
- [Game servers (Fragnet)](#game-servers-fragnet) - [Game servers (Fragnet)](#game-servers-fragnet)
- [Media managers](#media-managers-sonarr-radarr-prowlarr) - [Media managers](#media-managers-sonarr-radarr-prowlarr)
- [App versions](#app-versions)
- [AdGuard Home](#adguard-home) - [AdGuard Home](#adguard-home)
- [Grafana](#grafana) - [Grafana](#grafana)
- [Speedtest](#speedtest) - [Speedtest](#speedtest)
@@ -162,6 +163,7 @@ Every threshold here is a default; where it can be changed, the place is named.
| **Docker image has critical vulnerabilities** | Dockhand's own scans of a Docker host's images found one or more *critical* findings (high, medium and low are only shown — unless you set a number for high ones, below) | on | Settings → Notifications; [below](#docker-containers-that-are-broken) | | **Docker image has critical vulnerabilities** | Dockhand's own scans of a Docker host's images found one or more *critical* findings (high, medium and low are only shown — unless you set a number for high ones, below) | on | Settings → Notifications; [below](#docker-containers-that-are-broken) |
| **Docker host has many high vulnerabilities** | The same scans found at least the number of *high* findings you set (*Also warn when a Docker host has this many high vulnerabilities*; 0, the default, never) | off | Settings → Notifications; [below](#docker-containers-that-are-broken) | | **Docker host has many high vulnerabilities** | The same scans found at least the number of *high* findings you set (*Also warn when a Docker host has this many high vulnerabilities*; 0, the default, never) | off | Settings → Notifications; [below](#docker-containers-that-are-broken) |
| **Docker host has old running images** | A Docker host has running containers whose images were built more than the number of days you set ago (*Warn about running Docker images that were built more than …*; 0, the default, never) — scanned or not | off | Settings → Notifications; [below](#docker-containers-that-are-broken) | | **Docker host has old running images** | A Docker host has running containers whose images were built more than the number of days you set ago (*Warn about running Docker images that were built more than …*; 0, the default, never) — scanned or not | off | Settings → Notifications; [below](#docker-containers-that-are-broken) |
| **App behind its newest release** | An application has had a newer release out for the number of days set under Operations → App versions (0, the default, never) — needs the lookup of releases switched on | off | App versions page; [below](#app-versions) |
| **Automation failed** | A Semaphore template's or Gitea repository's latest run failed (a run still going or cancelled does not count) | | not configurable | | **Automation failed** | A Semaphore template's or Gitea repository's latest run failed (a run still going or cancelled does not count) | | not configurable |
| **Integration failing** | An integration's calls have failed several times in a row ([Integrations](#integrations)) | 3 calls | Settings → Notifications | | **Integration failing** | An integration's calls have failed several times in a row ([Integrations](#integrations)) | 3 calls | Settings → Notifications |
| **Endpoint check failing / certificate running out** | See [below](#endpoint-checks-and-heartbeats) | | per check | | **Endpoint check failing / certificate running out** | See [below](#endpoint-checks-and-heartbeats) | | per check |
@@ -288,7 +290,7 @@ Hetzner does not say when a server went off or when rescue mode was switched on,
## Media managers (Sonarr, Radarr, Prowlarr) ## Media managers (Sonarr, Radarr, Prowlarr)
An integration of type *Sonarr*, *Radarr* or *Prowlarr* ([INTEGRATIONS.md](../INTEGRATIONS.md#sonarr-radarr-and-prowlarr)) is read every five minutes, and **Apps → Media managers** shows, for each: what the application says is wrong with itself, its download queue (the stuck ones first, with why), its disks, how many wanted items have no file (Sonarr, Radarr) and, for Prowlarr, which indexers it has switched off after failures. Nothing is searched, grabbed or changed. An integration of type *Sonarr*, *Radarr*, *Prowlarr*, *Lidarr* or *Readarr* ([INTEGRATIONS.md](../INTEGRATIONS.md#sonarr-radarr-and-prowlarr)) is read every five minutes, and **Apps → Media managers** shows, for each: what the application says is wrong with itself, its download queue (the stuck ones first, with why), its disks, how many wanted items have no file (Sonarr, Radarr) and, for Prowlarr, which indexers it has switched off after failures. Nothing is searched, grabbed or changed.
| Alert | When | Severity | | Alert | When | Severity |
|---|---|---| |---|---|---|
@@ -335,6 +337,18 @@ An integration of type *Grafana* ([INTEGRATIONS.md](../INTEGRATIONS.md#grafana))
Grafana can notify about its own alerts as well; with both switched on you hear twice. Maintenance on the integration holds the alerts; a reading older than 20 minutes is not believed. `/metrics` has the rules, those firing, those in error and the failing data sources. Grafana can notify about its own alerts as well; with both switched on you hear twice. Maintenance on the integration holds the alerts; a reading older than 20 minutes is not believed. `/metrics` has the rules, those firing, those in error and the failing data sources.
## App versions
**Operations → App versions** lists each connected application whose version is known, with the newest release of it and how far behind it is: *a patch* (the third number), *a minor update* (the second), *a major version* (the first; for Authentik, whose versions are dates, a new year). A build number after the third number is ignored, and prefixes (`v`, `n8n@`, `version/`) are taken off. What is newer here than the newest release (a beta) is *ahead*.
The newest release is asked of **GitHub** (`GET /repos/{owner}/{repo}/releases/latest`, which leaves out pre-releases and drafts) **once a day, and only for an application whose version has been read** — and **only while an administrator has switched the lookup on**. It sends nothing about you or your systems but the name of the repository, and is the only thing this app asks of an address that is not yours. GitHub limits requests without a sign-in (60 an hour); a lookup that is refused keeps the release it had, says why on the page and is tried again at the next pass.
| Alert | When | Severity |
|---|---|---|
| **App behind its newest release** | A newer release has been out for the number of days set (*Warn when a newer release has been out for … days*; 0, the default, never) and the application has not caught up. The alert says which release, how long, and whether it is a patch, a minor update or a new major version | warning |
Maintenance on the integration holds the alert. It is under *Updates* on the Alerts page. `/metrics` has `homelab_app_update_available` (1 if a newer release is out) for each application, while the lookup is on.
## Speedtest ## Speedtest
An integration of type *Speedtest Tracker* ([INTEGRATIONS.md](../INTEGRATIONS.md#speedtest-tracker)) is read every ten minutes, and **Monitoring → Speedtest** shows the newest test, the middle of the last three and charts of download, upload and ping over up to 90 days. This app never starts a test: Speedtest Tracker runs them on its own schedule. An integration of type *Speedtest Tracker* ([INTEGRATIONS.md](../INTEGRATIONS.md#speedtest-tracker)) is read every ten minutes, and **Monitoring → Speedtest** shows the newest test, the middle of the last three and charts of download, upload and ping over up to 90 days. This app never starts a test: Speedtest Tracker runs them on its own schedule.
+1
View File
@@ -313,6 +313,7 @@ All gauges. The same table is checked by a test against what `/metrics` really s
| `homelab_arr_queue_items`, `homelab_arr_queue_stuck_items` | `app`, `name` | The downloads in the queue of Sonarr or Radarr, and how many are stuck (judged wrong, failed or blocked for two hours or more). | | `homelab_arr_queue_items`, `homelab_arr_queue_stuck_items` | `app`, `name` | The downloads in the queue of Sonarr or Radarr, and how many are stuck (judged wrong, failed or blocked for two hours or more). |
| `homelab_arr_wanted_missing` | `app`, `name` | Wanted items Sonarr or Radarr has no file for. | | `homelab_arr_wanted_missing` | `app`, `name` | Wanted items Sonarr or Radarr has no file for. |
| `homelab_arr_disk_free_bytes` | `app`, `name`, `path` | Free space of a disk Sonarr, Radarr or Prowlarr uses. | | `homelab_arr_disk_free_bytes` | `app`, `name`, `path` | Free space of a disk Sonarr, Radarr or Prowlarr uses. |
| `homelab_app_update_available` | `app`, `name`, `running`, `latest` | 1 if a newer release of the application is out than the one it runs, else 0. Only while Operations → App versions is switched on, and for an application whose newest release has been looked up. |
| `homelab_adguard_running`, `homelab_adguard_protection_enabled`, `homelab_adguard_dns_up` | `name` | 1 if AdGuard Home's DNS server runs, if its protection is on, and if it answered the app's test DNS question; else 0. | | `homelab_adguard_running`, `homelab_adguard_protection_enabled`, `homelab_adguard_dns_up` | `name` | 1 if AdGuard Home's DNS server runs, if its protection is on, and if it answered the app's test DNS question; else 0. |
| `homelab_adguard_dns_probe_seconds` | `name` | How long it took to answer that question; left out when it did not. | | `homelab_adguard_dns_probe_seconds` | `name` | How long it took to answer that question; left out when it did not. |
| `homelab_adguard_queries`, `homelab_adguard_blocked`, `homelab_adguard_avg_processing_seconds` | `name` | The queries it answered in its statistics period, how many it blocked, and its average processing time. | | `homelab_adguard_queries`, `homelab_adguard_blocked`, `homelab_adguard_avg_processing_seconds` | `name` | The queries it answered in its statistics period, how many it blocked, and its average processing time. |
@@ -0,0 +1,8 @@
CREATE TABLE `release_cache` (
`repo` text PRIMARY KEY NOT NULL,
`tag` text NOT NULL,
`published_at` text,
`url` text,
`checked_at` text NOT NULL,
`error` text
);
File diff suppressed because it is too large. Load diff
+7
View File
@@ -547,6 +547,13 @@
"when": 1793134873418, "when": 1793134873418,
"tag": "0077_opposite_catseye", "tag": "0077_opposite_catseye",
"breakpoints": true "breakpoints": true
},
{
"idx": 78,
"version": "6",
"when": 1793138206158,
"tag": "0078_sturdy_typhoid_mary",
"breakpoints": true
} }
] ]
} }
+2
View File
@@ -73,6 +73,7 @@ import { arrRouter } from "./routes/arr.js";
import { wallRouter } from "./routes/wall.js"; import { wallRouter } from "./routes/wall.js";
import { adguardRouter } from "./routes/adguard.js"; import { adguardRouter } from "./routes/adguard.js";
import { grafanaRouter } from "./routes/grafana.js"; import { grafanaRouter } from "./routes/grafana.js";
import { versionsRouter } from "./routes/versions.js";
import { costsRouter } from "./routes/costs.js"; import { costsRouter } from "./routes/costs.js";
import { nextcloudRouter } from "./routes/nextcloud.js"; import { nextcloudRouter } from "./routes/nextcloud.js";
import { serviceLevelsRouter } from "./routes/serviceLevels.js"; import { serviceLevelsRouter } from "./routes/serviceLevels.js";
@@ -224,6 +225,7 @@ export function createApp() {
app.use("/api/wall", wallRouter); app.use("/api/wall", wallRouter);
app.use("/api/adguard", adguardRouter); app.use("/api/adguard", adguardRouter);
app.use("/api/grafana", grafanaRouter); app.use("/api/grafana", grafanaRouter);
app.use("/api/versions", versionsRouter);
app.use("/api/costs", costsRouter); app.use("/api/costs", costsRouter);
app.use("/api/nextcloud", nextcloudRouter); app.use("/api/nextcloud", nextcloudRouter);
app.use("/api/service-levels", serviceLevelsRouter); app.use("/api/service-levels", serviceLevelsRouter);
+15
View File
@@ -572,6 +572,8 @@ export const integrationTypes = [
"prowlarr", "prowlarr",
"adguard", "adguard",
"grafana", "grafana",
"lidarr",
"readarr",
] as const; ] as const;
export type IntegrationType = (typeof integrationTypes)[number]; export type IntegrationType = (typeof integrationTypes)[number];
@@ -1364,6 +1366,19 @@ export const grafanaState = sqliteTable("grafana_state", {
updatedAt: text("updated_at").notNull(), // ISO updatedAt: text("updated_at").notNull(), // ISO
}); });
// ─── The newest releases ────────────────────────────────────────────────────
// The newest release of each application there is a connected integration for, as GitHub last said (only while Operations → Versions is switched on): one row for each repository. A lookup that
// failed keeps the tag it had and says why in `error`. Read by services/appVersions.ts.
export const releaseCache = sqliteTable("release_cache", {
repo: text("repo").primaryKey(), // "owner/name"
tag: text("tag").notNull(), // the release's tag, as published ("v4.0.9.2244"); empty when none has been found
publishedAt: text("published_at"), // ISO
url: text("url"), // the release's page
checkedAt: text("checked_at").notNull(), // ISO: when it was last looked up
error: text("error"), // why the last lookup failed; null when it worked
});
// ─── NetBox ───────────────────────────────────────────────────────────────── // ─── NetBox ─────────────────────────────────────────────────────────────────
// How a NetBox (an integration of type "netbox") is right now, one row each, replaced at every read: its version, how many background workers are running, the counts and the IP // How a NetBox (an integration of type "netbox") is right now, one row each, replaced at every read: its version, how many background workers are running, the counts and the IP
+2
View File
@@ -44,6 +44,7 @@ import { initFragnetSync } from "./services/fragnetSync.js";
import { initArrSync } from "./services/arrSync.js"; import { initArrSync } from "./services/arrSync.js";
import { initAdguardSync } from "./services/adguardSync.js"; import { initAdguardSync } from "./services/adguardSync.js";
import { initGrafanaSync } from "./services/grafanaSync.js"; import { initGrafanaSync } from "./services/grafanaSync.js";
import { initVersionCheck } from "./services/appVersions.js";
import { backfillMetricRollups } from "./services/serverMetrics.js"; import { backfillMetricRollups } from "./services/serverMetrics.js";
import { backfillEndpointHourly } from "./services/endpointChecks.js"; import { backfillEndpointHourly } from "./services/endpointChecks.js";
import { initSelfBackupScheduler } from "./services/selfBackupScheduler.js"; import { initSelfBackupScheduler } from "./services/selfBackupScheduler.js";
@@ -109,6 +110,7 @@ initFragnetSync();
initArrSync(); initArrSync();
initAdguardSync(); initAdguardSync();
initGrafanaSync(); initGrafanaSync();
initVersionCheck();
// The hourly and daily figures are built from what is already kept, once, the first time after they were added (in the background: nothing waits for it). // The hourly and daily figures are built from what is already kept, once, the first time after they were added (in the background: nothing waits for it).
void Promise.all([backfillMetricRollups(), backfillEndpointHourly()]).catch((err) => console.error("[history] couldn't build the hourly and daily figures:", err)); void Promise.all([backfillMetricRollups(), backfillEndpointHourly()]).catch((err) => console.error("[history] couldn't build the hourly and daily figures:", err));
+12 -8
View File
@@ -1,20 +1,24 @@
/** /**
* Sonarr, Radarr and Prowlarr adapter — the three speak the same API (the "*arr" family), so one adapter serves all of them, told which it is. Config: url, apiKey (the application's own: Settings → * Sonarr, Radarr, Prowlarr, Lidarr and Readarr adapter — they speak the same API (the "*arr" family), so one adapter serves all of them, told which it is. Config: url, apiKey (the application's own: Settings →
* General → Security → API Key; sent as `X-Api-Key`) and optionally insecure. * General → Security → API Key; sent as `X-Api-Key`) and optionally insecure.
* *
* Reading only: what the application says is wrong with itself (`/health`), its download queue (`/queue`: what is downloading, what is stuck or failed and why), the disks it uses * Reading only: what the application says is wrong with itself (`/health`), its download queue (`/queue`: what is downloading, what is stuck or failed and why), the disks it uses
* (`/diskspace`), how many wanted items are missing (`/wanted/missing`, Sonarr and Radarr) and — for Prowlarr — its indexers and which of them it has switched off after failures (`/indexer`, * (`/diskspace`), how many wanted items are missing (`/wanted/missing`, all but Prowlarr) and — for Prowlarr — its indexers and which of them it has switched off after failures (`/indexer`,
* `/indexerstatus`). Nothing is sent but GET requests: no search is started, nothing is grabbed, removed or changed. * `/indexerstatus`). Nothing is sent but GET requests: no search is started, nothing is grabbed, removed or changed.
* *
* Sonarr and Radarr answer under `/api/v3`, Prowlarr under `/api/v1`. Endpoints and field names are from their published OpenAPI files and source (HealthResource, QueueResource, * Sonarr and Radarr answer under `/api/v3`; Prowlarr, Lidarr and Readarr under `/api/v1`. Endpoints and field names are from their published OpenAPI files and source (HealthResource, QueueResource,
* DiskSpaceResource, IndexerStatusResource) and have not been checked against live installs here; every field is read defensively. * DiskSpaceResource, IndexerStatusResource) and have not been checked against live installs here; every field is read defensively.
*/ */
import { requestJson } from "../httpJson.js"; import { requestJson } from "../httpJson.js";
import { withDiagLogging } from "../../services/diagLog.js"; import { withDiagLogging } from "../../services/diagLog.js";
export type ArrKind = "sonarr" | "radarr" | "prowlarr"; export type ArrKind = "sonarr" | "radarr" | "prowlarr" | "lidarr" | "readarr";
export const ARR_KINDS: ArrKind[] = ["sonarr", "radarr", "prowlarr"]; export const ARR_KINDS: ArrKind[] = ["sonarr", "radarr", "prowlarr", "lidarr", "readarr"];
export const ARR_NAMES: Record<ArrKind, string> = { sonarr: "Sonarr", radarr: "Radarr", prowlarr: "Prowlarr" }; export const ARR_NAMES: Record<ArrKind, string> = { sonarr: "Sonarr", radarr: "Radarr", prowlarr: "Prowlarr", lidarr: "Lidarr", readarr: "Readarr" };
/** The version of the API each answers under. */
const API_VERSION: Record<ArrKind, string> = { sonarr: "v3", radarr: "v3", prowlarr: "v1", lidarr: "v1", readarr: "v1" };
/** What each calls the items in its queue that it cannot match to anything it manages (asked for, so the queue is whole). */
const QUEUE_UNKNOWN: Record<ArrKind, string> = { sonarr: "&includeUnknownSeriesItems=true", radarr: "&includeUnknownMovieItems=true", prowlarr: "", lidarr: "&includeUnknownArtistItems=true", readarr: "&includeUnknownAuthorItems=true" };
export interface ArrConfig { export interface ArrConfig {
url: string; url: string;
@@ -181,7 +185,7 @@ export function parseIndexers(indexers: unknown, status: unknown): ArrIndexer[]
} }
export function createArrAdapter(kind: ArrKind, config: ArrConfig): ArrAdapter { export function createArrAdapter(kind: ArrKind, config: ArrConfig): ArrAdapter {
const base = `${String(config.url ?? "").trim().replace(/\/+$/, "")}/api/${kind === "prowlarr" ? "v1" : "v3"}`; const base = `${String(config.url ?? "").trim().replace(/\/+$/, "")}/api/${API_VERSION[kind]}`;
const insecure = config.insecure === true; const insecure = config.insecure === true;
const headers = { "X-Api-Key": String(config.apiKey ?? "").trim(), Accept: "application/json" }; const headers = { "X-Api-Key": String(config.apiKey ?? "").trim(), Accept: "application/json" };
const name = ARR_NAMES[kind]; const name = ARR_NAMES[kind];
@@ -207,7 +211,7 @@ export function createArrAdapter(kind: ArrKind, config: ArrConfig): ArrAdapter {
// The status and the health must work: they are what says it is there and what is wrong. // The status and the health must work: they are what says it is there and what is wrong.
const status = obj(await get("/system/status")); const status = obj(await get("/system/status"));
const health = parseHealth(await get("/health")); const health = parseHealth(await get("/health"));
const include = kind === "sonarr" ? "&includeUnknownSeriesItems=true" : kind === "radarr" ? "&includeUnknownMovieItems=true" : ""; const include = QUEUE_UNKNOWN[kind];
const [queue, disks, missing, indexers, indexerStatus] = await Promise.all([ const [queue, disks, missing, indexers, indexerStatus] = await Promise.all([
kind === "prowlarr" ? Promise.resolve(null) : soft(async () => parseQueue(await get(`/queue?page=1&pageSize=${QUEUE_PAGE}${include}`))), kind === "prowlarr" ? Promise.resolve(null) : soft(async () => parseQueue(await get(`/queue?page=1&pageSize=${QUEUE_PAGE}${include}`))),
soft(async () => parseDisks(await get("/diskspace"))), soft(async () => parseDisks(await get("/diskspace"))),
+10
View File
@@ -124,6 +124,16 @@ export const INTEGRATION_FIELDS: Partial<Record<IntegrationType, IntegrationFiel
{ key: "apiKey", label: "API key", secret: true, type: "password", placeholder: "Settings → General → Security → API Key" }, { key: "apiKey", label: "API key", secret: true, type: "password", placeholder: "Settings → General → Security → API Key" },
{ key: "insecure", label: "Allow self-signed certificate", secret: false, type: "checkbox" }, { key: "insecure", label: "Allow self-signed certificate", secret: false, type: "checkbox" },
], ],
lidarr: [
{ key: "url", label: "Lidarr URL", secret: false, placeholder: "http://lidarr.example.lan:8686" },
{ key: "apiKey", label: "API key", secret: true, type: "password", placeholder: "Settings → General → Security → API Key" },
{ key: "insecure", label: "Allow self-signed certificate", secret: false, type: "checkbox" },
],
readarr: [
{ key: "url", label: "Readarr URL", secret: false, placeholder: "http://readarr.example.lan:8787" },
{ key: "apiKey", label: "API key", secret: true, type: "password", placeholder: "Settings → General → Security → API Key" },
{ key: "insecure", label: "Allow self-signed certificate", secret: false, type: "checkbox" },
],
prowlarr: [ prowlarr: [
{ key: "url", label: "Prowlarr URL", secret: false, placeholder: "http://prowlarr.example.lan:9696" }, { key: "url", label: "Prowlarr URL", secret: false, placeholder: "http://prowlarr.example.lan:9696" },
{ key: "apiKey", label: "API key", secret: true, type: "password", placeholder: "Settings → General → Security → API Key" }, { key: "apiKey", label: "API key", secret: true, type: "password", placeholder: "Settings → General → Security → API Key" },
+11 -1
View File
@@ -4,6 +4,7 @@
*/ */
import * as http from "node:http"; import * as http from "node:http";
import * as https from "node:https"; import * as https from "node:https";
import { isInspecting, pathForReport, recordCall, shapeOf } from "./inspector.js";
export interface JsonResponse { export interface JsonResponse {
status: number; status: number;
@@ -27,6 +28,8 @@ export function requestJson(url: string, opts: { method?: "GET" | "POST"; header
const lib = isHttps ? https : http; const lib = isHttps ? https : http;
const payload = opts.body === undefined ? null : JSON.stringify(opts.body); const payload = opts.body === undefined ? null : JSON.stringify(opts.body);
const limit = opts.timeoutMs ?? TIMEOUT_MS; const limit = opts.timeoutMs ?? TIMEOUT_MS;
const started = Date.now();
const method = opts.method ?? "GET";
const req = lib.request( const req = lib.request(
{ {
hostname: parsed.hostname, hostname: parsed.hostname,
@@ -35,6 +38,8 @@ export function requestJson(url: string, opts: { method?: "GET" | "POST"; header
method: opts.method ?? "GET", method: opts.method ?? "GET",
headers: { Accept: "application/json", ...(payload ? { "Content-Type": "application/json", "Content-Length": String(Buffer.byteLength(payload)) } : {}), ...opts.headers }, headers: { Accept: "application/json", ...(payload ? { "Content-Type": "application/json", "Content-Length": String(Buffer.byteLength(payload)) } : {}), ...opts.headers },
timeout: limit, timeout: limit,
// A new connection for each request, not one kept from the last: a system that has closed an idle connection (most do, within seconds) would otherwise answer the next read, minutes later, with a reset.
agent: false,
...(isHttps ? { rejectUnauthorized: !opts.insecure } : {}), ...(isHttps ? { rejectUnauthorized: !opts.insecure } : {}),
}, },
(res) => { (res) => {
@@ -50,12 +55,17 @@ export function requestJson(url: string, opts: { method?: "GET" | "POST"; header
} catch { } catch {
// not JSON: the text is still there // not JSON: the text is still there
} }
// While the inspector runs a read, what was asked and the shape of the answer (never its values) are written down.
if (isInspecting()) recordCall({ method, path: pathForReport(parsed), status: res.statusCode ?? 0, ms: Date.now() - started, bytes: Buffer.byteLength(text), shape: json === null ? null : shapeOf(json) });
resolve({ status: res.statusCode ?? 0, json, text, headers: res.headers }); resolve({ status: res.statusCode ?? 0, json, text, headers: res.headers });
}); });
}, },
); );
req.on("timeout", () => req.destroy(new Error(`timed out after ${limit / 1000}s`))); req.on("timeout", () => req.destroy(new Error(`timed out after ${limit / 1000}s`)));
req.on("error", reject); req.on("error", (err) => {
if (isInspecting()) recordCall({ method, path: pathForReport(parsed), status: 0, ms: Date.now() - started, bytes: 0, shape: null, error: err.message });
reject(err);
});
if (payload) req.write(payload); if (payload) req.write(payload);
req.end(); req.end();
}); });
+123
View File
@@ -0,0 +1,123 @@
/**
* What lets an integration be inspected: while a read is run for the inspector, every request the shared HTTP helper (httpJson.ts) makes is written down — what was asked, what came back, how long
* it took, and the *shape* of the answer — so that what an integration's reader makes of an answer can be put beside what the answer was. Only shapes are kept: the names and kinds of the values
* (`{ id: number, name: string, tags: array(3) of string }`), never the values themselves, so a report can be copied into a message without giving away what the system holds.
*/
import { AsyncLocalStorage } from "node:async_hooks";
export interface InspectedCall {
method: string;
/** The path, with the names of the query parameters but not their values. */
path: string;
/** 0 when there was no answer (a refused connection, a timeout). */
status: number;
ms: number;
bytes: number;
/** What the answer looked like; null when it was not JSON or there was none. */
shape: string | null;
error?: string;
}
const store = new AsyncLocalStorage<InspectedCall[]>();
export const isInspecting = (): boolean => store.getStore() !== undefined;
export function recordCall(call: InspectedCall): void {
store.getStore()?.push(call);
}
/** Runs something and returns every request the helper made while it ran, whatever happened to it. */
export async function inspectCalls<T>(run: () => Promise<T>): Promise<{ calls: InspectedCall[]; value: T | undefined; error: string | null }> {
const calls: InspectedCall[] = [];
return store.run(calls, async () => {
try {
return { calls, value: await run(), error: null };
} catch (err) {
return { calls, value: undefined, error: err instanceof Error ? err.message : String(err) };
}
});
}
/** Pure: a path for a report — the query's parameter names, not what they were set to. */
export function pathForReport(url: URL): string {
const names = [...new Set([...url.searchParams.keys()])];
return url.pathname + (names.length > 0 ? `?${names.join("&")}` : "");
}
const MAX_SHAPE = 1800;
const MAX_KEYS = 30;
const MAX_DEPTH = 3;
const SAMPLE = 20;
const kindOf = (v: unknown): string => (v === null ? "null" : Array.isArray(v) ? "array" : typeof v);
function describe(value: unknown, depth: number): string {
if (Array.isArray(value)) {
if (value.length === 0) return "array(0)";
const sample = value.slice(0, SAMPLE);
const objects = sample.filter((x) => x !== null && typeof x === "object" && !Array.isArray(x)) as Record<string, unknown>[];
if (objects.length === sample.length && depth < MAX_DEPTH) {
// Objects: the keys of all of them, marked with ? where some of them lack it.
const keys = new Map<string, { count: number; first: unknown }>();
for (const o of objects) for (const [k, v] of Object.entries(o)) keys.set(k, { count: (keys.get(k)?.count ?? 0) + 1, first: keys.get(k)?.first ?? v });
const parts = [...keys.entries()].slice(0, MAX_KEYS).map(([k, { count, first }]) => `${k}${count < objects.length ? "?" : ""}: ${describe(first, depth + 1)}`);
return `array(${value.length}) of { ${parts.join(", ")}${keys.size > MAX_KEYS ? ", …" : ""} }`;
}
const kinds = [...new Set(sample.map(kindOf))];
return `array(${value.length}) of ${kinds.join(" | ")}`;
}
if (value !== null && typeof value === "object") {
if (depth >= MAX_DEPTH) return "object";
const entries = Object.entries(value as Record<string, unknown>);
if (entries.length === 0) return "{}";
return `{ ${entries
.slice(0, MAX_KEYS)
.map(([k, v]) => `${k}: ${describe(v, depth + 1)}`)
.join(", ")}${entries.length > MAX_KEYS ? ", …" : ""} }`;
}
return kindOf(value);
}
/** Pure: what a JSON value looks like — its keys and the kinds of what they hold, to three levels, never the values. */
export function shapeOf(value: unknown): string {
const text = describe(value, 0);
return text.length > MAX_SHAPE ? `${text.slice(0, MAX_SHAPE)} …` : text;
}
export interface EmptyField {
path: string;
/** null: the field is null; text: it is an empty string; list: an empty list; every-item: null in every item of a list. */
what: "null" | "text" | "list" | "every-item";
}
/**
* Pure: the places in what a reader made of an answer where there is nothing — a field that is null, an empty text, an empty list, or a field that is null in *every* item of a list. A reader
* that expects a field the system names differently ends up here: it is where a mismatch shows. (Something being empty is often right — no containers, nothing wrong — which is why this is
* a list to look at, not a verdict.)
*/
export function emptyFields(value: unknown, limit = 40): EmptyField[] {
const out: EmptyField[] = [];
const add = (path: string, what: EmptyField["what"]) => {
if (out.length < limit) out.push({ path: path || "(the result)", what });
};
const walk = (v: unknown, path: string, depth: number) => {
if (out.length >= limit || depth > MAX_DEPTH) return;
if (v === null || v === undefined) return add(path, "null");
if (typeof v === "string") return v.trim() === "" ? add(path, "text") : undefined;
if (Array.isArray(v)) {
if (v.length === 0) return add(path, "list");
const items = v.slice(0, 50).filter((x) => x !== null && typeof x === "object" && !Array.isArray(x)) as Record<string, unknown>[];
if (items.length === 0 || depth >= MAX_DEPTH) return;
const keys = new Set(items.flatMap((o) => Object.keys(o)));
for (const k of keys) {
const values = items.map((o) => o[k]);
if (values.every((x) => x === null || x === undefined)) add(`${path}[].${k}`, "every-item");
else if (values.some((x) => x !== null && typeof x === "object")) walk(values.find((x) => x !== null && typeof x === "object"), `${path}[].${k}`, depth + 1);
}
return;
}
if (typeof v === "object") for (const [k, x] of Object.entries(v as Record<string, unknown>)) walk(x, path ? `${path}.${k}` : k, depth + 1);
};
walk(value, "", 0);
return out;
}
+2
View File
@@ -91,6 +91,8 @@ export function createIntegrationAdapter(type: IntegrationType, config: Integrat
case "sonarr": case "sonarr":
case "radarr": case "radarr":
case "prowlarr": case "prowlarr":
case "lidarr":
case "readarr":
return createArrAdapter(type, config as any); return createArrAdapter(type, config as any);
case "adguard": case "adguard":
return createAdguardAdapter(config as any); return createAdguardAdapter(config as any);
+36 -3
View File
@@ -14,6 +14,7 @@ import {
} from "../integrations/fieldSchemas.js"; } from "../integrations/fieldSchemas.js";
import { createIntegrationAdapter } from "../integrations/registry.js"; import { createIntegrationAdapter } from "../integrations/registry.js";
import { loadIntegrationConfig } from "../integrations/loadIntegration.js"; import { loadIntegrationConfig } from "../integrations/loadIntegration.js";
import { emptyFields, inspectCalls, shapeOf } from "../integrations/inspector.js";
import { createTailscaleAdapter, isKeyExpiringSoon } from "../integrations/tailscale/adapter.js"; import { createTailscaleAdapter, isKeyExpiringSoon } from "../integrations/tailscale/adapter.js";
import { createGiteaAdapter } from "../integrations/gitea/adapter.js"; import { createGiteaAdapter } from "../integrations/gitea/adapter.js";
import { createDockhandAdapter } from "../integrations/dockhand/adapter.js"; import { createDockhandAdapter } from "../integrations/dockhand/adapter.js";
@@ -40,6 +41,7 @@ import { forgetHetzner, syncHetzner } from "../services/hetznerSync.js";
import { forgetSpeedtest, syncSpeedtest } from "../services/speedtestSync.js"; import { forgetSpeedtest, syncSpeedtest } from "../services/speedtestSync.js";
import { forgetFragnet, syncFragnet } from "../services/fragnetSync.js"; import { forgetFragnet, syncFragnet } from "../services/fragnetSync.js";
import { forgetArr, syncArr } from "../services/arrSync.js"; import { forgetArr, syncArr } from "../services/arrSync.js";
import { ARR_KINDS } from "../integrations/arr/adapter.js";
import { forgetAdguard, syncAdguard } from "../services/adguardSync.js"; import { forgetAdguard, syncAdguard } from "../services/adguardSync.js";
import { forgetGrafana, syncGrafana } from "../services/grafanaSync.js"; import { forgetGrafana, syncGrafana } from "../services/grafanaSync.js";
import { forgetNextcloud, syncNextcloud } from "../services/nextcloudSync.js"; import { forgetNextcloud, syncNextcloud } from "../services/nextcloudSync.js";
@@ -156,7 +158,7 @@ integrationsRouter.post("/", requireRole("admin"), asyncHandler(async (req, res)
if (type === "hetzner") void syncHetzner(created.id).then(() => invalidateAlertsCache()); if (type === "hetzner") void syncHetzner(created.id).then(() => invalidateAlertsCache());
if (type === "speedtest") void syncSpeedtest(created.id).then(() => invalidateAlertsCache()); if (type === "speedtest") void syncSpeedtest(created.id).then(() => invalidateAlertsCache());
if (type === "fragnet") void syncFragnet(created.id).then(() => invalidateAlertsCache()); if (type === "fragnet") void syncFragnet(created.id).then(() => invalidateAlertsCache());
if (type === "sonarr" || type === "radarr" || type === "prowlarr") void syncArr(created.id).then(() => invalidateAlertsCache()); if ((ARR_KINDS as string[]).includes(type)) void syncArr(created.id).then(() => invalidateAlertsCache());
if (type === "adguard") void syncAdguard(created.id).then(() => invalidateAlertsCache()); if (type === "adguard") void syncAdguard(created.id).then(() => invalidateAlertsCache());
if (type === "grafana") void syncGrafana(created.id).then(() => invalidateAlertsCache()); if (type === "grafana") void syncGrafana(created.id).then(() => invalidateAlertsCache());
if (type === "nextcloud") void syncNextcloud(created.id).then(() => invalidateAlertsCache()); if (type === "nextcloud") void syncNextcloud(created.id).then(() => invalidateAlertsCache());
@@ -274,7 +276,7 @@ integrationsRouter.patch("/:id", requireRole("admin"), asyncHandler(async (req,
if (updated.type === "hetzner" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncHetzner(updated.id).then(() => invalidateAlertsCache()); if (updated.type === "hetzner" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncHetzner(updated.id).then(() => invalidateAlertsCache());
if (updated.type === "speedtest" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncSpeedtest(updated.id).then(() => invalidateAlertsCache()); if (updated.type === "speedtest" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncSpeedtest(updated.id).then(() => invalidateAlertsCache());
if (updated.type === "fragnet" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncFragnet(updated.id).then(() => invalidateAlertsCache()); if (updated.type === "fragnet" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncFragnet(updated.id).then(() => invalidateAlertsCache());
if ((updated.type === "sonarr" || updated.type === "radarr" || updated.type === "prowlarr") && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncArr(updated.id).then(() => invalidateAlertsCache()); if ((ARR_KINDS as string[]).includes(updated.type) && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncArr(updated.id).then(() => invalidateAlertsCache());
if (updated.type === "adguard" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncAdguard(updated.id).then(() => invalidateAlertsCache()); if (updated.type === "adguard" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncAdguard(updated.id).then(() => invalidateAlertsCache());
if (updated.type === "grafana" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncGrafana(updated.id).then(() => invalidateAlertsCache()); if (updated.type === "grafana" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncGrafana(updated.id).then(() => invalidateAlertsCache());
if (updated.type === "nextcloud" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncNextcloud(updated.id).then(() => invalidateAlertsCache()); if (updated.type === "nextcloud" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncNextcloud(updated.id).then(() => invalidateAlertsCache());
@@ -340,6 +342,37 @@ integrationsRouter.post("/:id/test", requireRole("admin"), asyncHandler(async (r
res.json(result); res.json(result);
})); }));
/** How long an inspection may take: a read makes a handful of requests, each with a timeout of its own. */
const INSPECT_LIMIT_MS = 90_000;
// Runs an integration's reader once, here and now, and says what it asked, what came back (the shape of each answer — names and kinds, never values) and where what the reader made of it is empty.
// Nothing is stored. It is how a reader written from documentation is checked against the real thing: an answer with a field the reader did not expect shows up as an empty field in the result.
integrationsRouter.post("/:id/inspect", requireRole("admin"), asyncHandler(async (req, res) => {
const id = Number(req.params.id);
if (!Number.isInteger(id)) return res.status(400).json({ error: "invalid_id" });
const loaded = await loadIntegrationConfig(id);
if (!loaded) return res.status(404).json({ error: "not_found" });
if (!loaded.integration.enabled) return res.status(400).json({ error: "integration_disabled", message: "It is switched off." });
const adapter = createIntegrationAdapter(loaded.integration.type, loaded.config) as unknown as { read?: () => Promise<unknown> };
if (typeof adapter.read !== "function") return res.status(400).json({ error: "not_inspectable", message: "This kind of integration has no reader to inspect." });
const read = adapter.read.bind(adapter);
const started = Date.now();
const timeout = new Promise<never>((_, reject) => setTimeout(() => reject(new Error(`took more than ${INSPECT_LIMIT_MS / 1000} s`)), INSPECT_LIMIT_MS).unref());
const { calls, value, error } = await inspectCalls(() => Promise.race([read(), timeout]));
await recordAudit({ actor: req.currentUser!, category: "integration", action: "inspect", targetType: "integration", targetId: id, detail: { type: loaded.integration.type, ok: error === null } });
res.json({
type: loaded.integration.type,
name: loaded.integration.name,
ok: error === null,
error,
ms: Date.now() - started,
calls,
/** What the reader made of the answers, as a shape. */
resultShape: value === undefined ? null : shapeOf(value),
empty: value === undefined ? [] : emptyFields(value),
});
}));
integrationsRouter.delete("/:id", requireRole("admin"), asyncHandler(async (req, res) => { integrationsRouter.delete("/:id", requireRole("admin"), asyncHandler(async (req, res) => {
const id = Number(req.params.id); const id = Number(req.params.id);
const [existing] = await db.select().from(integrations).where(eq(integrations.id, id)).limit(1); const [existing] = await db.select().from(integrations).where(eq(integrations.id, id)).limit(1);
@@ -368,7 +401,7 @@ integrationsRouter.delete("/:id", requireRole("admin"), asyncHandler(async (req,
if (existing.type === "hetzner") await forgetHetzner(id); if (existing.type === "hetzner") await forgetHetzner(id);
if (existing.type === "speedtest") await forgetSpeedtest(id); if (existing.type === "speedtest") await forgetSpeedtest(id);
if (existing.type === "fragnet") await forgetFragnet(id); if (existing.type === "fragnet") await forgetFragnet(id);
if (existing.type === "sonarr" || existing.type === "radarr" || existing.type === "prowlarr") await forgetArr(id); if ((ARR_KINDS as string[]).includes(existing.type)) await forgetArr(id);
if (existing.type === "adguard") await forgetAdguard(id); if (existing.type === "adguard") await forgetAdguard(id);
if (existing.type === "grafana") await forgetGrafana(id); if (existing.type === "grafana") await forgetGrafana(id);
if (existing.type === "nextcloud") await forgetNextcloud(id); if (existing.type === "nextcloud") await forgetNextcloud(id);
+45
View File
@@ -0,0 +1,45 @@
import { Router } from "express";
import { z } from "zod";
import { requireAuth, requireRole } from "../auth/middleware.js";
import { recordAudit } from "../services/audit.js";
import { invalidateAlertsCache } from "../services/alerts.js";
import { refreshReleases, versionRows } from "../services/appVersions.js";
import { getSettings, updateSettings } from "../services/settingsStore.js";
import { asyncHandler } from "../utils/asyncHandler.js";
export const versionsRouter = Router();
versionsRouter.use(requireAuth);
async function payload() {
const { versionChecks } = await getSettings();
const rows = await versionRows();
const checked = rows.map((r) => r.checkedAt).filter((c): c is string => c !== null).sort();
return { enabled: versionChecks.enabled, alertAfterDays: versionChecks.alertAfterDays, lastCheckedAt: checked.length > 0 ? checked[checked.length - 1] : null, rows };
}
/** What each connected application runs against the newest release kept for it. Everyone signed in can see it; nothing is asked of GitHub by looking. */
versionsRouter.get("/", asyncHandler(async (_req, res) => {
res.json(await payload());
}));
const settingsSchema = z.object({ enabled: z.boolean(), alertAfterDays: z.number().int().min(0).max(365) });
/** Switches the lookup of newest releases on or off, and says after how many days a newer release is a warning (0: never). Switching it on looks them up at once. */
versionsRouter.put("/settings", requireRole("admin"), asyncHandler(async (req, res) => {
const parsed = settingsSchema.safeParse(req.body);
if (!parsed.success) return res.status(400).json({ error: "invalid_body", message: "Say whether to look up the newest releases, and after how many days a newer one is a warning.", details: parsed.error.flatten() });
const before = (await getSettings()).versionChecks;
await updateSettings({ versionChecks: parsed.data });
await recordAudit({ actor: req.currentUser!, category: "settings", action: "update", targetType: "versionChecks", detail: { enabled: parsed.data.enabled, alertAfterDays: parsed.data.alertAfterDays } });
if (parsed.data.enabled && !before.enabled) await refreshReleases(new Date(), true).catch(() => undefined);
invalidateAlertsCache();
res.json(await payload());
}));
/** Looks up the newest releases now, instead of at the next daily pass. Only while the lookup is switched on. */
versionsRouter.post("/refresh", requireRole("admin"), asyncHandler(async (_req, res) => {
if (!(await getSettings()).versionChecks.enabled) return res.status(400).json({ error: "switched_off", message: "Looking up the newest releases is switched off." });
const result = await refreshReleases(new Date(), true);
invalidateAlertsCache();
res.json({ ...(await payload()), result });
}));
+8 -3
View File
@@ -88,8 +88,11 @@ const SOURCES: Record<string, { label: string; link: string }> = {
fragnet: { label: "Fragnet", link: "/game-servers" }, fragnet: { label: "Fragnet", link: "/game-servers" },
sonarr: { label: "Sonarr", link: "/media" }, sonarr: { label: "Sonarr", link: "/media" },
radarr: { label: "Radarr", link: "/media" }, radarr: { label: "Radarr", link: "/media" },
lidarr: { label: "Lidarr", link: "/media" },
readarr: { label: "Readarr", link: "/media" },
prowlarr: { label: "Prowlarr", link: "/media" }, prowlarr: { label: "Prowlarr", link: "/media" },
adguard: { label: "AdGuard Home", link: "/adguard" }, adguard: { label: "AdGuard Home", link: "/adguard" },
versions: { label: "Versions", link: "/versions" },
grafana: { label: "Grafana", link: "/grafana" }, grafana: { label: "Grafana", link: "/grafana" },
secrets: { label: "Secrets", link: "/secrets" }, secrets: { label: "Secrets", link: "/secrets" },
domains: { label: "Domains", link: "/dns?view=domains" }, domains: { label: "Domains", link: "/dns?view=domains" },
@@ -133,11 +136,13 @@ function originOfConditionKey(key: string): { type: string; id: number | null; c
if (m) return { type: "speedtest", id: Number(m[1]), category: "monitoring" }; if (m) return { type: "speedtest", id: Number(m[1]), category: "monitoring" };
m = /^hetzner-(off|stuck|rescue|traffic|forecast|backup|lb|cert|fip):(\d+):/.exec(key); m = /^hetzner-(off|stuck|rescue|traffic|forecast|backup|lb|cert|fip):(\d+):/.exec(key);
if (m) return { type: "hetzner", id: Number(m[2]), category: m[1] === "traffic" || m[1] === "forecast" ? "monitoring" : m[1] === "backup" ? "backup" : "services" }; if (m) return { type: "hetzner", id: Number(m[2]), category: m[1] === "traffic" || m[1] === "forecast" ? "monitoring" : m[1] === "backup" ? "backup" : "services" };
m = /^version-behind:(\d+)$/.exec(key);
if (m) return { type: "versions", id: Number(m[1]), category: "updates" };
m = /^adguard-(stopped|dns|protection|filters):(\d+)/.exec(key); m = /^adguard-(stopped|dns|protection|filters):(\d+)/.exec(key);
if (m) return { type: "adguard", id: Number(m[2]), category: m[1] === "stopped" || m[1] === "dns" ? "services" : "monitoring" }; if (m) return { type: "adguard", id: Number(m[2]), category: m[1] === "stopped" || m[1] === "dns" ? "services" : "monitoring" };
m = /^grafana-(firing|rule-error|ds):(\d+):/.exec(key); m = /^grafana-(firing|rule-error|ds):(\d+):/.exec(key);
if (m) return { type: "grafana", id: Number(m[2]), category: m[1] === "ds" ? "integration" : "monitoring" }; if (m) return { type: "grafana", id: Number(m[2]), category: m[1] === "ds" ? "integration" : "monitoring" };
m = /^(sonarr|radarr|prowlarr)-(health|queue|disk):(\d+)/.exec(key); m = /^(sonarr|radarr|prowlarr|lidarr|readarr)-(health|queue|disk):(\d+)/.exec(key);
if (m) return { type: m[1], id: Number(m[3]), category: m[2] === "disk" ? "disk" : "services" }; if (m) return { type: m[1], id: Number(m[3]), category: m[2] === "disk" ? "disk" : "services" };
m = /^fragnet-(disk|memory|backup):(\d+):/.exec(key); m = /^fragnet-(disk|memory|backup):(\d+):/.exec(key);
if (m) return { type: "fragnet", id: Number(m[2]), category: m[1] === "disk" ? "disk" : m[1] === "backup" ? "backup" : "monitoring" }; if (m) return { type: "fragnet", id: Number(m[2]), category: m[1] === "disk" ? "disk" : m[1] === "backup" ? "backup" : "monitoring" };
@@ -283,7 +288,7 @@ export async function collectAlerts(): Promise<AlertsReport> {
type: origin.type, type: origin.type,
message: c.message, message: c.message,
key: c.key, key: c.key,
link: origin.type === "server" && origin.id !== null ? `/servers/${origin.id}${/^(failed-units|smart|zfs):/.test(c.key) ? "?view=health" : c.key.startsWith("new-port:") ? "?view=network" : c.key.startsWith("rule:") || c.key.startsWith("baseline:") ? "?view=history" : ""}` : origin.type === "unifi" ? "/network/devices" : origin.type === "ups" ? "/ups" : origin.type === "slo" ? "/service-levels" : origin.type === "dockhand" ? "/docker" : origin.type === "duplicati" ? "/duplicati" : origin.type === "jellyfin" ? "/jellyfin" : origin.type === "authentik" ? "/authentik" : origin.type === "prometheus" ? "/prometheus" : origin.type === "immich" ? "/immich" : origin.type === "paperless" ? "/paperless" : origin.type === "nextcloud" ? "/nextcloud" : origin.type === "netbox" ? "/netbox" : origin.type === "n8n" ? "/n8n" : origin.type === "hetzner" ? "/hetzner" : origin.type === "speedtest" ? "/speedtest" : origin.type === "fragnet" ? "/game-servers" : origin.type === "sonarr" || origin.type === "radarr" || origin.type === "prowlarr" ? "/media" : origin.type === "adguard" ? "/adguard" : origin.type === "grafana" ? "/grafana" : undefined, link: origin.type === "server" && origin.id !== null ? `/servers/${origin.id}${/^(failed-units|smart|zfs):/.test(c.key) ? "?view=health" : c.key.startsWith("new-port:") ? "?view=network" : c.key.startsWith("rule:") || c.key.startsWith("baseline:") ? "?view=history" : ""}` : origin.type === "unifi" ? "/network/devices" : origin.type === "ups" ? "/ups" : origin.type === "slo" ? "/service-levels" : origin.type === "dockhand" ? "/docker" : origin.type === "duplicati" ? "/duplicati" : origin.type === "jellyfin" ? "/jellyfin" : origin.type === "authentik" ? "/authentik" : origin.type === "prometheus" ? "/prometheus" : origin.type === "immich" ? "/immich" : origin.type === "paperless" ? "/paperless" : origin.type === "nextcloud" ? "/nextcloud" : origin.type === "netbox" ? "/netbox" : origin.type === "n8n" ? "/n8n" : origin.type === "hetzner" ? "/hetzner" : origin.type === "speedtest" ? "/speedtest" : origin.type === "fragnet" ? "/game-servers" : origin.type === "sonarr" || origin.type === "radarr" || origin.type === "prowlarr" || origin.type === "lidarr" || origin.type === "readarr" ? "/media" : origin.type === "adguard" ? "/adguard" : origin.type === "grafana" ? "/grafana" : undefined,
silenced: subject !== null && subjects.has(subject), target: targetOfSubject(subject), silenced: subject !== null && subjects.has(subject), target: targetOfSubject(subject),
}); });
} }
@@ -293,7 +298,7 @@ export async function collectAlerts(): Promise<AlertsReport> {
if (held.has("slo")) cannotCheck("Service-level targets", "couldn't be judged — see the server log", "Server and storage health"); if (held.has("slo")) cannotCheck("Service-level targets", "couldn't be judged — see the server log", "Server and storage health");
if (held.has("server:baseline")) cannotCheck("What is usual for the time", "couldn't be judged — see the server log", "Server and storage health"); if (held.has("server:baseline")) cannotCheck("What is usual for the time", "couldn't be judged — see the server log", "Server and storage health");
for (const h of held) { for (const h of held) {
const m = /^(proxmox|synology|unifi|ups|dockhand|jellyfin|authentik|prometheus|immich|paperless|nextcloud|netbox|n8n|hetzner|speedtest|fragnet|sonarr|radarr|prowlarr|adguard|grafana):(\d+)(?::(.+))?$/.exec(h); const m = /^(proxmox|synology|unifi|ups|dockhand|jellyfin|authentik|prometheus|immich|paperless|nextcloud|netbox|n8n|hetzner|speedtest|fragnet|sonarr|radarr|prowlarr|lidarr|readarr|adguard|grafana):(\d+)(?::(.+))?$/.exec(h);
if (!m || (m[3] && held.has(`${m[1]}:${m[2]}`))) continue; if (!m || (m[3] && held.has(`${m[1]}:${m[2]}`))) continue;
const row = intById.get(Number(m[2])); const row = intById.get(Number(m[2]));
cannotCheck(`${SOURCES[m[1]].label} “${row?.name ?? `#${m[2]}`}”${m[3] === "vuln" ? " (vulnerability scans)" : m[3] === "images" ? " (age of the images)" : m[3] ? ` (node ${m[3]})` : ""}`, "couldn't be read — see the Diagnostic Log", "Server and storage health"); cannotCheck(`${SOURCES[m[1]].label} “${row?.name ?? `#${m[2]}`}”${m[3] === "vuln" ? " (vulnerability scans)" : m[3] === "images" ? " (age of the images)" : m[3] ? ` (node ${m[3]})` : ""}`, "couldn't be read — see the Diagnostic Log", "Server and storage health");
+207
View File
@@ -0,0 +1,207 @@
/**
* Which version each connected application runs — as its integration last read it — against the newest release of that application, so that what has fallen behind can be seen in one
* list. The newest release is asked of GitHub (`GET /repos/{owner}/{repo}/releases/latest`, which leaves out pre-releases and drafts) once a day for each application there is a connected
* integration for, and kept. **Nothing is asked unless the setting is on** (Operations → Versions): that is the only thing in this app that calls out to an address of its own choosing.
*
* Versions are compared by their first three numbers (a build number after them is ignored), after the usual prefixes — `v`, `n8n@`, `version/` — are taken off. A difference in the first number is a
* *major* update, in the second a *minor* one, in the third a *patch*; what is newer on this machine than the newest release is *ahead* (a beta, a nightly). With *alert after* days set, an
* application that has had a newer release out for that long raises a warning (and one that is merely a day behind does not).
*/
import { eq, inArray } from "drizzle-orm";
import { db } from "../db/client.js";
import { adguardState, arrState, authentikState, grafanaState, immichState, integrations, jellyfinState, netboxState, nextcloudState, paperlessState, prometheusState, releaseCache } from "../db/schema.js";
import { requestJson } from "../integrations/httpJson.js";
import { getSettings } from "./settingsStore.js";
export const CHECK_INTERVAL_MS = 12 * 60 * 60_000;
/** A release looked up more recently than this is not looked up again (unless asked to). */
export const FRESH_MS = 6 * 60 * 60_000;
const DAY = 86_400_000;
/** Where the newest release of each kind of application is published. A kind with no entry (Proxmox, Synology, Hetzner …) has no version to compare. */
export const RELEASE_SOURCES: Record<string, { repo: string; label: string }> = {
sonarr: { repo: "Sonarr/Sonarr", label: "Sonarr" },
radarr: { repo: "Radarr/Radarr", label: "Radarr" },
prowlarr: { repo: "Prowlarr/Prowlarr", label: "Prowlarr" },
lidarr: { repo: "Lidarr/Lidarr", label: "Lidarr" },
readarr: { repo: "Readarr/Readarr", label: "Readarr" },
adguard: { repo: "AdguardTeam/AdGuardHome", label: "AdGuard Home" },
grafana: { repo: "grafana/grafana", label: "Grafana" },
jellyfin: { repo: "jellyfin/jellyfin", label: "Jellyfin" },
immich: { repo: "immich-app/immich", label: "Immich" },
paperless: { repo: "paperless-ngx/paperless-ngx", label: "Paperless-ngx" },
nextcloud: { repo: "nextcloud/server", label: "Nextcloud" },
netbox: { repo: "netbox-community/netbox", label: "NetBox" },
authentik: { repo: "goauthentik/authentik", label: "Authentik" },
prometheus: { repo: "prometheus/prometheus", label: "Prometheus" },
};
export type Lag = "same" | "patch" | "minor" | "major" | "ahead" | "unknown";
/** Pure: the first three numbers of a version, after a prefix such as "v", "n8n@" or "version/"; null when there is no number. */
export function parseVersion(text: string | null | undefined): [number, number, number] | null {
if (!text) return null;
const last = String(text).split(/[@/]/).pop() ?? "";
const m = /^[^0-9]*(\d+)(?:\.(\d+))?(?:\.(\d+))?/.exec(last);
if (!m) return null;
return [Number(m[1]), Number(m[2] ?? 0), Number(m[3] ?? 0)];
}
/** Pure: how far the running version is behind the newest release. */
export function lagOf(running: string | null | undefined, latest: string | null | undefined): Lag {
const a = parseVersion(running);
const b = parseVersion(latest);
if (!a || !b) return "unknown";
for (let i = 0; i < 3; i++) {
if (a[i] === b[i]) continue;
if (a[i] > b[i]) return "ahead";
return i === 0 ? "major" : i === 1 ? "minor" : "patch";
}
return "same";
}
export interface LatestRelease {
tag: string;
publishedAt: string | null;
url: string | null;
}
type Fetcher = (repo: string) => Promise<LatestRelease>;
/** Asks GitHub for the newest release of a repository. Throws, with the reason, when it cannot say. */
async function githubLatest(repo: string): Promise<LatestRelease> {
const res = await requestJson(`https://api.github.com/repos/${repo}/releases/latest`, { headers: { Accept: "application/vnd.github+json", "User-Agent": "Homelab-Manager" } });
if (res.status === 403 || res.status === 429) throw new Error("GitHub's limit on requests without a sign-in was reached — it is tried again later");
if (res.status === 404) throw new Error("GitHub lists no releases for it");
if (res.status < 200 || res.status >= 300 || res.json === null || typeof res.json !== "object") throw new Error(`GitHub answered with HTTP ${res.status}`);
const o = res.json as Record<string, unknown>;
const tag = typeof o.tag_name === "string" && o.tag_name.trim() !== "" ? o.tag_name.trim() : null;
if (!tag) throw new Error("GitHub's answer has no release tag");
const at = typeof o.published_at === "string" && Number.isFinite(Date.parse(o.published_at)) ? new Date(o.published_at).toISOString() : null;
return { tag, publishedAt: at, url: typeof o.html_url === "string" ? o.html_url : null };
}
let fetcher: Fetcher = githubLatest;
/** For tests: what answers "what is the newest release of this" instead of GitHub. */
export function setReleaseFetcherForTests(f: Fetcher | null): void {
fetcher = f ?? githubLatest;
}
/** The versions the integrations last read, by integration id. */
async function readVersions(): Promise<Map<number, string>> {
const out = new Map<number, string>();
const tables = [arrState, adguardState, grafanaState, nextcloudState, netboxState, paperlessState, immichState, prometheusState, authentikState, jellyfinState];
for (const t of tables) {
for (const r of await db.select({ id: t.integrationId, version: t.version }).from(t)) if (r.version) out.set(r.id, r.version);
}
return out;
}
export interface VersionRow {
integrationId: number;
type: string;
label: string;
name: string;
running: string;
latest: { tag: string; publishedAt: string | null; url: string | null } | null;
/** The reason the newest release could not be looked up, when it could not. */
lookupError: string | null;
lag: Lag;
/** How many days the newer release has been out; null when it is not behind or the date is not known. */
daysOut: number | null;
checkedAt: string | null;
}
/** What each connected application runs against the newest release kept for it. Reads only what is stored: nothing is asked of GitHub here. */
export async function versionRows(nowMs: number = Date.now()): Promise<VersionRow[]> {
const rows = await db.select({ id: integrations.id, type: integrations.type, name: integrations.name }).from(integrations).where(eq(integrations.enabled, true));
const mine = rows.filter((r) => RELEASE_SOURCES[r.type]);
if (mine.length === 0) return [];
const [versions, cached] = await Promise.all([readVersions(), db.select().from(releaseCache).where(inArray(releaseCache.repo, [...new Set(mine.map((r) => RELEASE_SOURCES[r.type].repo))]))]);
const byRepo = new Map(cached.map((c) => [c.repo, c]));
const out: VersionRow[] = [];
for (const r of mine) {
const running = versions.get(r.id);
if (!running) continue;
const src = RELEASE_SOURCES[r.type];
const c = byRepo.get(src.repo);
const latest = c?.tag ? { tag: c.tag, publishedAt: c.publishedAt, url: c.url } : null;
const lag = lagOf(running, latest?.tag);
out.push({
integrationId: r.id,
type: r.type,
label: src.label,
name: r.name,
running,
latest,
lookupError: c?.error ?? null,
lag,
daysOut: latest?.publishedAt && (lag === "patch" || lag === "minor" || lag === "major") ? Math.max(0, Math.floor((nowMs - Date.parse(latest.publishedAt)) / DAY)) : null,
checkedAt: c?.checkedAt ?? null,
});
}
return out.sort((a, b) => a.label.localeCompare(b.label) || a.name.localeCompare(b.name));
}
/** Looks up the newest release of every application there is a connected integration for, and keeps it. A repository looked up lately is left alone unless `force`; one that cannot be looked up keeps what it had, with the reason. */
export async function refreshReleases(now: Date = new Date(), force = false): Promise<{ checked: number; failed: number }> {
// Only for an application whose version has been read: the newest release of one that has not is of no use yet.
const [rows, versions] = await Promise.all([db.select({ id: integrations.id, type: integrations.type }).from(integrations).where(eq(integrations.enabled, true)), readVersions()]);
const repos = [...new Set(rows.filter((r) => RELEASE_SOURCES[r.type] && versions.has(r.id)).map((r) => RELEASE_SOURCES[r.type].repo))];
const cached = new Map((await db.select().from(releaseCache)).map((c) => [c.repo, c]));
let checked = 0;
let failed = 0;
for (const repo of repos) {
const known = cached.get(repo);
if (!force && known && now.getTime() - Date.parse(known.checkedAt) < FRESH_MS && !known.error) continue;
try {
const l = await fetcher(repo);
const values = { tag: l.tag, publishedAt: l.publishedAt, url: l.url, checkedAt: now.toISOString(), error: null };
await db.insert(releaseCache).values({ repo, ...values }).onConflictDoUpdate({ target: releaseCache.repo, set: values });
checked++;
} catch (err) {
failed++;
const values = { checkedAt: now.toISOString(), error: err instanceof Error ? err.message : String(err) };
await db.insert(releaseCache).values({ repo, tag: known?.tag ?? "", publishedAt: known?.publishedAt ?? null, url: known?.url ?? null, ...values }).onConflictDoUpdate({ target: releaseCache.repo, set: values });
console.error(`[versions] couldn't look up ${repo}:`, values.error);
}
}
return { checked, failed };
}
export interface VersionProblem {
key: string;
message: string;
severity: "warning";
}
/** Pure: an application that has had a newer release out for `alertAfterDays` or more is a warning; 0 turns it off. */
export function versionProblems(rows: VersionRow[], alertAfterDays: number): VersionProblem[] {
if (alertAfterDays <= 0) return [];
const out: VersionProblem[] = [];
for (const r of rows) {
if (!r.latest || r.daysOut === null || r.daysOut < alertAfterDays) continue;
out.push({
key: `version-behind:${r.integrationId}`,
severity: "warning",
message: `${r.name}: ${r.label} ${r.running} is running; ${r.latest.tag} came out ${r.daysOut} days ago (${r.lag === "patch" ? "a patch" : r.lag === "minor" ? "a minor update" : "a new major version"})`,
});
}
return out;
}
let timer: ReturnType<typeof setInterval> | null = null;
export function initVersionCheck(): void {
if (timer) return;
const pass = async () => {
try {
if ((await getSettings()).versionChecks.enabled) await refreshReleases();
} catch (err) {
console.error("[versions] pass failed:", err);
}
};
timer = setInterval(() => void pass(), CHECK_INTERVAL_MS);
setTimeout(() => void pass(), 3 * 60_000);
console.log(`[versions] looking up the newest releases every ${CHECK_INTERVAL_MS / 3_600_000} hours, when switched on`);
}
+17
View File
@@ -32,6 +32,7 @@ import { dockerImageAgeProblems, dockerVulnProblems, dockerVulnSyncStatuses, isD
import { expectedOf as speedtestExpected, isSpeedtestFresh, recentResults as recentSpeedtestResults, speedtestProblems, speedtestSyncStatuses, type SpeedtestProblem } from "./speedtestSync.js"; import { expectedOf as speedtestExpected, isSpeedtestFresh, recentResults as recentSpeedtestResults, speedtestProblems, speedtestSyncStatuses, type SpeedtestProblem } from "./speedtestSync.js";
import { hetznerProblems, hetznerSyncStatuses, imagesOf as hetznerImagesOf, loadBalancersOf as hetznerLoadBalancersOf, floatingIpsOf as hetznerFloatingIpsOf, certificatesOf as hetznerCertificatesOf, isHetznerFresh, serversOf as hetznerServersOf, type HetznerProblem } from "./hetznerSync.js"; import { hetznerProblems, hetznerSyncStatuses, imagesOf as hetznerImagesOf, loadBalancersOf as hetznerLoadBalancersOf, floatingIpsOf as hetznerFloatingIpsOf, certificatesOf as hetznerCertificatesOf, isHetznerFresh, serversOf as hetznerServersOf, type HetznerProblem } from "./hetznerSync.js";
import { ARR_KINDS, type ArrKind } from "../integrations/arr/adapter.js"; import { ARR_KINDS, type ArrKind } from "../integrations/arr/adapter.js";
import { versionProblems, versionRows, type VersionProblem } from "./appVersions.js";
import { adguardProblems, adguardSyncStatuses, isAdguardFresh, storedOf as adguardStoredOf, type AdguardProblem } from "./adguardSync.js"; import { adguardProblems, adguardSyncStatuses, isAdguardFresh, storedOf as adguardStoredOf, type AdguardProblem } from "./adguardSync.js";
import { grafanaProblems, grafanaSyncStatuses, isGrafanaFresh, storedOf as grafanaStoredOf, type GrafanaProblem } from "./grafanaSync.js"; import { grafanaProblems, grafanaSyncStatuses, isGrafanaFresh, storedOf as grafanaStoredOf, type GrafanaProblem } from "./grafanaSync.js";
import { arrProblems, arrSyncStatuses, isArrFresh, storedOf as arrStoredOf, type ArrProblem } from "./arrSync.js"; import { arrProblems, arrSyncStatuses, isArrFresh, storedOf as arrStoredOf, type ArrProblem } from "./arrSync.js";
@@ -138,6 +139,8 @@ export interface HealthSnapshot {
arr?: { integrationId: number; kind: ArrKind; problems: ArrProblem[] }[]; arr?: { integrationId: number; kind: ArrKind; problems: ArrProblem[] }[];
/** What is wrong with each AdGuard Home (DNS not running or not answering, protection left off, filter lists not updating), only while that reading is recent (services/adguardSync.ts). */ /** What is wrong with each AdGuard Home (DNS not running or not answering, protection left off, filter lists not updating), only while that reading is recent (services/adguardSync.ts). */
adguard?: { integrationId: number; problems: AdguardProblem[] }[]; adguard?: { integrationId: number; problems: AdguardProblem[] }[];
/** Applications a newer release of which has been out for the number of days set (services/appVersions.ts), only while the lookup of releases is on. */
versions?: VersionProblem[];
/** What is wrong with each Grafana (rules firing or failing, data sources failing their check), only while that reading is recent (services/grafanaSync.ts). */ /** What is wrong with each Grafana (rules firing or failing, data sources failing their check), only while that reading is recent (services/grafanaSync.ts). */
grafana?: { integrationId: number; problems: GrafanaProblem[] }[]; grafana?: { integrationId: number; problems: GrafanaProblem[] }[];
/** Containers to leave alone. */ /** Containers to leave alone. */
@@ -281,6 +284,9 @@ export function evaluateHealth(snapshot: HealthSnapshot, thresholds: Thresholds,
// Speedtest Tracker: the internet slower than it is sold at, speed tests that keep failing. // Speedtest Tracker: the internet slower than it is sold at, speed tests that keep failing.
for (const s of snapshot.speedtest ?? []) for (const p of s.problems) out.push({ key: p.key, source: `speedtest:${s.integrationId}`, message: p.message, severity: p.severity }); for (const s of snapshot.speedtest ?? []) for (const p of s.problems) out.push({ key: p.key, source: `speedtest:${s.integrationId}`, message: p.message, severity: p.severity });
// Applications that have fallen behind their newest release.
for (const p of snapshot.versions ?? []) out.push({ key: p.key, source: "versions", message: p.message, severity: p.severity });
// AdGuard Home and Grafana. // AdGuard Home and Grafana.
for (const a of snapshot.adguard ?? []) for (const p of a.problems) out.push({ key: p.key, source: `adguard:${a.integrationId}`, message: p.message, severity: p.severity }); for (const a of snapshot.adguard ?? []) for (const p of a.problems) out.push({ key: p.key, source: `adguard:${a.integrationId}`, message: p.message, severity: p.severity });
for (const g of snapshot.grafana ?? []) for (const p of g.problems) out.push({ key: p.key, source: `grafana:${g.integrationId}`, message: p.message, severity: p.severity }); for (const g of snapshot.grafana ?? []) for (const p of g.problems) out.push({ key: p.key, source: `grafana:${g.integrationId}`, message: p.message, severity: p.severity });
@@ -557,6 +563,17 @@ export async function collectSnapshot(): Promise<{ snapshot: HealthSnapshot; hel
console.error("[health] couldn't judge Speedtest Tracker:", err instanceof Error ? err.message : err); console.error("[health] couldn't judge Speedtest Tracker:", err instanceof Error ? err.message : err);
} }
// The newest releases, as the daily lookup stored them (nothing is asked of GitHub here), only with the lookup and the alert switched on.
try {
const { versionChecks } = await getSettings();
if (versionChecks.enabled && versionChecks.alertAfterDays > 0) {
const problems = versionProblems(await versionRows(), versionChecks.alertAfterDays);
if (problems.length > 0) snapshot.versions = problems;
}
} catch (err) {
console.error("[health] couldn't judge the versions:", err instanceof Error ? err.message : err);
}
// AdGuard Home: read by its own pass every two minutes; what is judged is what that pass stored, and only while it is recent. // AdGuard Home: read by its own pass every two minutes; what is judged is what that pass stored, and only while it is recent.
try { try {
const rowsG = await db.select({ id: integrations.id, name: integrations.name }).from(integrations).where(and(eq(integrations.type, "adguard"), eq(integrations.enabled, true))); const rowsG = await db.select({ id: integrations.id, name: integrations.name }).from(integrations).where(and(eq(integrations.type, "adguard"), eq(integrations.enabled, true)));
+1 -1
View File
@@ -4,7 +4,7 @@
*/ */
export const INTEGRATION_GROUPS: { id: string; label: string; types: string[] }[] = [ export const INTEGRATION_GROUPS: { id: string; label: string; types: string[] }[] = [
{ id: "infrastructure", label: "Infrastructure", types: ["proxmox", "synology", "dockhand", "hetzner", "fragnet", "tailscale"] }, { id: "infrastructure", label: "Infrastructure", types: ["proxmox", "synology", "dockhand", "hetzner", "fragnet", "tailscale"] },
{ id: "apps", label: "Apps", types: ["homeassistant", "jellyfin", "sonarr", "radarr", "prowlarr", "immich", "paperless", "nextcloud", "authentik"] }, { id: "apps", label: "Apps", types: ["homeassistant", "jellyfin", "sonarr", "radarr", "lidarr", "readarr", "prowlarr", "immich", "paperless", "nextcloud", "authentik"] },
{ id: "backups", label: "Backups", types: ["pbs"] }, { id: "backups", label: "Backups", types: ["pbs"] },
{ id: "network", label: "Network", types: ["unifi", "npm", "phpipam", "netbox", "adguard"] }, { id: "network", label: "Network", types: ["unifi", "npm", "phpipam", "netbox", "adguard"] },
{ id: "monitoring", label: "Monitoring", types: ["uptimekuma", "prometheus", "grafana", "ups", "speedtest"] }, { id: "monitoring", label: "Monitoring", types: ["uptimekuma", "prometheus", "grafana", "ups", "speedtest"] },
+3 -1
View File
@@ -43,7 +43,9 @@ export function subjectOfConditionKey(key: string): string | null {
if (m) return `integration:${m[1]}`; if (m) return `integration:${m[1]}`;
m = /^fragnet-(?:disk|memory|backup):(\d+):/.exec(key); m = /^fragnet-(?:disk|memory|backup):(\d+):/.exec(key);
if (m) return `integration:${m[1]}`; if (m) return `integration:${m[1]}`;
m = /^(?:sonarr|radarr|prowlarr)-(?:health|queue|disk):(\d+)/.exec(key); m = /^(?:sonarr|radarr|prowlarr|lidarr|readarr)-(?:health|queue|disk):(\d+)/.exec(key);
if (m) return `integration:${m[1]}`;
m = /^version-behind:(\d+)$/.exec(key);
if (m) return `integration:${m[1]}`; if (m) return `integration:${m[1]}`;
m = /^adguard-(?:stopped|dns|protection|filters):(\d+)/.exec(key); m = /^adguard-(?:stopped|dns|protection|filters):(\d+)/.exec(key);
if (m) return `integration:${m[1]}`; if (m) return `integration:${m[1]}`;
+11
View File
@@ -15,6 +15,7 @@ import { dockerVulnSyncStatuses, isDockerVulnFresh } from "./dockerVulnSync.js";
import { isSpeedtestFresh, recentResults as recentSpeedtestResults, speedtestSyncStatuses } from "./speedtestSync.js"; import { isSpeedtestFresh, recentResults as recentSpeedtestResults, speedtestSyncStatuses } from "./speedtestSync.js";
import { costOf as hetznerCostOf, hetznerSyncStatuses, isHetznerFresh, serversOf as hetznerServersOf, loadBalancersOf as hetznerLoadBalancersOf, certificatesOf as hetznerCertificatesOf, trafficForecast as hetznerTrafficForecast } from "./hetznerSync.js"; import { costOf as hetznerCostOf, hetznerSyncStatuses, isHetznerFresh, serversOf as hetznerServersOf, loadBalancersOf as hetznerLoadBalancersOf, certificatesOf as hetznerCertificatesOf, trafficForecast as hetznerTrafficForecast } from "./hetznerSync.js";
import { ARR_KINDS } from "../integrations/arr/adapter.js"; import { ARR_KINDS } from "../integrations/arr/adapter.js";
import { versionRows as appVersionRows } from "./appVersions.js";
import { adguardSyncStatuses, isAdguardFresh, storedOf as adguardStoredOf } from "./adguardSync.js"; import { adguardSyncStatuses, isAdguardFresh, storedOf as adguardStoredOf } from "./adguardSync.js";
import { grafanaSyncStatuses, isGrafanaFresh, storedOf as grafanaStoredOf } from "./grafanaSync.js"; import { grafanaSyncStatuses, isGrafanaFresh, storedOf as grafanaStoredOf } from "./grafanaSync.js";
import { arrSyncStatuses, distinctDisks as arrDisks, isArrFresh, stuckItems as arrStuck, storedOf as arrStoredOf } from "./arrSync.js"; import { arrSyncStatuses, distinctDisks as arrDisks, isArrFresh, stuckItems as arrStuck, storedOf as arrStoredOf } from "./arrSync.js";
@@ -105,6 +106,15 @@ async function addSpeedtestMetrics(out: Exposition, now: Date) {
} }
} }
/** Each connected application whose newest release is known, for /metrics: 1 if a newer release is out than the one it runs, else 0. Only while the lookup of releases is switched on. */
async function addVersionMetrics(out: Exposition) {
if (!(await getSettings()).versionChecks.enabled) return;
for (const r of await appVersionRows()) {
if (r.lag === "unknown" || r.latest === null) continue;
out.add("homelab_app_update_available", "gauge", "1 if a newer release of the application is out than the one it runs, else 0.", r.lag === "same" || r.lag === "ahead" ? 0 : 1, { app: r.type, name: r.name, running: r.running, latest: r.latest.tag });
}
}
/** Each AdGuard Home, for /metrics: whether its DNS server runs and protects, whether the test question was answered and how fast, and what it has done over its statistics period. Only while the reading is recent. */ /** Each AdGuard Home, for /metrics: whether its DNS server runs and protects, whether the test question was answered and how fast, and what it has done over its statistics period. Only while the reading is recent. */
async function addAdguardMetrics(out: Exposition, now: Date) { async function addAdguardMetrics(out: Exposition, now: Date) {
const statuses = await adguardSyncStatuses(); const statuses = await adguardSyncStatuses();
@@ -542,6 +552,7 @@ export async function buildMetrics(now: Date = new Date()): Promise<string> {
await addFragnetMetrics(out, now); await addFragnetMetrics(out, now);
await addArrMetrics(out, now); await addArrMetrics(out, now);
await addAdguardMetrics(out, now); await addAdguardMetrics(out, now);
await addVersionMetrics(out);
await addGrafanaMetrics(out, now); await addGrafanaMetrics(out, now);
await addHeartbeatMetrics(out, now); await addHeartbeatMetrics(out, now);
return out.toString(); return out.toString();
+9
View File
@@ -262,6 +262,13 @@ export interface AlertTicketSettings {
reopenAfterMinutes: number; reopenAfterMinutes: number;
} }
/** Looking up the newest release of each connected application (Operations → Versions). The one thing this app asks of an address that is not one of yours, so it is off until it is switched on. */
export interface VersionCheckSettings {
enabled: boolean;
/** A newer release that has been out this many days is a warning for the application that has not caught up; 0 = never. */
alertAfterDays: number;
}
export interface AppSettings { export interface AppSettings {
gotify: GotifySettings; gotify: GotifySettings;
ntfy: NtfySettings; ntfy: NtfySettings;
@@ -291,9 +298,11 @@ export interface AppSettings {
consistency: ConsistencySettings; consistency: ConsistencySettings;
nameGenerator: NameGeneratorSettings; nameGenerator: NameGeneratorSettings;
alertTickets: AlertTicketSettings; alertTickets: AlertTicketSettings;
versionChecks: VersionCheckSettings;
} }
const DEFAULTS: AppSettings = { const DEFAULTS: AppSettings = {
versionChecks: { enabled: false, alertAfterDays: 0 },
gotify: { enabled: false, url: "", token: "", priority: 5 }, gotify: { enabled: false, url: "", token: "", priority: 5 },
ntfy: { enabled: false, url: "https://ntfy.sh", topic: "", token: "", priority: 3 }, ntfy: { enabled: false, url: "https://ntfy.sh", topic: "", token: "", priority: 3 },
smtp: { enabled: false, host: "", port: 587, secure: false, username: "", password: "", from: "", to: "" }, smtp: { enabled: false, host: "", port: 587, secure: false, username: "", password: "", from: "", to: "" },
+1 -1
View File
@@ -123,7 +123,7 @@ export async function buildWallExtras(
if (known) tiles.push({ id: "vulnerabilities", label: `critical ${plural(critical, "vulnerability", "vulnerabilities")}`, figure: String(critical), state: critical > 0 ? "bad" : "ok" }); if (known) tiles.push({ id: "vulnerabilities", label: `critical ${plural(critical, "vulnerability", "vulnerabilities")}`, figure: String(critical), state: critical > 0 ? "bad" : "ok" });
} }
const media = ids(["sonarr", "radarr"]); const media = ids(["sonarr", "radarr", "lidarr", "readarr"]);
if (media.length > 0) { if (media.length > 0) {
const statuses = await arrSyncStatuses(); const statuses = await arrSyncStatuses();
let stuck = 0; let stuck = 0;
+73
View File
@@ -0,0 +1,73 @@
/**
* Grouping the alerts on the Alerts page (by source, by kind of problem, by type of alert): the pure function the page uses, run here against the web's own source file — the page has no test
* runner of its own — and the words for a type of alert.
*
* Run with: npm test --workspace server
*/
import { test, describe } from "node:test";
import assert from "node:assert/strict";
import { pathToFileURL } from "node:url";
import { join } from "node:path";
const web = (file: string) => pathToFileURL(join(import.meta.dirname, "../../web/src/utils", file)).href;
const { groupAlerts, isGroup, GROUP_BY_LABELS } = (await import(web("alertGroups.ts"))) as typeof import("../../web/src/utils/alertGroups");
const { kindWords } = (await import(web("alertKinds.ts"))) as typeof import("../../web/src/utils/alertKinds");
type A = Parameters<typeof groupAlerts>[0][number];
const alert = (id: string, source: string, severity: A["severity"], extra: Partial<A> = {}): A => ({ id, source, severity, silenced: false, category: "services", kind: "disk", ...extra });
const label = (c: string) => ({ services: "Services", disk: "Disk", backup: "Backups" })[c] ?? c;
const none = new Set<string>();
const shape = (list: ReturnType<typeof groupAlerts>) => list.map((x) => (isGroup(x) ? `# ${x.label} (${x.count}: ${x.critical}c ${x.warning}w ${x.info}i ${x.silenced}s)` : (x as A).id));
describe("the words for a type of alert", () => {
test("a type that has words says them; one that has not is its own name", () => {
assert.equal(kindWords("hetzner-cert"), "a Hetzner certificate about to end");
assert.equal(kindWords("sonarr-queue"), "stuck downloads in Sonarr");
assert.equal(kindWords("lidarr-queue"), "stuck downloads in Lidarr");
assert.equal(kindWords("something-new"), "something new alerts");
});
});
describe("grouping alerts", () => {
const list = [
alert("1", "Hetzner", "warning", { category: "services", kind: "hetzner-cert" }),
alert("2", "Sonarr", "critical", { category: "disk", kind: "sonarr-disk" }),
alert("3", "Hetzner", "warning", { category: "backup", kind: "hetzner-backup" }),
alert("4", "Sonarr", "warning", { category: "services", kind: "sonarr-queue" }),
alert("5", "Hetzner", "info", { category: "services", kind: "hetzner-cert" }),
alert("6", "Fragnet", "warning", { category: "backup", kind: "fragnet-backup", silenced: true }),
];
test("not grouped is the list as it was", () => {
assert.deepEqual(groupAlerts(list, "none", none, label), list);
assert.deepEqual(Object.keys(GROUP_BY_LABELS), ["none", "source", "category", "kind"]);
});
test("by source: the worst group first, then the biggest; each group a header with its counts and then its alerts in the order given; a group of only held-back alerts last", () => {
assert.deepEqual(shape(groupAlerts(list, "source", none, label)), ["# Sonarr (2: 1c 1w 0i 0s)", "2", "4", "# Hetzner (3: 0c 2w 1i 0s)", "1", "3", "5", "# Fragnet (1: 0c 0w 0i 1s)", "6"]);
});
test("by kind of problem it is the category's name; by type of alert the words for the type", () => {
assert.deepEqual(shape(groupAlerts(list, "category", none, label)).filter((x) => x.startsWith("#")), ["# Disk (1: 1c 0w 0i 0s)", "# Services (3: 0c 2w 1i 0s)", "# Backups (2: 0c 1w 0i 1s)"]);
assert.deepEqual(
shape(groupAlerts(list, "kind", none, label)).filter((x) => x.startsWith("#")),
["# a disk on Sonarr (1: 1c 0w 0i 0s)".replace("a disk on Sonarr", kindWords("sonarr-disk")), `# ${kindWords("hetzner-cert")} (2: 0c 1w 1i 0s)`, `# ${kindWords("hetzner-backup")} (1: 0c 1w 0i 0s)`, `# ${kindWords("sonarr-queue")} (1: 0c 1w 0i 0s)`, `# ${kindWords("fragnet-backup")} (1: 0c 0w 0i 1s)`],
);
});
test("a folded group keeps its header, with the count, and shows none of its alerts", () => {
const folded = groupAlerts(list, "source", new Set(["source:Hetzner"]), label);
assert.deepEqual(shape(folded), ["# Sonarr (2: 1c 1w 0i 0s)", "2", "4", "# Hetzner (3: 0c 2w 1i 0s)", "# Fragnet (1: 0c 0w 0i 1s)", "6"]);
assert.equal((folded.find((x) => isGroup(x) && x.label === "Hetzner") as { folded: boolean }).folded, true);
});
test("what can be picked from a group is what is not held back; ids are by grouping, so folding one grouping does not fold another", () => {
const g = groupAlerts(list, "source", none, label).filter(isGroup);
assert.deepEqual(g.map((x) => [x.id, x.pickable]), [["source:Sonarr", ["2", "4"]], ["source:Hetzner", ["1", "3", "5"]], ["source:Fragnet", []]]);
assert.equal(groupAlerts(list, "category", new Set(["source:Hetzner"]), label).filter((x) => !isGroup(x)).length, 6, "nothing folded");
});
test("no alerts is no groups", () => {
assert.deepEqual(groupAlerts([], "source", none, label), []);
});
});
+211
View File
@@ -0,0 +1,211 @@
/**
* App versions: comparing what each integration last read with the newest release (kept from GitHub, only while switched on), the page's data and its switch, the optional alert and /metrics.
*
* Run with: npm test --workspace server
*/
import { test, describe, before, after } from "node:test";
import assert from "node:assert/strict";
import type { Server } from "node:http";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { AddressInfo } from "node:net";
const dir = mkdtempSync(join(tmpdir(), "hm-versions-"));
process.env.DATABASE_PATH = join(dir, "test.sqlite");
process.env.SESSION_DIR = join(dir, "sessions");
process.env.CREDENTIALS_ENCRYPTION_KEY = "ab".repeat(32);
process.env.HEALTH_STARTUP_GRACE_MS = "0";
const express = (await import("express")).default;
const { db } = await import("../src/db/client.js");
const { users, integrations, arrState, grafanaState, adguardState, auditLog } = await import("../src/db/schema.js");
const { runMigrations } = await import("../src/db/migrate.js");
const av = await import("../src/services/appVersions.js");
const { versionsRouter } = await import("../src/routes/versions.js");
const { evaluateHealth, collectSnapshot } = await import("../src/services/healthMonitor.js");
const { subjectOfConditionKey } = await import("../src/services/maintenance.js");
const { getAlerts, invalidateAlertsCache } = await import("../src/services/alerts.js");
const { buildMetrics } = await import("../src/services/metrics.js");
const { getSettings, updateSettings } = await import("../src/services/settingsStore.js");
const { eq } = await import("drizzle-orm");
await runMigrations();
const DAY = 86_400_000;
const iso = (ms: number) => new Date(ms).toISOString();
describe("reading a version", () => {
test("the first three numbers, after the prefixes projects put on theirs", () => {
assert.deepEqual(av.parseVersion("v4.0.9.2244"), [4, 0, 9]);
assert.deepEqual(av.parseVersion("4.0.9.2244"), [4, 0, 9]);
assert.deepEqual(av.parseVersion("n8n@1.60.0"), [1, 60, 0]);
assert.deepEqual(av.parseVersion("version/2024.8.3"), [2024, 8, 3]);
assert.deepEqual(av.parseVersion("v0.107.52"), [0, 107, 52]);
assert.deepEqual(av.parseVersion("2.12.0-rc1"), [2, 12, 0]);
assert.deepEqual(av.parseVersion("v10"), [10, 0, 0]);
for (const none of [null, undefined, "", "latest", "nightly"]) assert.equal(av.parseVersion(none), null, String(none));
});
test("how far behind: the first number is major, the second minor, the third a patch; a build number is not a difference; newer than the newest is ahead", () => {
assert.equal(av.lagOf("4.0.9.2244", "v4.0.9.2300"), "same");
assert.equal(av.lagOf("4.0.9.2244", "v4.0.10.2300"), "patch");
assert.equal(av.lagOf("11.2.0", "v11.3.1"), "minor");
assert.equal(av.lagOf("10.9.11", "v11.0.0"), "major");
assert.equal(av.lagOf("2024.8.3", "version/2025.2.0"), "major", "a new year of Authentik");
assert.equal(av.lagOf("v0.107.52", "v0.107.51"), "ahead");
assert.equal(av.lagOf("1.0.0", "v1.0.0"), "same");
assert.equal(av.lagOf(null, "v1.0.0"), "unknown");
assert.equal(av.lagOf("1.0.0", ""), "unknown");
});
});
describe("what is kept and shown", () => {
const ids: Record<string, number> = {};
const mk = async (type: string, name: string, enabled = true) => (await db.insert(integrations).values({ type: type as "sonarr", name, baseUrl: "http://x", enabled, config: JSON.stringify({}) }).returning({ id: integrations.id }))[0].id;
const asked: string[] = [];
let failing = new Set<string>();
const published = (daysAgo: number) => iso(Date.now() - daysAgo * DAY);
before(async () => {
ids.sonarr = await mk("sonarr", "TV");
ids.sonarr2 = await mk("sonarr", "TV 4K");
ids.grafana = await mk("grafana", "Dashboards");
ids.adguard = await mk("adguard", "DNS");
ids.radarr = await mk("radarr", "Films"); // never read: no version
ids.proxmox = await mk("proxmox", "Cluster"); // no release to compare
ids.off = await mk("grafana", "Old grafana", false);
const at = new Date().toISOString();
await db.insert(arrState).values([{ integrationId: ids.sonarr, version: "4.0.9.2244", health: "[]", updatedAt: at }, { integrationId: ids.sonarr2, version: "4.0.9.2244", health: "[]", updatedAt: at }]);
await db.insert(grafanaState).values({ integrationId: ids.grafana, version: "11.2.0", updatedAt: at });
await db.insert(adguardState).values({ integrationId: ids.adguard, version: "v0.107.52", updatedAt: at });
av.setReleaseFetcherForTests(async (repo) => {
asked.push(repo);
if (failing.has(repo)) throw new Error("GitHub's limit on requests without a sign-in was reached — it is tried again later");
const tags: Record<string, [string, number]> = { "Sonarr/Sonarr": ["v4.0.10.2400", 90], "grafana/grafana": ["v11.2.0", 200], "AdguardTeam/AdGuardHome": ["v0.107.57", 10] };
const t = tags[repo];
if (!t) throw new Error("GitHub lists no releases for it");
return { tag: t[0], publishedAt: published(t[1]), url: `https://github.com/${repo}/releases/tag/${t[0]}` };
});
});
after(() => av.setReleaseFetcherForTests(null));
test("nothing is listed that has no release to compare with, no version read, or is switched off; before any lookup the newest is not known", async () => {
const rows = await av.versionRows();
assert.deepEqual(rows.map((r) => [r.label, r.name, r.running, r.latest, r.lag]), [["AdGuard Home", "DNS", "v0.107.52", null, "unknown"], ["Grafana", "Dashboards", "11.2.0", null, "unknown"], ["Sonarr", "TV", "4.0.9.2244", null, "unknown"], ["Sonarr", "TV 4K", "4.0.9.2244", null, "unknown"]]);
});
test("a lookup asks once for each repository — two Sonarrs are one question — and keeps what it learns", async () => {
asked.length = 0;
const r = await av.refreshReleases(new Date(), true);
assert.deepEqual([r.checked, r.failed], [3, 0]);
assert.deepEqual([...asked].sort(), ["AdguardTeam/AdGuardHome", "Sonarr/Sonarr", "grafana/grafana"]);
const rows = await av.versionRows();
const by = (n: string) => rows.find((x) => x.name === n)!;
assert.deepEqual([by("TV").lag, by("TV").latest?.tag, by("TV").daysOut], ["patch", "v4.0.10.2400", 90]);
assert.deepEqual([by("Dashboards").lag, by("Dashboards").daysOut], ["same", null]);
assert.deepEqual([by("DNS").lag, by("DNS").daysOut], ["patch", 10], "0.107.52 against 0.107.57");
});
test("a repository looked up lately is not asked again; with force it is", async () => {
asked.length = 0;
await av.refreshReleases(new Date());
assert.deepEqual(asked, []);
await av.refreshReleases(new Date(), true);
assert.equal(asked.length, 3);
});
test("a lookup that fails keeps the release it had and says why; a later pass tries it again at once", async () => {
failing = new Set(["grafana/grafana"]);
const r = await av.refreshReleases(new Date(), true);
assert.deepEqual([r.checked, r.failed], [2, 1]);
const g = (await av.versionRows()).find((x) => x.name === "Dashboards")!;
assert.deepEqual([g.latest?.tag, g.lookupError], ["v11.2.0", "GitHub's limit on requests without a sign-in was reached — it is tried again later"]);
failing = new Set();
asked.length = 0;
await av.refreshReleases(new Date());
assert.deepEqual(asked, ["grafana/grafana"], "only the one that had failed");
assert.equal((await av.versionRows()).find((x) => x.name === "Dashboards")!.lookupError, null);
});
test("the alert: a newer release out for the number of days or more; not before; never at 0", async () => {
const rows = await av.versionRows();
assert.deepEqual(av.versionProblems(rows, 0), []);
const p = av.versionProblems(rows, 60);
assert.deepEqual(p.map((x) => x.key), [`version-behind:${ids.sonarr}`, `version-behind:${ids.sonarr2}`]);
assert.equal(p[0].message, "TV: Sonarr 4.0.9.2244 is running; v4.0.10.2400 came out 90 days ago (a patch)");
assert.deepEqual(av.versionProblems(rows, 91), []);
assert.equal(av.versionProblems(rows, 5).length, 3, "the two Sonarrs and AdGuard Home (Grafana is up to date)");
});
describe("the page, its switch, the health pass, the Alerts page and /metrics", () => {
let app: Server;
let base = "";
const call = (sub: string, method: string, path: string, body?: unknown) => fetch(`${base}${path}`, { method, headers: { "content-type": "application/json", "x-sub": sub }, body: body === undefined ? undefined : JSON.stringify(body) });
before(async () => {
await db.insert(users).values([{ oidcSub: "viewer", name: "Vic", role: "viewer" }, { oidcSub: "admin", name: "Ada", role: "admin" }]);
const a = express();
a.use(express.json());
a.use((req, _res, next) => {
(req as unknown as { session: unknown }).session = { user: { sub: String(req.headers["x-sub"]) } };
next();
});
a.use("/api/versions", versionsRouter);
app = await new Promise<Server>((resolve) => {
const s = a.listen(0, "127.0.0.1", () => resolve(s));
});
base = `http://127.0.0.1:${(app.address() as AddressInfo).port}/api/versions`;
});
after(() => {
app.close();
try {
rmSync(dir, { recursive: true, force: true });
} catch {
// Windows keeps the database file locked until the process exits.
}
});
test("it is off by default; everyone signed in sees the list, only an administrator switches it, and a refresh while it is off is refused", async () => {
const off = (await (await call("viewer", "GET", "")).json()) as { enabled: boolean; alertAfterDays: number; rows: unknown[] };
assert.deepEqual([off.enabled, off.alertAfterDays, off.rows.length], [false, 0, 4]);
assert.equal((await call("viewer", "PUT", "/settings", { enabled: true, alertAfterDays: 0 })).status, 403);
assert.equal((await call("admin", "POST", "/refresh")).status, 400);
assert.equal((await call("viewer", "POST", "/refresh")).status, 403);
});
test("it must be given the whole of what it asks; switching on looks the releases up at once, and is in the audit log", async () => {
assert.equal((await call("admin", "PUT", "/settings", { enabled: true })).status, 400);
assert.equal((await call("admin", "PUT", "/settings", { enabled: true, alertAfterDays: 999 })).status, 400);
asked.length = 0;
const on = (await (await call("admin", "PUT", "/settings", { enabled: true, alertAfterDays: 60 })).json()) as { enabled: boolean; alertAfterDays: number; lastCheckedAt: string | null; rows: { name: string; lag: string }[] };
assert.deepEqual([on.enabled, on.alertAfterDays, on.lastCheckedAt !== null], [true, 60, true]);
assert.equal(on.rows.find((r) => r.name === "TV")!.lag, "patch");
assert.equal(asked.length, 3, "looked up on switching on");
assert.equal((await db.select().from(auditLog)).filter((e) => e.targetType === "versionChecks").length, 1);
});
test("with the lookup and a number set, the health pass raises it, the Alerts page puts it under Updates with a link, and maintenance on the integration holds it", async () => {
const { snapshot } = await collectSnapshot();
assert.deepEqual(snapshot.versions?.map((p) => p.key), [`version-behind:${ids.sonarr}`, `version-behind:${ids.sonarr2}`]);
const list = evaluateHealth(snapshot, { serverOfflineMinutes: 60, diskUsagePercent: 90 }, Date.now(), { skipServers: true });
assert.equal(list.find((x) => x.key === `version-behind:${ids.sonarr}`)!.source, "versions");
assert.equal(subjectOfConditionKey(`version-behind:${ids.sonarr}`), `integration:${ids.sonarr}`);
invalidateAlertsCache();
const a = (await getAlerts(true)).alerts.find((x) => x.id.endsWith(`version-behind:${ids.sonarr}`))!;
assert.deepEqual([a.category, a.source, a.link, a.target?.type, a.kind], ["updates", "Versions", "/versions", "integration", "version-behind"]);
});
test("/metrics says which have a newer release — only with the lookup on — and with a number of 0 no alert is raised", async () => {
const text = await buildMetrics(new Date());
assert.match(text, new RegExp(`homelab_app_update_available\\{app="sonarr",name="TV",running="4\\.0\\.9\\.2244",latest="v4\\.0\\.10\\.2400"\\} 1`));
assert.match(text, /homelab_app_update_available\{app="grafana",name="Dashboards",running="11\.2\.0",latest="v11\.2\.0"\} 0/);
const checks = (await getSettings()).versionChecks;
await updateSettings({ versionChecks: { ...checks, alertAfterDays: 0 } });
assert.equal((await collectSnapshot()).snapshot.versions, undefined);
await updateSettings({ versionChecks: { ...checks, enabled: false } });
assert.doesNotMatch(await buildMetrics(new Date()), /homelab_app_update_available/);
assert.equal((await collectSnapshot()).snapshot.versions, undefined);
});
});
});
+20
View File
@@ -128,6 +128,26 @@ describe("what the *arr API says", () => {
assert.ok(!seen.some((s) => s.path.includes("/queue"))); assert.ok(!seen.some((s) => s.path.includes("/queue")));
}); });
test("Lidarr and Readarr are under v1, ask for their own unknown items, and have a queue and the missing like Sonarr (and no indexers)", async () => {
for (const [kind, include] of [["lidarr", "includeUnknownArtistItems=true"], ["readarr", "includeUnknownAuthorItems=true"]] as const) {
seen.length = 0;
const r = await adapter.createArrAdapter(kind, { url, apiKey: "k" }).read();
assert.deepEqual([r.queue?.total, r.missing, r.indexers], [1, 7, null], kind);
assert.ok(seen.every((s) => s.path.startsWith("/api/v1/") && s.method === "GET"), kind);
assert.ok(seen.some((s) => s.path.includes(include)), kind);
assert.ok(!seen.some((s) => s.path.includes("/indexer")), kind);
}
assert.deepEqual(adapter.ARR_KINDS, ["sonarr", "radarr", "prowlarr", "lidarr", "readarr"]);
});
test("an address that is another one of the family is said, for the new kinds too", async () => {
appName = "Sonarr";
assert.match((await adapter.createArrAdapter("lidarr", { url, apiKey: "k" }).ping()).error ?? "", /That is Sonarr, not Lidarr/);
appName = "Lidarr";
assert.equal((await adapter.createArrAdapter("lidarr", { url, apiKey: "k" }).ping()).ok, true);
appName = "Sonarr";
});
test("a part that cannot be read is null, not empty; the status and the health are what must work", async () => { test("a part that cannot be read is null, not empty; the status and the health are what must work", async () => {
queueStatus = 500; queueStatus = 500;
const r = await adapter.createArrAdapter("sonarr", { url, apiKey: "k" }).read(); const r = await adapter.createArrAdapter("sonarr", { url, apiKey: "k" }).read();
+204
View File
@@ -0,0 +1,204 @@
/**
* The integration inspector: what an answer looks like (names and kinds, never values), where what a reader made of it is empty, the requests written down while a read runs, and the route that
* runs one for an administrator.
*
* Run with: npm test --workspace server
*/
import { test, describe, before, after } from "node:test";
import assert from "node:assert/strict";
import { createServer as createHttpServer, type Server } from "node:http";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { AddressInfo } from "node:net";
const dir = mkdtempSync(join(tmpdir(), "hm-inspect-"));
process.env.DATABASE_PATH = join(dir, "test.sqlite");
process.env.SESSION_DIR = join(dir, "sessions");
process.env.CREDENTIALS_ENCRYPTION_KEY = "ab".repeat(32);
const express = (await import("express")).default;
const { db } = await import("../src/db/client.js");
const { users, integrations, auditLog } = await import("../src/db/schema.js");
const { runMigrations } = await import("../src/db/migrate.js");
const insp = await import("../src/integrations/inspector.js");
const { requestJson } = await import("../src/integrations/httpJson.js");
const { integrationsRouter } = await import("../src/routes/integrations.js");
const { eq } = await import("drizzle-orm");
await runMigrations();
describe("what an answer looks like", () => {
test("keys and the kinds of what they hold — and never a value", () => {
const s = insp.shapeOf({ id: 7, name: "super-secret-name", ok: true, nothing: null, tags: ["a", "b"], nested: { when: "2026-01-01", n: 1.5 } });
assert.equal(s, "{ id: number, name: string, ok: boolean, nothing: null, tags: array(2) of string, nested: { when: string, n: number } }");
assert.doesNotMatch(s, /super-secret|2026|7\b/);
});
test("a list of objects shows the keys of them all, and marks a key that only some have", () => {
const s = insp.shapeOf([{ id: 1, name: "a" }, { id: 2, name: "b", extra: 1 }, { id: 3, name: "c" }]);
assert.equal(s, "array(3) of { id: number, name: string, extra?: number }");
assert.equal(insp.shapeOf([]), "array(0)");
assert.equal(insp.shapeOf([1, "x", null]), "array(3) of number | string | null");
assert.equal(insp.shapeOf("text"), "string");
});
test("it goes three levels down and no further, caps the keys and the length", () => {
assert.equal(insp.shapeOf({ a: { b: { c: { d: { e: 1 } } } } }), "{ a: { b: { c: object } } }");
const many = Object.fromEntries(Array.from({ length: 40 }, (_, i) => [`k${i}`, i]));
assert.match(insp.shapeOf(many), /, … \}$/);
assert.ok(insp.shapeOf(Object.fromEntries(Array.from({ length: 30 }, (_, i) => [`key${i}_${"x".repeat(80)}`, "v"]))).length <= 1810);
});
test("a path is reported with the names of its query, not what they were set to", () => {
assert.equal(insp.pathForReport(new URL("http://x/api/v3/queue?page=1&pageSize=100&apikey=SECRET")), "/api/v3/queue?page&pageSize&apikey");
assert.equal(insp.pathForReport(new URL("http://x/api/health")), "/api/health");
});
});
describe("where it is empty", () => {
test("null, empty text, empty lists — and a field null in every item of a list", () => {
const e = insp.emptyFields({ version: null, name: "", queue: { total: 0, items: [] }, servers: [{ id: 1, usage: null, name: "a" }, { id: 2, usage: null, name: "b" }], fine: "yes" });
assert.deepEqual(e.map((x) => [x.path, x.what]), [["version", "null"], ["name", "text"], ["queue.items", "list"], ["servers[].usage", "every-item"]]);
});
test("a field that is null in some items only is not reported; numbers such as 0 are not empty", () => {
assert.deepEqual(insp.emptyFields({ servers: [{ usage: null }, { usage: 5 }], count: 0, ok: false }), []);
});
test("a result that is itself empty, and a limit", () => {
assert.deepEqual(insp.emptyFields(null), [{ path: "(the result)", what: "null" }]);
assert.equal(insp.emptyFields(Object.fromEntries(Array.from({ length: 80 }, (_, i) => [`f${i}`, null])), 10).length, 10);
});
});
describe("the requests written down while a read runs", () => {
let http: Server;
let url = "";
before(async () => {
http = createHttpServer((req, res) => {
res.setHeader("content-type", "application/json");
if (req.url?.startsWith("/json")) return void res.end(JSON.stringify({ items: [{ id: 1, secret: "hidden-value" }], total: 1 }));
if (req.url?.startsWith("/html")) return void res.end("<html>nope</html>");
res.statusCode = 404;
res.end("{}");
});
await new Promise<void>((r) => http.listen(0, "127.0.0.1", r));
url = `http://127.0.0.1:${(http.address() as AddressInfo).port}`;
});
after(() => http.close());
test("each request: method, path (no values), status, time, size and the shape of the answer; a failed one too", async () => {
const { calls, value, error } = await insp.inspectCalls(async () => {
await requestJson(`${url}/json?token=hidden&page=2`);
await requestJson(`${url}/html`);
await requestJson(`${url}/missing`);
await requestJson("http://127.0.0.1:1/refused").catch(() => undefined);
return "done";
});
assert.deepEqual([value, error], ["done", null]);
assert.deepEqual(calls.map((c) => [c.method, c.path, c.status]), [["GET", "/json?token&page", 200], ["GET", "/html", 200], ["GET", "/missing", 404], ["GET", "/refused", 0]]);
assert.equal(calls[0].shape, "{ items: array(1) of { id: number, secret: string }, total: number }");
assert.equal(calls[1].shape, null, "not JSON");
assert.ok(calls[0].bytes > 10 && calls[0].ms >= 0);
assert.ok(calls[3].error);
assert.doesNotMatch(JSON.stringify(calls), /hidden-value|hidden/);
});
test("a read that throws is reported with what it asked before it did; and nothing is written down outside an inspection", async () => {
const { calls, value, error } = await insp.inspectCalls(async () => {
await requestJson(`${url}/json`);
throw new Error("the reader gave up");
});
assert.deepEqual([calls.length, value, error], [1, undefined, "the reader gave up"]);
assert.equal(insp.isInspecting(), false);
await requestJson(`${url}/json`);
const second = await insp.inspectCalls(async () => 1);
assert.deepEqual(second.calls, [], "the earlier request was not kept");
});
test("two inspections at once keep their own requests", async () => {
const [a, b] = await Promise.all([insp.inspectCalls(async () => requestJson(`${url}/json?a=1`)), insp.inspectCalls(async () => requestJson(`${url}/html?b=1`))]);
assert.deepEqual([a.calls.map((c) => c.path), b.calls.map((c) => c.path)], [["/json?a"], ["/html?b"]]);
});
});
describe("the route", () => {
let grafana: Server;
let app: Server;
let base = "";
let grafanaId = 0;
let tailscaleId = 0;
let offId = 0;
const call = (sub: string, id: number) => fetch(`${base}/${id}/inspect`, { method: "POST", headers: { "content-type": "application/json", "x-sub": sub } });
before(async () => {
await db.insert(users).values([{ oidcSub: "viewer", name: "Vic", role: "viewer" }, { oidcSub: "admin", name: "Ada", role: "admin" }]);
grafana = createHttpServer((req, res) => {
res.setHeader("content-type", "application/json");
const u = new URL(req.url ?? "", "http://x");
if (u.pathname === "/api/health") return void res.end(JSON.stringify({ database: "ok", version: "11.2.0" }));
if (u.pathname === "/api/prometheus/grafana/api/v1/rules") return void res.end(JSON.stringify({ status: "success", data: { groups: [{ name: "g", file: "f", rules: [{ name: "r", state: "inactive", health: "ok" }] }] } }));
if (u.pathname === "/api/datasources") return void res.end(JSON.stringify([]));
res.statusCode = 404;
res.end("{}");
});
await new Promise<void>((r) => grafana.listen(0, "127.0.0.1", r));
const gurl = `http://127.0.0.1:${(grafana.address() as AddressInfo).port}`;
grafanaId = (await db.insert(integrations).values({ type: "grafana", name: "Dashboards", baseUrl: gurl, config: JSON.stringify({ url: gurl, token: "glsa_x" }) }).returning({ id: integrations.id }))[0].id;
tailscaleId = (await db.insert(integrations).values({ type: "tailscale", name: "Tailnet", baseUrl: "http://x", config: JSON.stringify({}) }).returning({ id: integrations.id }))[0].id;
offId = (await db.insert(integrations).values({ type: "grafana", name: "Off", baseUrl: gurl, enabled: false, config: JSON.stringify({ url: gurl, token: "t" }) }).returning({ id: integrations.id }))[0].id;
const a = express();
a.use(express.json());
a.use((req, _res, next) => {
(req as unknown as { session: unknown }).session = { user: { sub: String(req.headers["x-sub"]) } };
next();
});
a.use("/api/integrations", integrationsRouter);
app = await new Promise<Server>((resolve) => {
const s = a.listen(0, "127.0.0.1", () => resolve(s));
});
base = `http://127.0.0.1:${(app.address() as AddressInfo).port}/api/integrations`;
});
after(() => {
app.close();
grafana.close();
try {
rmSync(dir, { recursive: true, force: true });
} catch {
// Windows keeps the database file locked until the process exits.
}
});
test("an administrator gets what it asked, what came back (as shapes) and where the result is empty; the token is nowhere in it", async () => {
const res = await call("admin", grafanaId);
assert.equal(res.status, 200);
const text = await res.text();
assert.doesNotMatch(text, /glsa_x/);
const body = JSON.parse(text) as { ok: boolean; type: string; calls: { path: string; status: number; shape: string | null }[]; resultShape: string; empty: { path: string; what: string }[] };
assert.deepEqual([body.ok, body.type], [true, "grafana"]);
assert.ok(body.calls.some((c) => c.path === "/api/health" && c.status === 200 && c.shape === "{ database: string, version: string }"));
assert.ok(body.calls.some((c) => c.path === "/api/prometheus/grafana/api/v1/rules"));
assert.match(body.resultShape, /^\{ version: string, rules: \{ total: number, items: array\(0\) \}, datasources: array\(0\) \}$/);
assert.ok(body.empty.some((e) => e.path === "rules.items" && e.what === "list"));
assert.ok(body.empty.some((e) => e.path === "datasources" && e.what === "list"));
});
test("a failed read is reported with what it asked, not as an error of the route", async () => {
await db.update(integrations).set({ config: JSON.stringify({ url: "http://127.0.0.1:1", token: "t" }) }).where(eq(integrations.id, grafanaId));
const body = (await (await call("admin", grafanaId)).json()) as { ok: boolean; error: string; calls: { status: number; error?: string }[]; resultShape: string | null };
assert.deepEqual([body.ok, body.resultShape, body.calls.some((c) => c.status === 0)], [false, null, true]);
assert.ok(body.error);
});
test("only an administrator; a kind with no reader, a switched-off integration and an unknown one are refused; each inspection is in the audit log", async () => {
assert.equal((await call("viewer", grafanaId)).status, 403);
assert.equal((await call("admin", tailscaleId)).status, 400);
assert.equal(((await (await call("admin", tailscaleId)).json()) as { error: string }).error, "not_inspectable");
assert.equal(((await (await call("admin", offId)).json()) as { error: string }).error, "integration_disabled");
assert.equal((await call("admin", 99999)).status, 404);
const entries = (await db.select().from(auditLog)).filter((e) => e.action === "inspect");
assert.equal(entries.length, 2, "the two that ran");
assert.deepEqual(entries.map((e) => [e.category, e.targetType]), [["integration", "integration"], ["integration", "integration"]]);
});
});
+2
View File
@@ -52,6 +52,7 @@ import Alerts from "./pages/Alerts";
import Generator from "./pages/Generator"; import Generator from "./pages/Generator";
import Maintenance from "./pages/Maintenance"; import Maintenance from "./pages/Maintenance";
import Updates from "./pages/Updates"; import Updates from "./pages/Updates";
import Versions from "./pages/Versions";
import IncidentDetail from "./pages/IncidentDetail"; import IncidentDetail from "./pages/IncidentDetail";
import Inventory from "./pages/Inventory"; import Inventory from "./pages/Inventory";
import InventoryLabels from "./pages/InventoryLabels"; import InventoryLabels from "./pages/InventoryLabels";
@@ -184,6 +185,7 @@ function Shelled({ user }: { user: CurrentUser }) {
<Route path="/hetzner" element={<Hetzner />} /> <Route path="/hetzner" element={<Hetzner />} />
<Route path="/game-servers" element={<GameServers user={user} />} /> <Route path="/game-servers" element={<GameServers user={user} />} />
<Route path="/media" element={<Media />} /> <Route path="/media" element={<Media />} />
<Route path="/versions" element={<Versions user={user} />} />
<Route path="/adguard" element={<AdguardHome />} /> <Route path="/adguard" element={<AdguardHome />} />
<Route path="/grafana" element={<Grafana />} /> <Route path="/grafana" element={<Grafana />} />
<Route path="/speedtest" element={<Speedtest />} /> <Route path="/speedtest" element={<Speedtest />} />
+39 -2
View File
@@ -1463,7 +1463,7 @@ export interface FragnetAccount {
/** One Sonarr, Radarr or Prowlarr (an integration of that type), as the media page shows it. */ /** One Sonarr, Radarr or Prowlarr (an integration of that type), as the media page shows it. */
export interface ArrApp { export interface ArrApp {
id: number; id: number;
kind: "sonarr" | "radarr" | "prowlarr"; kind: "sonarr" | "radarr" | "prowlarr" | "lidarr" | "readarr";
name: string; name: string;
enabled: boolean; enabled: boolean;
url: string; url: string;
@@ -1528,6 +1528,37 @@ export interface GrafanaInstance {
problems: { key: string; message: string; severity: "warning" | "critical" }[]; problems: { key: string; message: string; severity: "warning" | "critical" }[];
} }
/** What running an integration's reader once showed: each request it made (and the shape of the answer, never its values), what it made of them, and where that is empty. */
export interface IntegrationInspection {
type: IntegrationType;
name: string;
ok: boolean;
error: string | null;
ms: number;
calls: { method: string; path: string; status: number; ms: number; bytes: number; shape: string | null; error?: string }[];
resultShape: string | null;
empty: { path: string; what: "null" | "text" | "list" | "every-item" }[];
}
/** What each connected application runs, against the newest release kept for it. */
export interface AppVersions {
enabled: boolean;
alertAfterDays: number;
lastCheckedAt: string | null;
rows: {
integrationId: number;
type: string;
label: string;
name: string;
running: string;
latest: { tag: string; publishedAt: string | null; url: string | null } | null;
lookupError: string | null;
lag: "same" | "patch" | "minor" | "major" | "ahead" | "unknown";
daysOut: number | null;
checkedAt: string | null;
}[];
}
export type FragnetSignal = "start" | "stop" | "restart" | "kill"; export type FragnetSignal = "start" | "stop" | "restart" | "kill";
export interface CostItem { export interface CostItem {
@@ -2663,7 +2694,7 @@ export interface ManualTaskInput {
enabled?: boolean; enabled?: boolean;
} }
export type IntegrationType = "proxmox" | "synology" | "semaphore" | "tailscale" | "gitea" | "dockhand" | "uptimekuma" | "phpipam" | "pbs" | "osticket" | "unifi" | "npm" | "homeassistant" | "ups" | "jellyfin" | "authentik" | "prometheus" | "immich" | "paperless" | "nextcloud" | "netbox" | "n8n" | "hetzner" | "speedtest" | "fragnet" | "sonarr" | "radarr" | "prowlarr" | "adguard" | "grafana"; export type IntegrationType = "proxmox" | "synology" | "semaphore" | "tailscale" | "gitea" | "dockhand" | "uptimekuma" | "phpipam" | "pbs" | "osticket" | "unifi" | "npm" | "homeassistant" | "ups" | "jellyfin" | "authentik" | "prometheus" | "immich" | "paperless" | "nextcloud" | "netbox" | "n8n" | "hetzner" | "speedtest" | "fragnet" | "sonarr" | "radarr" | "prowlarr" | "adguard" | "grafana" | "lidarr" | "readarr";
export interface IntegrationField { export interface IntegrationField {
key: string; key: string;
@@ -3265,6 +3296,11 @@ export const api = {
list: () => request<{ speedtests: SpeedtestInfo[] }>("/api/speedtest"), list: () => request<{ speedtests: SpeedtestInfo[] }>("/api/speedtest"),
history: (id: number, days: number) => request<SpeedtestHistory>(`/api/speedtest/${id}/history?days=${days}`), history: (id: number, days: number) => request<SpeedtestHistory>(`/api/speedtest/${id}/history?days=${days}`),
}, },
versions: {
get: () => request<AppVersions>("/api/versions"),
save: (data: { enabled: boolean; alertAfterDays: number }) => request<AppVersions>("/api/versions/settings", { method: "PUT", body: JSON.stringify(data) }),
refresh: () => request<AppVersions & { result: { checked: number; failed: number } }>("/api/versions/refresh", { method: "POST" }),
},
adguard: { adguard: {
list: () => request<{ instances: AdguardInstance[] }>("/api/adguard"), list: () => request<{ instances: AdguardInstance[] }>("/api/adguard"),
}, },
@@ -3843,6 +3879,7 @@ export const api = {
integration: { id: number; type: IntegrationType; name: string; enabled: boolean }; integration: { id: number; type: IntegrationType; name: string; enabled: boolean };
config: Record<string, string | boolean>; config: Record<string, string | boolean>;
}>(`/api/integrations/${id}/config`), }>(`/api/integrations/${id}/config`),
inspect: (id: number) => request<IntegrationInspection>(`/api/integrations/${id}/inspect`, { method: "POST" }),
testExisting: (id: number, config: Record<string, string | boolean>) => testExisting: (id: number, config: Record<string, string | boolean>) =>
request<{ ok: boolean; latencyMs?: number; error?: string }>(`/api/integrations/${id}/test`, { request<{ ok: boolean; latencyMs?: number; error?: string }>(`/api/integrations/${id}/test`, {
method: "POST", method: "POST",
+24
View File
@@ -0,0 +1,24 @@
import type { AlertGroup } from "../utils/alertGroups";
/** The header row of one group: fold it, and pick everything in it. */
export default function AlertGroupRow({ group, colSpan, canPick, allPicked, onToggle, onPick }: { group: AlertGroup; colSpan: number; canPick: boolean; allPicked: boolean; onToggle: () => void; onPick: () => void }) {
return (
<tr className="alert-group-row">
<td colSpan={colSpan} className="bg-body-tertiary py-2">
<div className="d-flex flex-wrap align-items-center gap-2">
{canPick && group.pickable.length > 0 && <input type="checkbox" className="form-check-input align-middle" aria-label={`Pick all in ${group.label}`} checked={allPicked} onChange={onPick} />}
<button type="button" className="btn btn-link btn-sm p-0 text-reset text-decoration-none fw-bold" aria-expanded={!group.folded} onClick={onToggle}>
<span aria-hidden="true">{group.folded ? "▸" : "▾"}</span> {group.label}
</button>
<span className="text-secondary small">
{group.count} {group.count === 1 ? "alert" : "alerts"}
</span>
{group.critical > 0 && <span className="badge bg-red-lt text-red">{group.critical} critical</span>}
{group.warning > 0 && <span className="badge bg-yellow-lt text-yellow">{group.warning} warning</span>}
{group.info > 0 && <span className="badge bg-blue-lt text-blue">{group.info} info</span>}
{group.silenced > 0 && <span className="badge bg-secondary-lt text-secondary">{group.silenced} held back</span>}
</div>
</td>
</tr>
);
}
+1 -32
View File
@@ -5,42 +5,11 @@ import { ALERT_KIND_RUNBOOKS_CHANGED, useAlertKindRunbooks } from "../hooks/useA
import { readableError } from "../utils/errors"; import { readableError } from "../utils/errors";
import { removeWithUndo } from "../utils/undoable"; import { removeWithUndo } from "../utils/undoable";
import { toast } from "../utils/toast"; import { toast } from "../utils/toast";
import { kindWords } from "../utils/alertKinds";
import DetailDrawer from "./DetailDrawer"; import DetailDrawer from "./DetailDrawer";
import RunbookText from "./RunbookText"; import RunbookText from "./RunbookText";
import TimeAgo from "./TimeAgo"; import TimeAgo from "./TimeAgo";
/** How a kind of alert reads in a sentence; any kind not listed is its own name with the dashes taken out. */
const KIND_WORDS: Record<string, string> = {
offline: "a server that stopped reporting",
disk: "a disk filling up",
"failed-units": "failed services",
smart: "a disk reporting trouble",
zfs: "a ZFS pool in trouble",
"docker-vuln": "critical vulnerabilities in Docker images",
"docker-vuln-high": "many high vulnerabilities in Docker images",
"container-restarting": "a container that keeps restarting",
"container-unhealthy": "an unhealthy container",
"container-stopped": "a container that stopped",
"hetzner-off": "a Hetzner server that is off",
"hetzner-traffic": "a Hetzner traffic quota nearly used",
"hetzner-forecast": "a Hetzner traffic quota the month will pass",
"hetzner-backup": "a Hetzner backup that is late",
"hetzner-lb": "an unhealthy Hetzner load balancer",
"hetzner-cert": "a Hetzner certificate about to end",
"hetzner-fip": "a Hetzner floating IP pointing at nothing",
"fragnet-backup": "a game server backup that is late",
"fragnet-disk": "a game server disk filling up",
"fragnet-memory": "a game server's memory running out",
"sonarr-health": "Sonarr reporting a problem",
"sonarr-queue": "stuck downloads in Sonarr",
"radarr-health": "Radarr reporting a problem",
"radarr-queue": "stuck downloads in Radarr",
"prowlarr-health": "Prowlarr reporting a problem",
"speedtest-slow": "a slow internet line",
"speedtest-failing": "failing speed tests",
};
export const kindWords = (kind: string): string => KIND_WORDS[kind] ?? `${kind.replace(/[-_]+/g, " ")} alerts`;
/** /**
* A small "What to do" link beside an alert: what is written down for this KIND of alert, whichever server or integration it is about (the runbooks of one thing * A small "What to do" link beside an alert: what is written down for this KIND of alert, whichever server or integration it is about (the runbooks of one thing
+132
View File
@@ -0,0 +1,132 @@
import { useEffect, useState } from "react";
import { IconCopy } from "@tabler/icons-react";
import { api, type IntegrationInspection, type IntegrationSummary } from "../api/client";
import { readableError } from "../utils/errors";
import { toast } from "../utils/toast";
import { TYPE_LABELS } from "../utils/integrationTypes";
import DetailDrawer from "./DetailDrawer";
import { BlockSkeleton } from "./Skeleton";
const EMPTY_WORDS: Record<IntegrationInspection["empty"][number]["what"], string> = {
null: "is empty (null)",
text: "is an empty text",
list: "is an empty list",
"every-item": "is empty in every item of the list",
};
/** The report as plain text, to be pasted into a message: only shapes, no values. */
function reportText(i: IntegrationSummary, r: IntegrationInspection): string {
const lines = [`Inspection of ${TYPE_LABELS[i.type]} "${i.name}" — ${r.ok ? "the read worked" : `the read failed: ${r.error}`} (${r.ms} ms)`, ""];
lines.push("What was asked and what came back:");
for (const c of r.calls) {
lines.push(` ${c.method} ${c.path} → ${c.status === 0 ? `no answer (${c.error ?? "?"})` : c.status} · ${c.ms} ms · ${c.bytes} bytes`);
if (c.shape) lines.push(` ${c.shape}`);
}
lines.push("", "What the reader made of it:", ` ${r.resultShape ?? "(nothing)"}`, "", "Empty in it:");
if (r.empty.length === 0) lines.push(" nothing");
for (const e of r.empty) lines.push(` ${e.path} ${EMPTY_WORDS[e.what]}`);
return lines.join("\n");
}
/**
* Runs one integration's reader here and now and shows what it asked, what came back (the shape of each answer — names and kinds of values, never the values) and where what the reader made of
* it is empty. For checking a reader written from documentation against the real system: an answer with a field the reader did not expect shows as an empty field below.
*/
export default function IntegrationInspector({ integration, onClose }: { integration: IntegrationSummary; onClose: () => void }) {
const [report, setReport] = useState<IntegrationInspection | null>(null);
const [error, setError] = useState<string | null>(null);
const [run, setRun] = useState(0);
useEffect(() => {
let alive = true;
setReport(null);
setError(null);
api.integrations
.inspect(integration.id)
.then((r) => alive && setReport(r))
.catch((err) => alive && setError(readableError(err)));
return () => {
alive = false;
};
}, [integration.id, run]);
return (
<DetailDrawer title="Check what can be read" subtitle={`${TYPE_LABELS[integration.type]} · ${integration.name}`} onClose={onClose}>
<p className="text-secondary small">
Reads it once, now, as the app does every few minutes, and shows what each request got back. Only the <strong>names and kinds</strong> of the values are shown — never the values — so the report can be copied into a message. Nothing is stored or changed.
</p>
{!report && !error && <BlockSkeleton lines={4} />}
{error && <div className="text-danger mb-3">{error}</div>}
{report && (
<>
<div className="d-flex flex-wrap align-items-center gap-2 mb-3">
{report.ok ? <span className="badge bg-green-lt text-green">The read worked</span> : <span className="badge bg-red-lt text-red">The read failed</span>}
<span className="text-secondary small">{report.ms} ms · {report.calls.length} {report.calls.length === 1 ? "request" : "requests"}</span>
<span className="ms-auto btn-list flex-nowrap">
<button type="button" className="btn btn-sm" onClick={() => setRun((n) => n + 1)}>
Again
</button>
<button
type="button"
className="btn btn-sm d-inline-flex align-items-center gap-1"
onClick={() => {
void navigator.clipboard?.writeText(reportText(integration, report)).then(
() => toast.success("Copied the report."),
() => toast.error("Couldn't copy it."),
);
}}
>
<IconCopy size={14} aria-hidden="true" /> Copy the report
</button>
</span>
</div>
{report.error && <div className="text-red mb-3">{report.error}</div>}
<h4>What was asked</h4>
{report.calls.length === 0 && <div className="text-secondary small mb-3">No request was made through the shared helper.</div>}
{report.calls.map((c, i) => (
<div key={i} className="border rounded p-2 mb-2">
<div className="d-flex flex-wrap align-items-center gap-2">
<code className="text-break">
{c.method} {c.path}
</code>
{c.status === 0 ? <span className="badge bg-red-lt text-red">no answer</span> : c.status >= 200 && c.status < 300 ? <span className="badge bg-green-lt text-green">{c.status}</span> : <span className="badge bg-yellow-lt text-yellow">{c.status}</span>}
<span className="text-secondary small ms-auto">
{c.ms} ms · {c.bytes.toLocaleString()} bytes
</span>
</div>
{c.error && <div className="text-red small">{c.error}</div>}
{c.shape && <pre className="small text-secondary mb-0 mt-1" style={{ whiteSpace: "pre-wrap", wordBreak: "break-word" }}>{c.shape}</pre>}
{!c.shape && c.status !== 0 && <div className="text-secondary small">The answer was not JSON{c.bytes === 0 ? " (it was empty)" : ""}.</div>}
</div>
))}
<h4 className="mt-3">What the reader made of it</h4>
{report.resultShape ? (
<pre className="small text-secondary" style={{ whiteSpace: "pre-wrap", wordBreak: "break-word" }}>{report.resultShape}</pre>
) : (
<div className="text-secondary small">Nothing — the read failed.</div>
)}
<h4 className="mt-3">Empty in it</h4>
{report.empty.length === 0 ? (
<div className="text-secondary small">Nothing: every field it reads has something in it.</div>
) : (
<>
<p className="text-secondary small">
Often right (no containers, nothing wrong), but where an answer has a field the reader did not expect — named another way, nested somewhere else — it ends up here. Compare with what came back above.
</p>
<ul className="small">
{report.empty.map((e) => (
<li key={`${e.path}-${e.what}`}>
<code>{e.path}</code> {EMPTY_WORDS[e.what]}
</li>
))}
</ul>
</>
)}
</>
)}
</DetailDrawer>
);
}
+2
View File
@@ -36,6 +36,8 @@ const TYPE_LABELS: Record<string, string> = {
prowlarr: "Prowlarr", prowlarr: "Prowlarr",
adguard: "AdGuard Home", adguard: "AdGuard Home",
grafana: "Grafana", grafana: "Grafana",
lidarr: "Lidarr",
readarr: "Readarr",
}; };
export type WallLevel = "critical" | "warning" | "ok" | "unsure"; export type WallLevel = "critical" | "warning" | "ok" | "unsure";
+2
View File
@@ -49,6 +49,7 @@ import {
IconGitCommit, IconGitCommit,
IconShieldCheck, IconShieldCheck,
IconShieldLock, IconShieldLock,
IconVersions,
IconChartAreaLine, IconChartAreaLine,
IconActivityHeartbeat, IconActivityHeartbeat,
IconCopyCheck, IconCopyCheck,
@@ -178,6 +179,7 @@ export const NAV: NavEntry[] = [
items: [ items: [
{ to: "/maintenance", label: "Maintenance", icon: <IconTool size={20} /> }, { to: "/maintenance", label: "Maintenance", icon: <IconTool size={20} /> },
{ to: "/updates", label: "Updates", icon: <IconDownload size={20} /> }, { to: "/updates", label: "Updates", icon: <IconDownload size={20} /> },
{ to: "/versions", label: "App versions", icon: <IconVersions size={20} /> },
{ to: "/coming-up", label: "Coming up", icon: <IconCalendarEvent size={20} /> }, { to: "/coming-up", label: "Coming up", icon: <IconCalendarEvent size={20} /> },
{ to: "/todo", label: "To do", icon: <IconListDetails size={20} /> }, { to: "/todo", label: "To do", icon: <IconListDetails size={20} /> },
{ to: "/osticket", label: "osTicket", icon: <IconTicket size={20} /> }, { to: "/osticket", label: "osTicket", icon: <IconTicket size={20} /> },
+53 -2
View File
@@ -16,6 +16,8 @@ import SilenceButton from "../components/SilenceButton";
import RunPlaybookButton from "../components/RunPlaybookButton"; import RunPlaybookButton from "../components/RunPlaybookButton";
import RunbookEntry from "../components/RunbookEntry"; import RunbookEntry from "../components/RunbookEntry";
import AlertKindRunbookEntry from "../components/AlertKindRunbook"; import AlertKindRunbookEntry from "../components/AlertKindRunbook";
import AlertGroupRow from "../components/AlertGroupRow";
import { GROUP_BY_LABELS, groupAlerts, isGroup, type GroupBy } from "../utils/alertGroups";
import AckDrawer, { ackWords } from "../components/AckDrawer"; import AckDrawer, { ackWords } from "../components/AckDrawer";
import CloseDrawer, { CloseBulkDrawer } from "../components/CloseDrawer"; import CloseDrawer, { CloseBulkDrawer } from "../components/CloseDrawer";
import SnoozeDrawer, { SnoozeBulkDrawer, snoozeWords } from "../components/SnoozeDrawer"; import SnoozeDrawer, { SnoozeBulkDrawer, snoozeWords } from "../components/SnoozeDrawer";
@@ -122,6 +124,25 @@ function AlertsNow({ user, tabs }: { user: CurrentUser; tabs: React.ReactNode })
} }
}); });
const [open, setOpen] = useState<Set<string>>(new Set()); const [open, setOpen] = useState<Set<string>>(new Set());
// The alerts in groups — by where they come from, what kind of problem, or what type of alert — each group folding away; remembered. (Not the same as the nesting above, which is by cause.)
const [groupBy, setGroupBy] = useState<GroupBy>(() => {
try {
const v = localStorage.getItem("alerts.groupBy");
return v === "source" || v === "category" || v === "kind" ? v : "none";
} catch {
return "none";
}
});
const [foldedGroups, setFoldedGroups] = useState<Set<string>>(new Set());
function chooseGroupBy(by: GroupBy) {
setGroupBy(by);
setFoldedGroups(new Set());
try {
localStorage.setItem("alerts.groupBy", by);
} catch {
// a remembered choice is only a convenience
}
}
function chooseGrouped(on: boolean) { function chooseGrouped(on: boolean) {
setGrouped(on); setGrouped(on);
try { try {
@@ -182,6 +203,17 @@ function AlertsNow({ user, tabs }: { user: CurrentUser; tabs: React.ReactNode })
const { sorted, sortKey, sortDir, requestSort } = useSortable(rows); const { sorted, sortKey, sortDir, requestSort } = useSortable(rows);
const { pageItems, page, setPage, pageCount, totalCount } = usePagination(sorted); const { pageItems, page, setPage, pageCount, totalCount } = usePagination(sorted);
// Grouped, every alert is shown (a page is cut across groups); not grouped, a page at a time.
const shown = useMemo(() => groupAlerts(groupBy === "none" ? (pageItems ?? []) : (sorted ?? []), groupBy, foldedGroups, (c) => CATEGORY_LABELS[c]), [groupBy, pageItems, sorted, foldedGroups]);
function toggleGroup(id: string) {
setFoldedGroups((prev) => {
const next = new Set(prev);
if (next.has(id)) next.delete(id);
else next.add(id);
return next;
});
}
const presentCategories = useMemo(() => [...new Set((report?.alerts ?? []).map((a) => a.category))], [report]); const presentCategories = useMemo(() => [...new Set((report?.alerts ?? []).map((a) => a.category))], [report]);
function exportCsv() { function exportCsv() {
@@ -225,6 +257,15 @@ function AlertsNow({ user, tabs }: { user: CurrentUser; tabs: React.ReactNode })
return next; return next;
}); });
} }
/** Ticks everything in a group that can be ticked — or, when all of it is, unticks it. */
function pickGroup(ids: string[]) {
setPicked((prev) => {
const next = new Set(prev);
if (ids.length > 0 && ids.every((id) => next.has(id))) for (const id of ids) next.delete(id);
else for (const id of ids) next.add(id);
return next;
});
}
function afterBulk() { function afterBulk() {
void load(true); void load(true);
reloadBadge(); reloadBadge();
@@ -429,6 +470,13 @@ function AlertsNow({ user, tabs }: { user: CurrentUser; tabs: React.ReactNode })
<option value="warning">Warning</option> <option value="warning">Warning</option>
<option value="info">Info</option> <option value="info">Info</option>
</select> </select>
<select className="form-select form-select-sm w-auto" value={groupBy} aria-label="Grouping" onChange={(e) => chooseGroupBy(e.target.value as GroupBy)}>
{(Object.keys(GROUP_BY_LABELS) as GroupBy[]).map((g) => (
<option key={g} value={g}>
{GROUP_BY_LABELS[g]}
</option>
))}
</select>
<select className="form-select form-select-sm w-auto" value={category} onChange={(e) => setCategory(e.target.value as "all" | AlertCategory)}> <select className="form-select form-select-sm w-auto" value={category} onChange={(e) => setCategory(e.target.value as "all" | AlertCategory)}>
<option value="all">All kinds</option> <option value="all">All kinds</option>
{presentCategories.map((c) => ( {presentCategories.map((c) => (
@@ -496,7 +544,10 @@ function AlertsNow({ user, tabs }: { user: CurrentUser; tabs: React.ReactNode })
</tr> </tr>
</thead> </thead>
<tbody> <tbody>
{(pageItems ?? []).map((a) => ( {shown.map((a) =>
isGroup(a) ? (
<AlertGroupRow key={a.id} group={a} colSpan={canSilence ? 5 : 4} canPick={canSilence} allPicked={a.pickable.length > 0 && a.pickable.every((id) => picked.has(id))} onToggle={() => toggleGroup(a.id)} onPick={() => pickGroup(a.pickable)} />
) : (
<Fragment key={a.id}> <Fragment key={a.id}>
<tr className={`alert-row alert-row-${a.severity}${a.silenced ? " text-secondary" : ""}`} style={a.silenced ? { opacity: 0.65 } : undefined}> <tr className={`alert-row alert-row-${a.severity}${a.silenced ? " text-secondary" : ""}`} style={a.silenced ? { opacity: 0.65 } : undefined}>
<td> <td>
@@ -648,7 +699,7 @@ function AlertsNow({ user, tabs }: { user: CurrentUser; tabs: React.ReactNode })
</tbody> </tbody>
</table> </table>
</div> </div>
<Pagination page={page} pageCount={pageCount} totalCount={totalCount} onPageChange={setPage} /> {groupBy === "none" && <Pagination page={page} pageCount={pageCount} totalCount={totalCount} onPageChange={setPage} />}
</div> </div>
)} )}
</> </>
+9 -1
View File
@@ -13,7 +13,8 @@ import { ErrorNotice } from "../components/Notice";
import { toast } from "../utils/toast"; import { toast } from "../utils/toast";
import UptimeRibbon from "../components/UptimeRibbon"; import UptimeRibbon from "../components/UptimeRibbon";
import RunbookEntry from "../components/RunbookEntry"; import RunbookEntry from "../components/RunbookEntry";
import { groupOfType, TYPE_GROUPS, TYPE_LABELS, TYPE_PAGES } from "../utils/integrationTypes"; import IntegrationInspector from "../components/IntegrationInspector";
import { groupOfType, INSPECTABLE, TYPE_GROUPS, TYPE_LABELS, TYPE_PAGES } from "../utils/integrationTypes";
/** "pve.example.lan" out of "https://pve.example.lan:8006/", or the text itself when it isn't an address. */ /** "pve.example.lan" out of "https://pve.example.lan:8006/", or the text itself when it isn't an address. */
function hostOf(baseUrl: string): string { function hostOf(baseUrl: string): string {
@@ -85,6 +86,7 @@ function IntegrationsList({ user, tabs }: { user: CurrentUser; tabs: ReactNode }
const [integrations, setIntegrations] = useState<IntegrationSummary[] | null>(null); const [integrations, setIntegrations] = useState<IntegrationSummary[] | null>(null);
const [adding, setAdding] = useState(false); const [adding, setAdding] = useState(false);
const [editingIntegration, setEditingIntegration] = useState<IntegrationSummary | null>(null); const [editingIntegration, setEditingIntegration] = useState<IntegrationSummary | null>(null);
const [inspecting, setInspecting] = useState<IntegrationSummary | null>(null);
const [error, setError] = useState<string | null>(null); const [error, setError] = useState<string | null>(null);
const [integrationColors, setIntegrationColors] = useState<Record<string, string>>({}); const [integrationColors, setIntegrationColors] = useState<Record<string, string>>({});
const [testingId, setTestingId] = useState<number | null>(null); const [testingId, setTestingId] = useState<number | null>(null);
@@ -184,6 +186,7 @@ function IntegrationsList({ user, tabs }: { user: CurrentUser; tabs: ReactNode }
</div> </div>
{tabs} {tabs}
<ErrorNotice error={error} /> <ErrorNotice error={error} />
{inspecting && <IntegrationInspector integration={inspecting} onClose={() => setInspecting(null)} />}
{editingIntegration ? ( {editingIntegration ? (
<IntegrationEditForm <IntegrationEditForm
@@ -293,6 +296,11 @@ function IntegrationsList({ user, tabs }: { user: CurrentUser; tabs: ReactNode }
<button className="btn btn-sm" onClick={() => testIntegration(i)} disabled={testingId === i.id}> <button className="btn btn-sm" onClick={() => testIntegration(i)} disabled={testingId === i.id}>
{testingId === i.id ? "Testing…" : "Test"} {testingId === i.id ? "Testing…" : "Test"}
</button> </button>
{i.enabled && INSPECTABLE.includes(i.type) && (
<button className="btn btn-sm" onClick={() => setInspecting(i)} title="Read it once now and see what each request got back — to check that what is shown is what is there">
Inspect
</button>
)}
<button className="btn btn-sm" onClick={() => setEditingIntegration(i)}> <button className="btn btn-sm" onClick={() => setEditingIntegration(i)}>
Edit Edit
</button> </button>
+3 -3
View File
@@ -8,7 +8,7 @@ import { ErrorNotice } from "../components/Notice";
import { formatBytes } from "./settings/StorageSettings"; import { formatBytes } from "./settings/StorageSettings";
import { readableError } from "../utils/errors"; import { readableError } from "../utils/errors";
const KIND_NAMES: Record<ArrApp["kind"], string> = { sonarr: "Sonarr", radarr: "Radarr", prowlarr: "Prowlarr" }; const KIND_NAMES: Record<ArrApp["kind"], string> = { sonarr: "Sonarr", radarr: "Radarr", prowlarr: "Prowlarr", lidarr: "Lidarr", readarr: "Readarr" };
const QUEUE_SHOWN = 25; const QUEUE_SHOWN = 25;
function Badge({ a }: { a: ArrApp }) { function Badge({ a }: { a: ArrApp }) {
@@ -214,7 +214,7 @@ export default function Media() {
<> <>
<h2 className="page-title">Media managers</h2> <h2 className="page-title">Media managers</h2>
<p className="text-secondary"> <p className="text-secondary">
Sonarr, Radarr and Prowlarr, read every few minutes (see <Link to="/integrations">Integrations</Link>) — nothing is searched, grabbed or changed. <strong>What an application says is wrong with itself</strong>, <strong>downloads stuck for two hours</strong> and{" "} Sonarr, Radarr, Prowlarr, Lidarr and Readarr, read every few minutes (see <Link to="/integrations">Integrations</Link>) — nothing is searched, grabbed or changed. <strong>What an application says is wrong with itself</strong>, <strong>downloads stuck for two hours</strong> and{" "}
<strong>a disk nearly full</strong> are alerts — see <Link to="/alerts">Alerts</Link>. <strong>a disk nearly full</strong> are alerts — see <Link to="/alerts">Alerts</Link>.
</p> </p>
<ErrorNotice error={error} /> <ErrorNotice error={error} />
@@ -222,7 +222,7 @@ export default function Media() {
{data && data.length === 0 && ( {data && data.length === 0 && (
<div className="card"> <div className="card">
<div className="card-body"> <div className="card-body">
<EmptyState title="No Sonarr, Radarr or Prowlarr connected" hint="Add one under Integrations with its address and API key (Settings → General → Security → API Key in the application)." /> <EmptyState title="No media manager connected" hint="Add one under Integrations with its address and API key (Settings → General → Security → API Key in the application)." />
</div> </div>
</div> </div>
)} )}
+187
View File
@@ -0,0 +1,187 @@
import { useEffect, useState } from "react";
import { Link } from "react-router-dom";
import { api, type AppVersions, type CurrentUser } from "../api/client";
import EmptyState from "../components/EmptyState";
import TimeAgo from "../components/TimeAgo";
import { BlockSkeleton } from "../components/Skeleton";
import { ErrorNotice } from "../components/Notice";
import { readableError } from "../utils/errors";
import { toast } from "../utils/toast";
type Row = AppVersions["rows"][number];
const LAG: Record<Row["lag"], { text: string; badge: string }> = {
same: { text: "Up to date", badge: "bg-green-lt text-green" },
patch: { text: "A patch behind", badge: "bg-blue-lt text-blue" },
minor: { text: "A minor update behind", badge: "bg-yellow-lt text-yellow" },
major: { text: "A major version behind", badge: "bg-red-lt text-red" },
ahead: { text: "Ahead of the newest release", badge: "bg-secondary-lt text-secondary" },
unknown: { text: "Not compared", badge: "bg-secondary-lt text-secondary" },
};
const ORDER: Record<Row["lag"], number> = { major: 0, minor: 1, patch: 2, unknown: 3, ahead: 4, same: 5 };
/** What each connected application runs, against the newest release of it — the lookup is switched on here, and is the only thing this app asks of an address of its own choosing. */
export default function Versions({ user }: { user: CurrentUser }) {
const isAdmin = user.role === "admin";
const [data, setData] = useState<AppVersions | null>(null);
const [error, setError] = useState<string | null>(null);
const [busy, setBusy] = useState(false);
const [days, setDays] = useState("0");
const adopt = (d: AppVersions) => {
setData(d);
setDays(String(d.alertAfterDays));
setError(null);
};
useEffect(() => {
api.versions
.get()
.then(adopt)
.catch((err) => setError(readableError(err)));
}, []);
async function save(enabled: boolean, alertAfterDays: number) {
setBusy(true);
try {
adopt(await api.versions.save({ enabled, alertAfterDays }));
toast.success(enabled ? "Saved." : "Switched off.");
} catch (err) {
toast.error(readableError(err));
} finally {
setBusy(false);
}
}
async function refresh() {
setBusy(true);
try {
const r = await api.versions.refresh();
adopt(r);
toast[r.result.failed > 0 ? "error" : "success"](r.result.failed > 0 ? `${r.result.failed} could not be looked up — see the list.` : `Looked up ${r.result.checked}.`);
} catch (err) {
toast.error(readableError(err));
} finally {
setBusy(false);
}
}
const rows = [...(data?.rows ?? [])].sort((a, b) => ORDER[a.lag] - ORDER[b.lag] || a.label.localeCompare(b.label));
const behind = rows.filter((r) => r.lag === "patch" || r.lag === "minor" || r.lag === "major").length;
return (
<>
<h2 className="page-title">App versions</h2>
<p className="text-secondary">
The version each connected application runs — as it was last read (see <Link to="/integrations">Integrations</Link>) — against the newest release of it. It looks the newest ones up on GitHub once a day; <strong>that is the only thing this app asks of an address that is not yours</strong>, so it is switched
off until you switch it on. Proxmox, Synology and the rest, which have no release to compare with, are not listed.
</p>
<ErrorNotice error={error} />
{!data && !error && <BlockSkeleton lines={4} />}
{data && (
<>
<div className="card mb-3">
<div className="card-body">
<label className="form-check mb-2">
<input type="checkbox" className="form-check-input" checked={data.enabled} disabled={!isAdmin || busy} onChange={(e) => void save(e.target.checked, Number(days) || 0)} />
<span className="form-check-label">
Look up the newest releases
<span className="form-hint d-block">
Asks <code>api.github.com</code> for the newest release of each application you have connected — its name, nothing about you or your systems. An administrator can switch this.
</span>
</span>
</label>
<div className="d-flex flex-wrap align-items-center gap-2">
<label htmlFor="alert-after">Warn when a newer release has been out for</label>
<input
id="alert-after"
className="form-control form-control-sm"
style={{ width: 80 }}
inputMode="numeric"
value={days}
disabled={!isAdmin || busy || !data.enabled}
onChange={(e) => setDays(e.target.value.replace(/[^0-9]/g, ""))}
onBlur={() => Number(days) !== data.alertAfterDays && void save(data.enabled, Math.min(365, Number(days) || 0))}
/>
<span>days (0 = never)</span>
{isAdmin && data.enabled && (
<button className="btn btn-sm ms-auto" disabled={busy} onClick={() => void refresh()}>
{busy ? "Looking…" : "Look up now"}
</button>
)}
</div>
<div className="text-secondary small mt-2">
{data.enabled ? (data.lastCheckedAt ? <>Last looked up <TimeAgo value={data.lastCheckedAt} />.</> : "Not looked up yet — it is done a few minutes after it is switched on.") : "Switched off."} A warning is raised under <Link to="/alerts">Alerts</Link> only with a number of days above 0.
</div>
</div>
</div>
{rows.length === 0 ? (
<div className="card">
<div className="card-body">
<EmptyState title="No versions known yet" hint="An application's version is known once its integration has been read: Sonarr, Radarr, Prowlarr, Lidarr, Readarr, AdGuard Home, Grafana, Jellyfin, Immich, Paperless-ngx, Nextcloud, NetBox, Authentik and Prometheus are compared." />
</div>
</div>
) : (
<div className="card">
<div className="card-header">
<h3 className="card-title">
{rows.length} {rows.length === 1 ? "application" : "applications"}
{data.enabled ? `, ${behind === 0 ? "none behind" : `${behind} behind`}` : ""}
</h3>
</div>
<div className="table-responsive">
<table className="table table-vcenter card-table">
<thead>
<tr>
<th>Application</th>
<th>Running</th>
<th>Newest release</th>
<th>Compared</th>
</tr>
</thead>
<tbody>
{rows.map((r) => (
<tr key={r.integrationId}>
<td>
<strong>{r.label}</strong>
<div className="text-secondary small">{r.name}</div>
</td>
<td>
<code>{r.running}</code>
</td>
<td>
{r.latest ? (
<>
{r.latest.url ? (
<a href={r.latest.url} target="_blank" rel="noopener noreferrer">
<code>{r.latest.tag}</code>
</a>
) : (
<code>{r.latest.tag}</code>
)}
{r.latest.publishedAt && (
<div className="text-secondary small">
out <TimeAgo value={r.latest.publishedAt} />
</div>
)}
</>
) : (
<span className="text-secondary">{data.enabled ? "not known yet" : "—"}</span>
)}
{r.lookupError && <div className="text-yellow small">{r.lookupError}</div>}
</td>
<td>
<span className={`badge ${LAG[r.lag].badge}`}>{LAG[r.lag].text}</span>
{r.daysOut !== null && <div className="text-secondary small">{r.daysOut === 0 ? "a newer one came out today" : r.daysOut === 1 ? "a newer one has been out a day" : `a newer one has been out ${r.daysOut} days`}</div>}
</td>
</tr>
))}
</tbody>
</table>
</div>
</div>
)}
</>
)}
</>
);
}
+58
View File
@@ -0,0 +1,58 @@
import type { AlertCategory, AlertSeverity } from "../api/client";
import { kindWords } from "./alertKinds";
export type GroupBy = "none" | "source" | "category" | "kind";
export const GROUP_BY_LABELS: Record<GroupBy, string> = { none: "Not grouped", source: "Group by source", category: "Group by kind", kind: "Group by type of alert" };
/** The line above a group of alerts on the Alerts page. */
export interface AlertGroup {
isHeader: true;
id: string;
label: string;
count: number;
critical: number;
warning: number;
info: number;
silenced: number;
/** The alerts in the group that can be picked (those not held back). */
pickable: string[];
folded: boolean;
}
interface Groupable {
id: string;
severity: AlertSeverity;
silenced: boolean;
source: string;
category: AlertCategory;
kind: string;
}
const SEVERITY_RANK: Record<AlertSeverity, number> = { critical: 0, warning: 1, info: 2 };
/**
* Pure: the alerts in groups — each group a header (how many, how serious, how many are held back) and then its alerts, unless the group is folded. The worst group comes first (then the
* biggest, then by name); inside a group the alerts keep the order they were given in. `categoryLabel` says what a category is called.
*/
export function groupAlerts<T extends Groupable>(alerts: T[], by: GroupBy, folded: ReadonlySet<string>, categoryLabel: (c: AlertCategory) => string): (T | AlertGroup)[] {
if (by === "none") return alerts;
const keyOf = (a: T) => (by === "source" ? a.source : by === "category" ? categoryLabel(a.category) : kindWords(a.kind));
const groups = new Map<string, T[]>();
for (const a of alerts) groups.set(keyOf(a), [...(groups.get(keyOf(a)) ?? []), a]);
const stat = (items: T[]) => {
const active = items.filter((a) => !a.silenced);
return { critical: active.filter((a) => a.severity === "critical").length, warning: active.filter((a) => a.severity === "warning").length, info: active.filter((a) => a.severity === "info").length };
};
const worst = (items: T[]) => Math.min(3, ...items.filter((a) => !a.silenced).map((a) => SEVERITY_RANK[a.severity]));
const out: (T | AlertGroup)[] = [];
for (const [label, items] of [...groups.entries()].sort((a, b) => worst(a[1]) - worst(b[1]) || b[1].length - a[1].length || a[0].localeCompare(b[0]))) {
const id = `${by}:${label}`;
const isFolded = folded.has(id);
out.push({ isHeader: true, id, label, count: items.length, ...stat(items), silenced: items.filter((a) => a.silenced).length, pickable: items.filter((a) => !a.silenced).map((a) => a.id), folded: isFolded });
if (!isFolded) out.push(...items);
}
return out;
}
export const isGroup = (x: unknown): x is AlertGroup => typeof x === "object" && x !== null && (x as { isHeader?: boolean }).isHeader === true;
+36
View File
@@ -0,0 +1,36 @@
/** How a kind of alert reads in a sentence; any kind not listed is its own name with the dashes taken out. */
const KIND_WORDS: Record<string, string> = {
offline: "a server that stopped reporting",
disk: "a disk filling up",
"failed-units": "failed services",
smart: "a disk reporting trouble",
zfs: "a ZFS pool in trouble",
"docker-vuln": "critical vulnerabilities in Docker images",
"docker-vuln-high": "many high vulnerabilities in Docker images",
"container-restarting": "a container that keeps restarting",
"container-unhealthy": "an unhealthy container",
"container-stopped": "a container that stopped",
"hetzner-off": "a Hetzner server that is off",
"hetzner-traffic": "a Hetzner traffic quota nearly used",
"hetzner-forecast": "a Hetzner traffic quota the month will pass",
"hetzner-backup": "a Hetzner backup that is late",
"hetzner-lb": "an unhealthy Hetzner load balancer",
"hetzner-cert": "a Hetzner certificate about to end",
"hetzner-fip": "a Hetzner floating IP pointing at nothing",
"fragnet-backup": "a game server backup that is late",
"fragnet-disk": "a game server disk filling up",
"fragnet-memory": "a game server's memory running out",
"sonarr-health": "Sonarr reporting a problem",
"sonarr-queue": "stuck downloads in Sonarr",
"radarr-health": "Radarr reporting a problem",
"radarr-queue": "stuck downloads in Radarr",
"lidarr-health": "Lidarr reporting a problem",
"lidarr-queue": "stuck downloads in Lidarr",
"readarr-health": "Readarr reporting a problem",
"readarr-queue": "stuck downloads in Readarr",
"prowlarr-health": "Prowlarr reporting a problem",
"speedtest-slow": "a slow internet line",
"speedtest-failing": "failing speed tests",
};
export const kindWords = (kind: string): string => KIND_WORDS[kind] ?? `${kind.replace(/[-_]+/g, " ")} alerts`;
+8 -1
View File
@@ -32,6 +32,8 @@ export const TYPE_LABELS: Record<IntegrationType, string> = {
prowlarr: "Prowlarr", prowlarr: "Prowlarr",
adguard: "AdGuard Home", adguard: "AdGuard Home",
grafana: "Grafana", grafana: "Grafana",
lidarr: "Lidarr",
readarr: "Readarr",
}; };
/** The page each kind of integration gets once it is connected. */ /** The page each kind of integration gets once it is connected. */
@@ -66,18 +68,23 @@ export const TYPE_PAGES: Record<IntegrationType, string> = {
prowlarr: "/media", prowlarr: "/media",
adguard: "/adguard", adguard: "/adguard",
grafana: "/grafana", grafana: "/grafana",
lidarr: "/media",
readarr: "/media",
}; };
/** The kinds of integration grouped the way the sidebar groups their pages, in the same order. */ /** The kinds of integration grouped the way the sidebar groups their pages, in the same order. */
export const TYPE_GROUPS: { id: string; label: string; types: IntegrationType[] }[] = [ export const TYPE_GROUPS: { id: string; label: string; types: IntegrationType[] }[] = [
{ id: "infrastructure", label: "Infrastructure", types: ["proxmox", "synology", "dockhand", "hetzner", "fragnet", "tailscale"] }, { id: "infrastructure", label: "Infrastructure", types: ["proxmox", "synology", "dockhand", "hetzner", "fragnet", "tailscale"] },
{ id: "apps", label: "Apps", types: ["homeassistant", "jellyfin", "sonarr", "radarr", "prowlarr", "immich", "paperless", "nextcloud", "authentik"] }, { id: "apps", label: "Apps", types: ["homeassistant", "jellyfin", "sonarr", "radarr", "lidarr", "readarr", "prowlarr", "immich", "paperless", "nextcloud", "authentik"] },
{ id: "backups", label: "Backups", types: ["pbs"] }, { id: "backups", label: "Backups", types: ["pbs"] },
{ id: "network", label: "Network", types: ["unifi", "npm", "phpipam", "netbox", "adguard"] }, { id: "network", label: "Network", types: ["unifi", "npm", "phpipam", "netbox", "adguard"] },
{ id: "monitoring", label: "Monitoring", types: ["uptimekuma", "prometheus", "grafana", "ups", "speedtest"] }, { id: "monitoring", label: "Monitoring", types: ["uptimekuma", "prometheus", "grafana", "ups", "speedtest"] },
{ id: "automation", label: "Automation", types: ["semaphore", "gitea", "osticket", "n8n"] }, { id: "automation", label: "Automation", types: ["semaphore", "gitea", "osticket", "n8n"] },
]; ];
/** The kinds whose reader can be inspected (Settings → Integrations → Check what can be read): the ones that read through the shared HTTP helper and have a reader to run. */
export const INSPECTABLE: IntegrationType[] = ["authentik", "immich", "jellyfin", "netbox", "nextcloud", "paperless", "prometheus", "hetzner", "speedtest", "fragnet", "sonarr", "radarr", "prowlarr", "lidarr", "readarr", "adguard", "grafana"];
/** The group a kind belongs to; a kind nobody has placed yet falls in "Other" rather than vanishing. */ /** The group a kind belongs to; a kind nobody has placed yet falls in "Other" rather than vanishing. */
export function groupOfType(type: IntegrationType): { id: string; label: string } { export function groupOfType(type: IntegrationType): { id: string; label: string } {
return TYPE_GROUPS.find((g) => g.types.includes(type)) ?? { id: "other", label: "Other" }; return TYPE_GROUPS.find((g) => g.types.includes(type)) ?? { id: "other", label: "Other" };