Three read-only integrations, each with a monitoring page, alerts, maintenance
silencing and held-when-unreadable handling.
Prometheus: scrape targets that stop answering (5 minutes, one alert per job)
and alerts that Prometheus's own rules fire (critical when labelled so).
Immich: disk under the library filling, failed background jobs, counts.
Paperless-ngx: worker/database/Redis in error, documents that failed to
import, index and sanity check, disk. Immich and Paperless also in /metrics.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
New Authentik integration (read-only, every 5 minutes): version, workers,
outposts and their health, certificates and API tokens that expire, failed
sign-ins and configuration errors. Monitoring -> Authentik page.
Alerts: an outpost that serves apps has gone quiet (critical), certificate or
token about to run out (critical close to it), no worker, failed sign-ins,
configuration errors. Held by maintenance, held when unreadable, in /metrics.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
New Jellyfin integration (read-only, every 2 minutes): version, restart wanted,
streams and transcoding, failed scheduled tasks, item counts, failed sign-ins.
Monitoring -> Jellyfin page with a peak-streams history, three alerts, /metrics.
When Jellyfin runs on a managed server, an update run asks first if people are
streaming (409 "watching" unless startAnyway), and a patch plan waits up to two
hours for them before updating that server.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Duplicati reports each operation to a secret address (its own send-http-url option), so
nothing calls Duplicati and no password for it is kept. Each machine is a source; each job
is learned by name and shows its last result, duration, size, changes and a history.
A failed run is a warning (critical from the second in a row); one that has not succeeded
for longer than expected (per job, 2 days by default) is a warning, critical at three
times that. Maintenance on the machine's server silences them; /metrics series added.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
With Dockhand connected, an image the app recognises (about 20 with a health endpoint of
their own, plus a few dozen web UIs) is offered as an endpoint check on that app's own
health address, with what a good answer looks like set up as a value check: Immich's ping
says pong, Nextcloud's status.php is not in maintenance, Gitea's health says pass, and so
on. Asked on the Docker host's address at the port the container publishes. Container
ports are now read from Dockhand's list.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
From Dockhand, on the health pass: a container that keeps restarting (critical, with
its exit code), one whose own health check fails, one that stopped unexpectedly, one
created and never started for an hour, and a Docker host Dockhand cannot reach. A stop
is only news when it was seen happen, so stacks switched off long ago stay quiet.
A Leave alone button on the Docker page and a list in Settings; on by default.
Also fixes a date-dependent storage-forecast test that pinned its clock while the code
under test reads the real one.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
For a one-person homelab, "on call" is when you can be reached. Settings ->
Notifications -> When -> Away holds notifications while you are away (a banner on
every page says so) and sends one digest when it ends, at its time or when you say
you are back. Critical alerts can still come through. Quiet hours get the same
"still send critical alerts" option. Escalation steps marked to send in quiet hours
are sent while away too. What is held for one reason is not released while the
other still holds it.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Monitoring -> Service levels: a target (99.9 %) on an endpoint check, a server or an
integration, measured over calendar months. Shows the figure, downtime used, the
error budget left, and how the last six months went. 80 % of the budget used is a
warning and an overspent budget a critical alert. Endpoints are measured day by day,
servers and integrations to the minute from the audit log.
The report page gains Quarter and Year (same figures, with a Service levels section),
and two settings send the quarterly and annual report after the month that ends them.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each server's CPU load and memory use get a baseline for every hour of the week
(median and robust spread over up to ten weeks, four needed). A clearly high last
hour that never dropped back toward normal is a warning; everything else stays
quiet. Server History -> Trends draws a usual line and band. Setting under
Notifications, on by default.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
New UPS integration (NUT or SNMP v2c, read only). Monitoring -> UPS shows charge,
runtime, load and mains voltage with history and the times mains power was lost.
Mains lost and a nearly empty battery are critical alerts; battery to be replaced
and a heavy load are warnings. The UPS is a node in the dependency map and counts
as down while on battery, so servers that go offline with it are folded under its
alert; the page lists what is on it in the order to shut it down. /metrics series
and docs added.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A web check can read a value out of a JSON answer (power.now, state,
data.result[0].value[1]) and judge it: below, above, equal to, contains, is there.
The value read is shown on the Endpoints page, charted under History, exported as
homelab_endpoint_value and in the read API. A check can also send one header (a
token), stored encrypted, never returned, and not forwarded to another host on a
redirect.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Settings -> Data -> Storage measures the database, backups and sessions and every
table (size, rows, growth, how long it is kept, how far back it reaches), and flags
histories that are not being trimmed or have no limit.
Event webhooks now keep a delivery log (last 100 per webhook, 14 days): what was
sent, how it went, the message itself, and Send again, which resends it exactly as it
was (same id and signature). Resends are audited and in the Change log.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- A switch (off by default) under Settings -> Notifications -> Who hears what
- ntfy gets buttons (three for one alert, one link for several); phone push gets two buttons handled by the service worker; Gotify, email and the chat webhook get the link as text; escalation messages carry them too
- Each link is a signed token for the alerts in that one message only, good for 24 hours, for three actions; opening it changes nothing, only a POST acts; everything is audited as "from a notification link"
- Docs updated
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- A card on the incident page, and GET /api/incidents/:id/before and /api/v1/incidents/:id/before: what the app knows happened in the 30 min to 24 h before an alert began (audit log, maintenance windows, update runs and patch plans, other alerts, a step in the server's own figures), scored by how related, what kind and how close
- A host going down is shown once; Likely, Possible and Background; at most ten
- Docs and roles updated
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- New docs/ folder: SSO, API, Prometheus and Grafana, webhooks, heartbeats and calendar, how monitoring works, and an index; the old root API.md is split across them
- Read API: alerts carry snoozedUntil
- DATABASE.md: table and migration counts, indexes, relations, integration config, settings and internal flags, delete rules, retention
- ROLES.md: the sidebar table rebuilt from the menu
- NOTIFICATIONS.md, INTEGRATIONS.md, BACKUP.md, AGENT_UPDATE.md: what had drifted
- README: links to the guides; accounts cannot be deleted in the app
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>