Move Notifications back to Settings and link to it from Alert settings
Alert settings keeps Alert rules, Service levels and Alert tickets; administrators get a link to Settings -> Notifications. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
4269213275
commit
ccb863e7c3
32 files changed
+152
-150
No files matched your search
@@ -187,7 +187,7 @@ All modules from the original plan are built:
|
||||
Backup page's job and run lists, the comparison table) have an Export CSV
|
||||
too, which writes the table as it is on screen — times as exact timestamps,
|
||||
action buttons left out.
|
||||
- **Alert settings** (Monitoring → Alert settings) — everything that decides what is an alert and who hears of it, on one page with four tabs: *Alert rules*, *Service levels*, *Notifications* and *Alert tickets* (`/alert-settings?view=rules|levels|notifications|tickets`). Everyone signed in sees the first two; the last two are administrators' (they used to be under Settings, and `/alert-rules`, `/service-levels` and `/settings/notifications` still lead to the right tab). Maintenance stays on its own page.
|
||||
- **Alert settings** (Monitoring → Alert settings) — what decides what is an alert, on one page with three tabs: *Alert rules*, *Service levels* and *Alert tickets* (`/alert-settings?view=rules|levels|tickets`). Everyone signed in sees the first two; *Alert tickets* is administrators' (it used to be under Settings, and `/alert-rules`, `/service-levels` and `/settings/alert-tickets` still lead to the right tab). Notifications — channels, what is sent, who hears what, quiet hours — serve more than alerts, so they stay under **Settings → Notifications**, with a link to them at the top of this page for administrators. Maintenance stays on its own page.
|
||||
- **Settings** (admin-only) — notification channels (Gotify, ntfy, SMTP,
|
||||
generic webhook, and push to phones and browsers) with per-channel test buttons, per-event toggles (DNS
|
||||
record added/updated/deleted, daily secret-expiry reminder and daily
|
||||
@@ -222,7 +222,7 @@ Server and storage health is watched every 15 minutes: a server whose agent
|
||||
stops reporting, a server disk / Proxmox storage / Synology volume passing a
|
||||
usage threshold, and a Synology volume or disk that's degraded or failing each
|
||||
raise one notification when the problem starts and one when it clears (both
|
||||
thresholds are set under Alert settings → Notifications). Active problems are
|
||||
thresholds are set under Settings → Notifications). Active problems are
|
||||
remembered across restarts, so a rebuild doesn't re-alert them.
|
||||
|
||||
**Tags** — servers can be tagged (prod, media, rack-1, …) from their detail
|
||||
@@ -306,7 +306,7 @@ like `lab.example.se` resolves to the `example.se` registration that actually
|
||||
expires); others can be added by hand. Each is looked up daily over RDAP where
|
||||
the TLD offers it, and otherwise over WHOIS via IANA's referral — which is what
|
||||
makes `.se`, `.nu` and `.io` work, since those aren't in the RDAP bootstrap.
|
||||
You're reminded daily from N days before expiry (Alert settings → Notifications,
|
||||
You're reminded daily from N days before expiry (Settings → Notifications,
|
||||
default 30) until it's renewed, and told when an expiry date couldn't be
|
||||
refreshed for several days. Registries that don't publish an expiry (`.de`,
|
||||
`.eu`) can be tracked but have no date to warn about. Private zones (`.lan`,
|
||||
@@ -340,7 +340,7 @@ send the notifications — so the two can't disagree — but ignores the on/off
|
||||
toggles, since it's for looking at rather than being interrupted by. Problems
|
||||
under a maintenance window stay listed, marked silenced and counted separately.
|
||||
It also looks ahead: from an agent server's own history (see its Trends card), a
|
||||
disk that is on course to be full within 14 days (Alert settings → Notifications → "Warn
|
||||
disk that is on course to be full within 14 days (Settings → Notifications → "Warn
|
||||
when a disk will fill within"; 0 turns it off) is flagged with its rate of growth
|
||||
and the days left — critical when that is three days or less. It needs at least
|
||||
three days of steady growth to say anything, skips a disk that jumped once or
|
||||
@@ -456,7 +456,7 @@ alerts always stand alone.
|
||||
|
||||
**Restore tests** (on the Backup coverage page). A backup nobody has restored is a hope, not a backup: when you try one, record it — which guest (or anything else: a
|
||||
share, a database), the day, how it went (restored fine, partly, failed, or *no need to test this one*) and a note. Each guest's newest test sits beside its backups.
|
||||
A **failed** test is always an alert (critical) until a newer one passes. Under Alert settings → Notifications → "A restore should have been tested within" you can also
|
||||
A **failed** test is always an alert (critical) until a newer one passes. Under Settings → Notifications → "A restore should have been tested within" you can also
|
||||
ask to be reminded: a guest that has backups but no passing test within that many days (or ever) becomes a warning, and so does anything else you chose to track;
|
||||
0, the default, leaves the reminder off. A guest marked *no need to test* never is. Operators record tests, administrators can remove a mistaken one; the history is
|
||||
kept, `/metrics` has each subject's age and result, and everything is in the change log.
|
||||
@@ -475,7 +475,7 @@ so it is folded only when every enabled instance of that kind is explained. A se
|
||||
is offline isn't announced; if it is still failing once the server is back, the next failed call alerts. (DNS providers raise no
|
||||
alerts of their own, so they aren't part of this.)
|
||||
|
||||
**Who hears what** (Alert settings → Notifications → Who hears what). *Routing rules* send a kind of notification, of at least a severity, about servers with
|
||||
**Who hears what** (Settings → Notifications → Who hears what). *Routing rules* send a kind of notification, of at least a severity, about servers with
|
||||
certain tags, to the channels you choose — the first matching rule decides, none matching means every channel that is on, and a rule with no channels tells
|
||||
nobody. Each phone or browser can also say it only wants warnings or critical alerts, and only some kinds. *Escalation* tells again, through louder channels
|
||||
and perhaps an extra email address, about an alert nobody has acknowledged after a time you set (up to three steps, each once; an acknowledgement stops the
|
||||
@@ -774,9 +774,9 @@ pace so far would end the month), and how each of the last six months went. When
|
||||
ended, how long they took, the longest, by kind, and which are still open), **updates** (update runs and patch plan runs, how many updates were installed, by server), **restore tests** recorded in the
|
||||
month and the **backups as they are now**, and what was **renewed** (a certificate or a domain whose expiry moved clearly later — noticed when the app re-reads the date). The month is on the clock of the notification time zone;
|
||||
the current month is "so far". It can only say what was recorded: endpoint availability starts from the last two weeks of results when the daily figures were added, renewals from when the app began noticing them,
|
||||
and the backups are a snapshot, not a history. The same page also gives a **quarter** (Q3 2026) or a **year** — the same figures over longer, with a **Service levels** section saying how each target did against the downtime it allowed over the whole stretch — chosen with the *Month · Quarter · Year* buttons. Alert settings → Notifications → *Monthly report* sends last month's report on a day (1–28) and time of your choice (off until switched on), with *Preview* and *Send now*; two more boxes add the **quarterly** report after the month that ends a quarter (March, June, September, December) and the **annual** one after December.
|
||||
and the backups are a snapshot, not a history. The same page also gives a **quarter** (Q3 2026) or a **year** — the same figures over longer, with a **Service levels** section saying how each target did against the downtime it allowed over the whole stretch — chosen with the *Month · Quarter · Year* buttons. Settings → Notifications → *Monthly report* sends last month's report on a day (1–28) and time of your choice (off until switched on), with *Preview* and *Send now*; two more boxes add the **quarterly** report after the month that ends a quarter (March, June, September, December) and the **annual** one after December.
|
||||
|
||||
**Weekly digest** (Alert settings → Notifications, off until switched on) sends one message a
|
||||
**Weekly digest** (Settings → Notifications, off until switched on) sends one message a
|
||||
week through your notification channels: what is wrong now, what was down and for how
|
||||
long, what happened, disks heading for full, things about to expire, updates waiting
|
||||
and maintenance coming up — each topic only if it has something to say, and "a quiet
|
||||
@@ -891,18 +891,18 @@ minimum in both themes. Motion is turned down for those who ask the system to.
|
||||
**New ports.** When an agent first reports what a server is listening on, that is taken as normal (the baseline). After that, a **TCP port that begins listening**, that is reachable from outside the machine, and that
|
||||
nobody has noted is a warning on the Alerts page (kind *Monitoring*) and a notification, with what is listening on it: "nas: a new port is listening — 8080/tcp (node)". UDP (clients and discovery open them all the time), ports from 49152 up
|
||||
(where programs pick a free one) and anything bound to localhost only are left out. The port's row on the server's **Ports** card says *New* and has **Mark as expected**, which notes it and ends the alert; so does a label or comment you write on
|
||||
it yourself. It also ends when the port stops listening, and maintenance on the server holds it back like its other alerts. It can be switched off under Alert settings → Notifications (*Servers and disks*); servers get their baseline from their next report after an upgrade, so nothing alerts for what was already there.
|
||||
it yourself. It also ends when the port stops listening, and maintenance on the server holds it back like its other alerts. It can be switched off under Settings → Notifications (*Servers and disks*); servers get their baseline from their next report after an upgrade, so nothing alerts for what was already there.
|
||||
|
||||
**Broken containers.** With Dockhand connected, the app now says when a **container is broken**, not only when an image has an update: one that **keeps restarting** (critical, with its exit code), one whose **own health check fails**, one that **stopped when nobody meant it to**, one **created and
|
||||
never started**, and a **Docker host Dockhand can no longer reach**. A stop is only news when it was seen happen — a stack switched off weeks ago stays quiet — and an alert stays until the container runs again. The Docker page marks each problem and has a **Leave alone** button;
|
||||
the same list is under Alert settings → Notifications (names, `host/name`, `*`). See docs/MONITORING.md.
|
||||
the same list is under Settings → Notifications (names, `host/name`, `*`). See docs/MONITORING.md.
|
||||
|
||||
**What is normal for the time.** For each server's CPU load and memory use the app learns what is usual for **the same hour of the same weekday** — "a Tuesday at 03:00" — from that server's own history (the median of that hour's
|
||||
averages over up to ten weeks, with a robust spread; it needs **four weeks** of that hour before it says anything, and the hours are those of the notification time zone). A server's *History → Trends* then draws a grey
|
||||
**usual** line with a band around it behind the real one, and says "usually about 8 % now". It is **quiet on purpose**: only a *high* reading is ever a warning, and only when the last hour averaged at least 50 % CPU (70 % memory), at least
|
||||
30 points (15) above the usual and four spreads above it, and **never dropped back toward normal during that hour** — so a spike, a job that always runs a bit late, or a server that is simply always busy at that hour is not an alert. The
|
||||
warning ("wiki-host: CPU load is unusually high for a Tuesday at 03:00 — it averaged 79 % over the last hour, where it is normally about 8 % (from 6 weeks of this hour)") is an ordinary *Monitoring* alert and notification: held back by
|
||||
maintenance on the server, snoozed, flapping-damped and routed like the others, and left to the server's offline alert when it has stopped reporting. Switch it off under Alert settings → Notifications (*Servers and disks*).
|
||||
maintenance on the server, snoozed, flapping-damped and routed like the others, and left to the server's offline alert when it has stopped reporting. Switch it off under Settings → Notifications (*Servers and disks*).
|
||||
|
||||
**A year of history.** The figures behind the trend lines and the answer-time charts used to reach back two weeks or five. A server's load, memory and disks are still kept as a sample every ten minutes for 35 days, and now also as an hour's and a day's average, lowest and highest (hours for four
|
||||
months, days for over two years); a check's answers are kept as hours for four months and, as before, days for years. A chart asks for 24 hours, 7, 30 or 90 days or a year and is drawn from whichever is fine enough. A disk that two weeks call flat is looked at again over ninety days, so a slow, steady climb is forecast too
|
||||
@@ -918,7 +918,7 @@ servers, alerts, incidents, inventory, network equipment, endpoint checks and ma
|
||||
gateways: whether each is up (and since when it was not), IP and MAC address, uptime, firmware (with *update available*), the device it is connected through, how many things depend on it, and — on the *Networks & Wi-Fi* tab —
|
||||
its networks (VLAN, address range) and Wi-Fi names (security, which network). It is read every five minutes and on **Read now**; nothing is ever written to the controller, and Wi-Fi passphrases are never read. The devices
|
||||
join the **dependency map** (Map → Dependencies): what each is connected through becomes a dependency by itself, you can write others ("the NAS needs the rack switch"), and when a device is down, what needs it is folded under
|
||||
its alert — "Hallway AP has been offline for 45 min — also takes down pi". A device offline for longer than a set time (Alert settings → Notifications, 10 minutes) is a critical *Server down* alert, can be silenced with maintenance
|
||||
its alert — "Hallway AP has been offline for 45 min — also takes down pi". A device offline for longer than a set time (Settings → Notifications, 10 minutes) is a critical *Server down* alert, can be silenced with maintenance
|
||||
(also in the Maintenance page and the command palette), and can have runbooks. Not yet tried against a real controller — see INTEGRATIONS.md.
|
||||
|
||||
**Which switch a server is plugged into.** Every read also asks the controller what is connected, and a client whose IP address is exactly one of a server's own (and only one server's) is taken to be that server: the server is then
|
||||
@@ -933,7 +933,7 @@ hours (maintenance), is handed to you or is still going is counted in the badge
|
||||
**A server's page in sections.** Overview (links, hardware, identity, runbooks — fold a card by clicking its header, and *Arrange cards* puts them in the order you like, for every server), Health, Network (what it is connected through, ports), History (trends, activity), Tasks (scheduled tasks, run a playbook) and Settings (Proxmox link, its own alert limits),
|
||||
switched by `?view=`, so a section can be linked to: an alert about a failing service opens the Health section, a new-port alert the Network section.
|
||||
|
||||
**Away mode, and critical alerts through the night.** For a one-person homelab, "on call" is about *when you can be reached*. **Alert settings → Notifications → When → Away** is a one-off "I'm away until Sunday evening": notifications are held while it lasts
|
||||
**Away mode, and critical alerts through the night.** For a one-person homelab, "on call" is about *when you can be reached*. **Settings → Notifications → When → Away** is a one-off "I'm away until Sunday evening": notifications are held while it lasts
|
||||
(a banner on every page says so and counts them) and sent as **one digest** when it ends — at the time you gave, or when you press *I'm back* — so a week away is one message to read, not a buzzing phone. **Critical** alerts (a server down, a disk full, mains
|
||||
power lost) can still come through at once, which is the default; untick it for true silence. You can plan one ahead (start later) and add a note. **Quiet hours** got the same option, *Still send critical alerts*, off by default: a full disk at 03:00 is
|
||||
sent, the warnings wait for the morning. Escalation steps marked *send even during quiet hours* are sent while away too. Held messages only: the alerts themselves are still on the Alerts page, in webhooks and in `/metrics`. See NOTIFICATIONS.md.
|
||||
@@ -943,7 +943,7 @@ all find something. Removing a dependency, an admin link, a runbook, a hand-made
|
||||
what was on screen. Things that could not come back as they were — a secret, a check with its history, a share link or token, a DNS record, a hardware entry that has labels printed with its number — still ask first.
|
||||
|
||||
**Flapping alerts and the noise report.** An alert that is raised and clears again and again is more tiring than one that stays: each raise and each recovery is a message. When the same
|
||||
alert has been **raised three times within an hour** (Alert settings → Notifications → *Servers and disks* → *Call an alert flapping after*; 0 turns it off) it is called **flapping**: one message says
|
||||
alert has been **raised three times within an hour** (Settings → Notifications → *Servers and disks* → *Call an alert flapping after*; 0 turns it off) it is called **flapping**: one message says
|
||||
so — "raised 3 times within an hour" — and its further raise and recovery messages are held back. The alert itself is still on the Alerts page, with a *flapping* badge, and keeps counting for the badge in the menu and for /metrics.
|
||||
Once it has been calm (no raise) for an hour, one message says whether it is **still active** or **cleared**, and it is treated as an ordinary alert again. The webhooks that receive every event (Settings → API → event webhooks)
|
||||
still hear every raise and recovery; only the human-facing messages are held. The state is kept in the database, so a restart does not forget what is flapping, and the start and end of each spell are in the change log.
|
||||
@@ -960,7 +960,7 @@ what was there before (the earlier acknowledgement text, the earlier tags, the e
|
||||
|
||||
**Operating-system support.** The agents now report which operating system a server runs (Linux agent 7, Windows agent 6; re-run the install one-liner to update). A server's page shows it in a *System* row with a
|
||||
badge — *supported*, *ends in N days* or *support ended* — worked out from a table of end-of-life dates for Ubuntu, Debian, CentOS, RHEL and its rebuilds, Fedora, Alpine, openSUSE Leap, Proxmox VE and Windows
|
||||
(client and server). A server whose system is past its end, or ends within **Alert settings → Notifications → *Flag an operating system whose support ends within*** (90 days; 0 turns it off) is on the Alerts page in the *Updates* category with a link to the server — a warning once
|
||||
(client and server). A server whose system is past its end, or ends within **Settings → Notifications → *Flag an operating system whose support ends within*** (90 days; 0 turns it off) is on the Alerts page in the *Updates* category with a link to the server — a warning once
|
||||
support has ended, a note while it is only coming. The dates are a static table in the app, not looked up: a release it does not know is shown without a badge, never guessed at. The read API gives it as `operatingSystem`.
|
||||
|
||||
**Suggested dependencies.** Under **Map → Dependencies**, a *Suggested* card offers what the app can work out without being told: a service (Gitea, Semaphore, Uptime Kuma, phpIPAM, Home Assistant…) that is reached at a
|
||||
@@ -1006,7 +1006,7 @@ the disk's last report, so it is judged right now and has no time.
|
||||
**Forecasts for Proxmox storage and Synology volumes.** The disk forecast used to work only for servers with an agent. Each health pass now also writes down the figures it has just read from Proxmox (each node's
|
||||
storages and root filesystem, a shared storage once) and Synology (each volume) — one reading an hour, kept 400 days — and the same straight-line forecast is made from them: *full in about 40 days*. It is shown beside
|
||||
each storage on the **Proxmox** page and each volume on the **Synology** page (click it for the history chart), is an alert in the *Disk* category when it is within the forecast window
|
||||
(Alert settings → Notifications → *Warn when a disk will fill within*, 14 days — the same setting as for server disks) and not yet over the usual "disk nearly full" limit, is a line in the weekly digest, and is
|
||||
(Settings → Notifications → *Warn when a disk will fill within*, 14 days — the same setting as for server disks) and not yet over the usual "disk nearly full" limit, is a line in the weekly digest, and is
|
||||
`homelab_storage_days_to_full` in /metrics. It needs a few days of readings and growth that is really steady; until then the page says it is collecting history.
|
||||
|
||||
**Snooze.** An alert on the Alerts page has **Snooze…** (operators): an hour, four hours, tomorrow at 09:00, in three days, next Monday, or a time of your own, up to 90 days. A snoozed alert is hidden like one under
|
||||
@@ -1025,7 +1025,7 @@ switch it is plugged into; something that needs it; a change that touches everyt
|
||||
*close* it was. They are shown as *Likely*, *Possible* or *Background* (the last behind a button). It is a list of suspects, not a diagnosis: the app can say what happened and when, not why, and it cannot see a
|
||||
change made on the machine itself. Everyone signed in can see it; it is also `GET /api/v1/incidents/:id/before` in the read API.
|
||||
|
||||
**Buttons on notifications.** With **Alert settings → Notifications → Who hears what → Buttons on notifications** switched on (off by default), a notification can be dealt with from the message itself, without opening the app
|
||||
**Buttons on notifications.** With **Settings → Notifications → Who hears what → Buttons on notifications** switched on (off by default), a notification can be dealt with from the message itself, without opening the app
|
||||
or signing in: **Acknowledge**, **Silence for an hour** (maintenance on what the alert is about) and **Snooze until tomorrow at 09:00** (in the notification time zone). A message about **one** alert gets them as
|
||||
buttons — three on an ntfy message, two on a phone push (the browser's own notification), with the rest on the page a tap opens — and a message about several gets one link to a page that lists each with its buttons;
|
||||
Gotify, email and the chat webhook get that link as text (and Gotify's tap opens it). Escalation messages carry them too. Each link is a **signed token for the alerts in that one message only**, good for **24 hours**, and it
|
||||
|
||||
Reference in new issue
Block a user