A UPS in the map: read over NUT or SNMP, alerts, shutdown order
New UPS integration (NUT or SNMP v2c, read only). Monitoring -> UPS shows charge, runtime, load and mains voltage with history and the times mains power was lost. Mains lost and a nearly empty battery are critical alerts; battery to be replaced and a heavy load are warnings. The UPS is a node in the dependency map and counts as down while on battery, so servers that go offline with it are folded under its alert; the page lists what is on it in the order to shut it down. /metrics series and docs added. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3cb3e4988f
commit
232a5661ab
38 files changed
+8049
-16
No files matched your search
+5
-2
@@ -47,6 +47,7 @@ Three ideas run through everything:
|
||||
| **Proxmox**, **Synology** | The health pass **reads** them: node and storage usage, volume and disk status. | Every **15 minutes** |
|
||||
| **UniFi** | A sync **reads** devices, networks, Wi-Fi names and clients and keeps them. | Every **5 minutes** |
|
||||
| **Home Assistant** | A sync **reads** the entities you chose to watch and keeps their state and a history. | Every **5 minutes** |
|
||||
| **UPS** (NUT or SNMP) | A sync **reads** each UPS and keeps its state and a history. | Every **minute** |
|
||||
| **Nginx Proxy Manager** | **Read** when its page is opened and by the alert check, at most once a minute. | On demand, cached 1 minute |
|
||||
| **Semaphore**, **Gitea** | The automation check **reads** the latest run of each template or workflow. | Every **15 minutes** |
|
||||
| **Endpoint checks** (web address, TCP port, TLS certificate) | The app **asks** each one itself. | Each check has its own interval (**5 minutes** by default); the scheduler looks **every minute** for the ones that are due |
|
||||
@@ -128,6 +129,7 @@ Every threshold here is a default; where it can be changed, the place is named.
|
||||
| **Endpoint check failing / certificate running out** | See [below](#endpoint-checks-and-heartbeats) | | per check |
|
||||
| **Heartbeat late / failed** | See [below](#endpoint-checks-and-heartbeats) | | per heartbeat |
|
||||
| **Home Assistant / Nginx Proxy Manager** | An entity unavailable 15 minutes or past your limit; a certificate in use ending within 14 days, or a host NPM cannot set up | | Home Assistant page |
|
||||
| **UPS** | Mains power lost (critical); battery nearly empty (critical); battery to be replaced; load of 90 % or more (warning) | | UPS page; [INTEGRATIONS.md](../INTEGRATIONS.md#ups-nut-or-snmp) |
|
||||
| **Your own rules** | [Below](#your-own-alert-rules) | | Monitoring → Alert rules |
|
||||
|
||||
## Your own alert rules
|
||||
@@ -185,7 +187,7 @@ using credentials an administrator enters under *Integrations*; what each needs
|
||||
restart starts the count again. A *down* notification goes out at the threshold and a *recovered* one on the next success. The moments are written to the audit log, and they feed the uptime record on the
|
||||
integration's page.
|
||||
- **When an integration cannot be read, what it reported is held, not cleared.** A Proxmox node that cannot be read does not make its storage alerts disappear (or appear); the Alerts page says it could not check.
|
||||
The synced ones (UniFi, Home Assistant) are only judged while their last good read is recent (**20 minutes**).
|
||||
The synced ones (UniFi, Home Assistant) are only judged while their last good read is recent (**20 minutes**; a UPS, **5**).
|
||||
- **An integration can explain other alerts.** A service that stops answering because the integration it needs is failing is folded under it ([Dependencies](#dependencies-one-outage-one-alert)).
|
||||
|
||||
## Forecasts
|
||||
@@ -280,6 +282,7 @@ hand, a config file edited — is not something the app can see.
|
||||
| Uptime of servers and integrations | The last weeks, worked out from the moments each stopped and resumed (in the audit log) | one figure a day |
|
||||
| Proxmox storage and Synology volume use | **400 days**, one reading an hour | |
|
||||
| Home Assistant watched numbers | **35 days**, one reading every few minutes | |
|
||||
| A UPS's charge, load and mains voltage | **35 days**, one reading every five minutes (every minute on battery) | |
|
||||
| Incidents | **400 days** after they resolve | |
|
||||
| Messages sent to event webhooks | The last **100 per webhook**, none older than **14 days** — see [WEBHOOKS.md](WEBHOOKS.md#the-delivery-log-and-sending-again) | |
|
||||
| The audit log and the Diagnostic Log | Until automatic purging is switched on under *Settings → Data* (then 90 days by default, configurable) | |
|
||||
@@ -297,7 +300,7 @@ switched on). The last two are listed at the top as needing a look. Nothing ther
|
||||
| **Dashboard**, **My work**, the menu badge | What needs attention now; what is yours |
|
||||
| **Alerts** (Now · History · Noise) | Every alert, with acknowledge, snooze, silence; the incident record; what keeps coming back |
|
||||
| A **server's page** | Its figures, history, health, ports, tasks, notes, jobs, what it runs on and what runs on it |
|
||||
| **Endpoints**, **Heartbeats**, **Proxmox**, **Synology**, **Network devices**, **Home Assistant**, **Proxy hosts** | Each source's own view |
|
||||
| **Endpoints**, **Heartbeats**, **Proxmox**, **Synology**, **Network devices**, **Home Assistant**, **UPS**, **Proxy hosts** | Each source's own view |
|
||||
| **Weekly digest** and **monthly report** | The week and the month in one message |
|
||||
| **`/metrics`** → Prometheus, Grafana | [PROMETHEUS-GRAFANA.md](PROMETHEUS-GRAFANA.md) |
|
||||
| **Read API** (`/api/v1`) | [API.md](API.md) |
|
||||
|
||||
@@ -292,6 +292,8 @@ All gauges. The same table is checked by a test against what `/metrics` really s
|
||||
| `homelab_endpoint_response_milliseconds` | `name`, `kind` | How long it took to answer the last time. |
|
||||
| `homelab_endpoint_value` | `name`, `kind` | The number a value check last read from the answer (checks that judge a value only, and only when it is a number). |
|
||||
| `homelab_endpoint_certificate_days_left` | `name`, `kind` | Days until the certificate it shows expires (negative once it has). |
|
||||
| `homelab_ups_on_battery` | `name` | 1 if the UPS has lost mains power and runs on its battery, else 0; left out for a UPS that has not been read in the last five minutes. |
|
||||
| `homelab_ups_battery_charge_percent`, `homelab_ups_runtime_seconds`, `homelab_ups_load_percent`, `homelab_ups_input_volts` | `name` | How full the battery is, how long it would last at the present load, how much of its capacity the load uses, and the mains voltage it sees — each left out when the UPS does not say. |
|
||||
| `homelab_heartbeat_up` | `name` | 1 if the job has reported in time and not said it failed, else 0; switched-off heartbeats are left out. |
|
||||
| `homelab_heartbeat_last_ping_timestamp_seconds` | `name` | When the job last reported that it finished (Unix time). |
|
||||
| `homelab_heartbeat_last_duration_seconds` | `name` | How long the job said its last run took (only when it sent `/start`). |
|
||||
|
||||
Reference in new issue
Block a user