Sonarr/Radarr/Prowlarr, Hetzner load balancers/floating IPs/certificates, traffic forecast
New integration kinds sonarr, radarr and prowlarr (shared adapter and sync, page under Apps -> Media managers): own health findings, stuck downloads, nearly full disks. Hetzner now reads load balancers (target health), floating IPs and certificates, includes them in the cost estimate, and forecasts the month's traffic against the quota. Migrations 0072 and 0073; docs, metrics and tests included. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9a3d5c466a
commit
d9083adb5b
38 files changed
+18374
-42
No files matched your search
+23
-1
@@ -16,6 +16,7 @@ This page explains what Homelab Manager watches, how often, how a problem become
|
||||
- [Nextcloud](#nextcloud)
|
||||
- [Hetzner Cloud](#hetzner-cloud)
|
||||
- [Game servers (Fragnet)](#game-servers-fragnet)
|
||||
- [Media managers](#media-managers-sonarr-radarr-prowlarr)
|
||||
- [Speedtest](#speedtest)
|
||||
- [NetBox](#netbox)
|
||||
- [n8n](#n8n)
|
||||
@@ -142,6 +143,7 @@ Every threshold here is a default; where it can be changed, the place is named.
|
||||
| **Docker updates** | A container image has a newer version | | Settings → Notifications |
|
||||
| **Nextcloud can't be used / disk low** | The database needs upgrading (critical), maintenance mode is on, or little room is left for the data (critical under 5 GB) | on, with the integration | Nextcloud page; [below](#nextcloud) |
|
||||
| **Hetzner server off / traffic / rescue / backup** | A Hetzner Cloud server that was running is off, is stuck migrating or rebuilding, has used 80% of its month's traffic (critical at 100%), has rescue mode left on, or has backups enabled but none recent | on, with the integration | Hetzner page; [below](#hetzner-cloud) |
|
||||
| **Sonarr / Radarr / Prowlarr** | An application reports a health warning or error, has downloads stuck for two hours, or a disk under 10% free | on, with the integration | Media managers page; [below](#media-managers-sonarr-radarr-prowlarr) |
|
||||
| **Game server backup / disk / memory** | A Fragnet game server's newest backup is older than the limit (7 days) or none, its disk is over 90% of the plan (critical from 98%), or its memory is over 90% while it runs | on, with the integration | Game servers page; [below](#game-servers-fragnet) |
|
||||
| **Internet slow / speed tests failing** | The middle of the last three speed tests is under the share you set of what the line is sold at (only with those numbers given), or the last three tests all failed | on, with the integration | Speedtest page; [below](#speedtest) |
|
||||
| **NetBox has no worker** | No background worker is running | on, with the integration | NetBox page; [below](#netbox) |
|
||||
@@ -263,13 +265,33 @@ An integration of type *Hetzner Cloud* ([INTEGRATIONS.md](../INTEGRATIONS.md#het
|
||||
| **Traffic** | A server has sent **80%** of its included traffic for the month; at **100%** it is critical (the excess is billed) | warning; critical at 100% |
|
||||
| **Rescue mode left on** | Rescue mode has been switched on for **an hour** without a reboot — the next reboot (a kernel update, say) would come up in the rescue system and the server would not come back | warning |
|
||||
| **Backups late** | A server has backups enabled and the newest is older than the *backup older than* limit (48 hours by default — the same setting as for Proxmox guests; 0 turns it off), or none has been made and the server is older than that | warning |
|
||||
| **Traffic forecast** | A server is **on course to pass its included traffic before the month ends** (by the last week's rate, or the month's average after three days) while it has used under 80% — the alert says on which day. Not raised from 80% on: the traffic alert covers that | warning |
|
||||
| **Load balancer unhealthy** | A load balancer has a target Hetzner judges unhealthy; critical when none of its targets is healthy | warning; critical when all |
|
||||
| **Certificate ending** | A certificate that is in use ends within **14 days** (critical within 3, or past its end), or Hetzner could not issue or renew a managed one. A certificate nothing uses is not raised | warning; critical within 3 days |
|
||||
| **Floating IP unattached** | A floating IP points at no server — it is billed all the same | warning |
|
||||
|
||||
Hetzner does not say when a server went off or when rescue mode was switched on, so "since when" is **when the app first saw it** in a read. Maintenance on the integration holds the alerts; a reading older than 20 minutes is not believed (they are neither raised nor cleared, and the Alerts page says it could not check). Snapshots, volumes (including ones attached to no server, which are billed all the same) and the cost are shown, not alerted.
|
||||
|
||||
**The cost** is an estimate from Hetzner's own price list, including VAT: the servers, their backups (a percentage of the server's price), volumes, snapshots and primary IPs. It leaves out load balancers, floating IPs and traffic over the quota, so it is a floor, not a bill.
|
||||
**The cost** is an estimate from Hetzner's own price list, including VAT: the servers, their backups (a percentage of the server's price), volumes, snapshots, primary IPs, load balancers and floating IPs. It leaves out traffic over the quota, so it is a floor, not a bill.
|
||||
|
||||
**The traffic forecast** is made from the month's outgoing traffic, written down about every six hours for two weeks: the rate of the last week (at least two days of it) or, with too little, the month's average so far (after three days). It assumes the month is the calendar month in UTC — not checked against a live project — and a counter that goes down starts the history over. The page shows under each server's traffic bar either the day the quota is passed or what the month is on course to come to.
|
||||
|
||||
**Load balancers, floating IPs and certificates** are listed on the Hetzner page — a load balancer with how many of its targets are healthy, a floating IP with the server it points at, a certificate with when it ends (and the ones nothing uses) — and their prices are in the cost estimate. `/metrics` has each load balancer's unhealthy targets, each certificate's end and each server's forecast month.
|
||||
|
||||
**On a server's page**, a *Hetzner Cloud* card appears when one of the project's servers is that server — matched by its public or private address, else by name — with its status, type, newest backup and traffic. A Hetzner server with no agent is listed as *not in the app's servers*. `/metrics` has `homelab_hetzner_server_running`, `homelab_hetzner_traffic_outgoing_bytes`, `homelab_hetzner_traffic_included_bytes` and `homelab_hetzner_monthly_cost`.
|
||||
|
||||
## Media managers (Sonarr, Radarr, Prowlarr)
|
||||
|
||||
An integration of type *Sonarr*, *Radarr* or *Prowlarr* ([INTEGRATIONS.md](../INTEGRATIONS.md#sonarr-radarr-and-prowlarr)) is read every five minutes, and **Apps → Media managers** shows, for each: what the application says is wrong with itself, its download queue (the stuck ones first, with why), its disks, how many wanted items have no file (Sonarr, Radarr) and, for Prowlarr, which indexers it has switched off after failures. Nothing is searched, grabbed or changed.
|
||||
|
||||
| Alert | When | Severity |
|
||||
|---|---|---|
|
||||
| **It says it is unwell** | One of the application's own health checks is a **warning** or an **error** (a download client that cannot be reached, a root folder gone, indexers unavailable…). The alert quotes it. A *notice* — an update being available — is shown on the page, not raised | warning |
|
||||
| **Downloads stuck** | Downloads that have been in the queue for **two hours or more** and are judged warning or error by the application, have failed, or are blocked from importing. One alert for all of an application's, naming the first three, how long the oldest has been there, and the reason it gives | warning |
|
||||
| **Disk nearly full** | A disk the application uses has under **10%** free, critical under 5%. Folders on one disk count as one (the same size and free space), and a disk under 2 GB is not a media disk | warning; critical under 5% |
|
||||
|
||||
Maintenance on the integration holds the alerts; a reading older than 20 minutes is not believed (they are neither raised nor cleared, and the Alerts page says it could not check). A finding is raised under a key made from the check and its words with the numbers left out, so "3 indexers unavailable" becoming "4" is the same alert, and different words are a new one.
|
||||
|
||||
## Game servers (Fragnet)
|
||||
|
||||
An integration of type *Fragnet game servers* ([INTEGRATIONS.md](../INTEGRATIONS.md#fragnet-game-servers)) is one account on Fragnet's game panel (a Pterodactyl fork), read every five minutes, and **Infrastructure → Game servers** shows its servers: the game, every address and port, whether it runs, memory and disk against the plan, the newest backup, and (opened) the backups, schedules and SFTP address. Operators can start, stop, restart and kill a server there.
|
||||
|
||||
@@ -305,7 +305,14 @@ All gauges. The same table is checked by a test against what `/metrics` really s
|
||||
| `homelab_speedtest_ping_seconds` | `name` | The ping of that test, in seconds. |
|
||||
| `homelab_hetzner_server_running` | `project`, `server` | 1 if the Hetzner Cloud server is running, else 0. |
|
||||
| `homelab_hetzner_traffic_outgoing_bytes`, `homelab_hetzner_traffic_included_bytes` | `project`, `server` | This month's outgoing traffic of a Hetzner Cloud server, and what its type includes per month (bytes). |
|
||||
| `homelab_hetzner_monthly_cost` | `project`, `currency` | The estimated monthly cost of a Hetzner Cloud project at list prices, including VAT (an estimate: no load balancers, floating IPs or traffic over the quota). |
|
||||
| `homelab_hetzner_monthly_cost` | `project`, `currency` | The estimated monthly cost of a Hetzner Cloud project at list prices, including VAT (an estimate: it has the load balancers and floating IPs, not traffic over the quota). |
|
||||
| `homelab_hetzner_traffic_forecast_bytes` | `project`, `server` | What a Hetzner Cloud server's outgoing traffic is forecast to come to by the end of the month; left out when there is too little to say. |
|
||||
| `homelab_hetzner_load_balancer_unhealthy_targets` | `project`, `load_balancer` | How many targets of a Hetzner Cloud load balancer are unhealthy. |
|
||||
| `homelab_hetzner_certificate_expiry_timestamp_seconds` | `project`, `certificate` | When a Hetzner Cloud certificate ends, in seconds since 1970. |
|
||||
| `homelab_arr_health_issues` | `app`, `name` | How many warnings or errors Sonarr, Radarr or Prowlarr reports about itself. |
|
||||
| `homelab_arr_queue_items`, `homelab_arr_queue_stuck_items` | `app`, `name` | The downloads in the queue of Sonarr or Radarr, and how many are stuck (judged wrong, failed or blocked for two hours or more). |
|
||||
| `homelab_arr_wanted_missing` | `app`, `name` | Wanted items Sonarr or Radarr has no file for. |
|
||||
| `homelab_arr_disk_free_bytes` | `app`, `name`, `path` | Free space of a disk Sonarr, Radarr or Prowlarr uses. |
|
||||
| `homelab_fragnet_server_running` | `account`, `server`, `game` | 1 if the Fragnet game server is running, else 0. |
|
||||
| `homelab_fragnet_disk_used_bytes`, `homelab_fragnet_disk_limit_bytes` | `account`, `server`, `game` | The disk a Fragnet game server uses, and the disk its plan allows (left out when unlimited). |
|
||||
| `homelab_fragnet_memory_used_bytes`, `homelab_fragnet_memory_limit_bytes` | `account`, `server`, `game` | The memory a Fragnet game server uses (0 while stopped), and its limit (left out when unlimited). |
|
||||
|
||||
Reference in new issue
Block a user