- New docs/ folder: SSO, API, Prometheus and Grafana, webhooks, heartbeats and calendar, how monitoring works, and an index; the old root API.md is split across them - Read API: alerts carry snoozedUntil - DATABASE.md: table and migration counts, indexes, relations, integration config, settings and internal flags, delete rules, retention - ROLES.md: the sidebar table rebuilt from the menu - NOTIFICATIONS.md, INTEGRATIONS.md, BACKUP.md, AGENT_UPDATE.md: what had drifted - README: links to the guides; accounts cannot be deleted in the app Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
15 KiB
Webhooks
Homelab Manager can tell other systems when something happens — Home Assistant, n8n, a status board, your own script — instead of making them ask. There are two different things called a webhook here, and they are easy to mix up:
| Event webhooks | The notification webhook | |
|---|---|---|
| For | Machines: a structured, signed JSON message per event | People: the same sentence a human would read, posted to a chat-style endpoint |
| Where | Settings → API and webhooks → Event webhooks | Settings → Notifications → Channels → Webhook |
| Sent when | An alert opens or resolves, a maintenance window starts or ends | Whenever a notification is sent to a channel (it is one of the five channels) |
| Body | { event, id, createdAt, source, data } |
{ "content": "**Title**\nmessage" } |
| Authenticity | X-HLM-Signature — an HMAC-SHA256 of the body |
a shared secret in X-Webhook-Secret (not a signature) |
| Held back by routing, quiet hours, flapping, snooze | No (it is a separate way of being told) | Yes (it is a notification like the others) |
Event webhooks
Setting one up
Administrators: Settings → API and webhooks → Event webhooks → Add a webhook.
| Field | Notes |
|---|---|
| Name | up to 60 characters; what the list shows |
| Address | http:// or https://, no user name or password in it. The address is not restricted to public hosts — a receiver on your LAN is the normal case. |
| Secret | optional, up to 200 characters. With one, every message is signed. Stored encrypted when CREDENTIALS_ENCRYPTION_KEY is set; never shown again after saving. |
| Events | at least one of the four below |
- Up to 20 webhooks. Each can be switched off and on, removed, and tested. The secret cannot be edited afterwards: to change it, remove the webhook and make a new one.
- Test sends
{"event": "test", …}right away, signed like the real thing, whether or not the webhook is switched on or wants that event, and says how it went (200, or what went wrong). - The list shows when the last message was sent and its result — a status code,
timed out, or the error. - Making, switching and removing a webhook are in the audit log with its name and events — never its address or secret (the address can carry a token of its own).
Events
| Event | Sent when |
|---|---|
alert.opened |
A problem begins. |
alert.resolved |
A problem ends. |
maintenance.started |
A maintenance window begins — by hand, in bulk, from a recurring schedule, or mirrored from Uptime Kuma. |
maintenance.ended |
A maintenance window finishes, on its own or ended early. |
What counts as an alert here. alert.opened and alert.resolved are sent for the alerts that go through the health-style bookkeeping — server offline, disks, failed services, SMART and ZFS problems, new
ports, network devices offline, your own alert rules, endpoint checks, heartbeats, and automation failures — on the same terms as a notification: a problem
under a maintenance window, or folded under a server that is down, is not announced until it stands alone. Two things that do hold back a notification do not hold back the webhook: a problem
being flapping, and one that is snoozed — a machine should hear every raise and clear. Summaries (an expiring secret, a stale backup, Docker updates, the weekly digest) are not events; read
those from the read API or /metrics.
The message
POST <your address>
Content-Type: application/json
User-Agent: HomelabManager-Webhook
X-HLM-Event: alert.opened
X-HLM-Delivery: 5b0c3e0a-1f4e-4a55-9d0e-6c1c1c1c1c1c (the same for a retry of the same message)
X-HLM-Signature: sha256=9f2c… (only if the webhook has a secret)
{
"event": "alert.opened",
"id": "5b0c3e0a-1f4e-4a55-9d0e-6c1c1c1c1c1c",
"createdAt": "2026-10-17T21:04:11.482Z",
"source": "homelab-manager",
"data": {
"alert": {
"key": "offline:server:3",
"source": "server",
"severity": "critical",
"message": "pve1 hasn't reported for 1 h — also takes down web and db"
}
}
}
data |
Fields |
|---|---|
alert.opened |
alert.key (stable for the same problem — see Alert keys), alert.source (where it came from, e.g. server, proxmox:3:pve1, endpoint, heartbeat), alert.severity (critical or warning), alert.message |
alert.resolved |
alert.key, alert.message (what it said when it opened) |
maintenance.started |
window: id, targetType (server, integration, dns_provider, device), targetId, targetName, reason, startedAt, endsAt, source (null for one started by hand, schedule:… for a recurring one, kuma:… for one mirrored from Uptime Kuma) |
maintenance.ended |
the same window, plus endedAt — one ended early has its endsAt brought forward |
test |
message |
A captured test message, as a receiver sees it (headers lower-cased by the receiving server):
{
"headers": {
"x-hlm-event": "test",
"x-hlm-delivery": "72118185-66dd-49d8-8886-aed8bdc40d61",
"x-hlm-signature": "sha256=c10f409aac74524e3bc84c0c062f536aa44b98c3bede082f970dfa496fb81f57",
"user-agent": "HomelabManager-Webhook",
"content-type": "application/json"
},
"body": {
"event": "test",
"id": "72118185-66dd-49d8-8886-aed8bdc40d61",
"createdAt": "2026-10-20T21:46:21.349Z",
"source": "homelab-manager",
"data": { "message": "This is a test from Homelab Manager." }
}
}
Checking the signature
With a secret, X-HLM-Signature is sha256= followed by the hex HMAC-SHA256 of the exact body bytes, with the secret as the key. Compute it over the raw body as received (not over parsed and
re-serialised JSON), compare in constant time, and reject a message that does not match.
import { createHmac, timingSafeEqual } from "node:crypto";
export function isFromHomelabManager(secret, rawBody, signatureHeader) {
const expected = Buffer.from(`sha256=${createHmac("sha256", secret).update(rawBody).digest("hex")}`);
const given = Buffer.from(signatureHeader ?? "");
return given.length === expected.length && timingSafeEqual(given, expected);
}
import hmac, hashlib
def is_from_homelab_manager(secret: str, raw_body: bytes, signature_header: str) -> bool:
expected = "sha256=" + hmac.new(secret.encode(), raw_body, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, signature_header or "")
# Quick check at a shell, with the body saved to a file:
printf 'sha256=%s\n' "$(openssl dgst -sha256 -hmac "$SECRET" -hex < body.json | sed 's/^.* //')"
The signature proves the message came from someone with the secret; it does not stop a captured message being sent again later. If that matters, remember the id (or X-HLM-Delivery) of what you
have handled and refuse repeats, and reject a createdAt that is far from your clock.
Delivery: what to rely on and what not to
- Delivery is best-effort and never holds anything up. One try; if it does not get a
2xxanswer, one more after two seconds (the same message, the sameidandX-HLM-Delivery); then the result is written on the webhook and forgotten. - A request is given five seconds. Redirects are not followed (a redirect counts as a failure) — give the final address.
- There is no queue and no replay. A receiver that is down for longer than those two tries misses the event. Messages for different events can arrive in any order. If you need to be certain, treat the
webhook as a nudge and reconcile with the read API (
GET /api/v1/alerts,/api/v1/maintenance), which always says what is true now. - Because a retry reuses the delivery id, handle each id once (idempotently).
- Answer quickly (
200is enough) and do slow work afterwards.
Alert keys
alert.key is the stable identity of a problem: the same problem has the same key every time, so you can match an opened with its resolved. The shapes:
| Key | About |
|---|---|
offline:server:<id> |
A server has stopped reporting |
offline:device:<id> |
A UniFi device has been offline too long |
disk:server:<id>:<mount> |
A disk is nearly full |
disk:proxmox:<integration>:…, disk:synology:<integration>:<volume> |
A Proxmox storage or Synology volume is nearly full |
synology-volume:<integration>:<volume>, synology-disk:<integration>:<disk> |
A Synology volume or disk is not healthy |
failed-units:server:<id> |
Services have failed on a server |
smart:server:<id>:<device>, zfs:server:<id>:<pool> |
A disk says it is failing, a pool is not online |
new-port:server:<id>:<proto>:<port> |
A new TCP port started listening |
rule:<rule id>:server:<id>[:<disk>] |
One of your own alert rules |
endpoint:<check id>, endpoint-cert:<check id> |
An endpoint check is failing, or its certificate is running out |
heartbeat:<id>, heartbeat-fail:<id> |
A job reported late, or said it failed |
automation:semaphore:…, automation:gitea:… |
A Semaphore task or a Gitea Actions run failed |
Treat unknown shapes as opaque; match on the whole key.
A small receiver
Node with Express — the one thing to get right is to verify the signature against the raw body:
import express from "express";
import { createHmac, timingSafeEqual } from "node:crypto";
const SECRET = process.env.HLM_WEBHOOK_SECRET;
const seen = new Set(); // use something persistent for real
const app = express();
app.post("/hlm", express.raw({ type: "application/json" }), (req, res) => {
const expected = Buffer.from("sha256=" + createHmac("sha256", SECRET).update(req.body).digest("hex"));
const given = Buffer.from(req.get("x-hlm-signature") ?? "");
if (given.length !== expected.length || !timingSafeEqual(given, expected)) return res.sendStatus(401);
const msg = JSON.parse(req.body.toString("utf8"));
if (seen.has(msg.id)) return res.sendStatus(200); // a retry of something already handled
seen.add(msg.id);
if (msg.event === "alert.opened") console.log(`[${msg.data.alert.severity}] ${msg.data.alert.message}`);
else if (msg.event === "alert.resolved") console.log(`resolved: ${msg.data.alert.message}`);
res.sendStatus(200);
});
app.listen(8080);
Python with Flask:
import hmac, hashlib, os
from flask import Flask, request, abort
SECRET = os.environ["HLM_WEBHOOK_SECRET"].encode()
app = Flask(__name__)
@app.post("/hlm")
def hook():
raw = request.get_data() # the exact bytes
expected = "sha256=" + hmac.new(SECRET, raw, hashlib.sha256).hexdigest()
if not hmac.compare_digest(expected, request.headers.get("X-HLM-Signature", "")):
abort(401)
msg = request.get_json()
if msg["event"] == "alert.opened":
print(msg["data"]["alert"]["severity"], msg["data"]["alert"]["message"])
return "", 200
Home Assistant and n8n
-
Home Assistant — an automation with a webhook trigger can take the message as it comes. Home Assistant's webhook URL has an unguessable id in it (
/api/webhook/<id>), which is the access control, so a secret here is optional; if you want the signature, check it in a small receiver like the ones above and forward.automation: - alias: "Homelab Manager critical alert" trigger: - platform: webhook webhook_id: homelab_manager_alerts # then the address is https://ha.example.lan/api/webhook/homelab_manager_alerts allowed_methods: [POST] local_only: true condition: "{{ trigger.json.event == 'alert.opened' and trigger.json.data.alert.severity == 'critical' }}" action: - service: notify.mobile_app_phone data: title: "Homelab alert" message: "{{ trigger.json.data.alert.message }}" -
n8n — a Webhook node (method POST) gives you the address; add a Code or IF node on
{{$json.body.event}}. To verify the signature in n8n, enable the node's Raw Body option and compute the HMAC in a Code node as above.
The notification webhook
The fifth notification channel (beside Gotify, ntfy, email and push) posts each notification to a URL you give it, as the sentence a person would read — handy for chat tools that accept an incoming-webhook post.
Settings → Notifications → Channels → Webhook: switch it on, give the address, and optionally a secret.
POST <your address>
Content-Type: application/json
X-Webhook-Secret: <the secret> (only if one is set)
{ "content": "**Homelab Manager — Health Alert**\n1 new issue:\n\n⚠ pve1 hasn't reported for 1 h" }
- The body is exactly
{ "content": "<title in bold>\n<message>" }— the shape Discord's incoming webhooks take (Slack and Mattermost expecttext, so they need a small adapter in between). X-Webhook-Secretis the secret as given, not a signature — it only lets a receiver reject strangers. For something verifiable, use an event webhook.- It follows everything a notification follows: the routing rules (which channels hear what), quiet hours, maintenance, flapping and snooze, and the toggle for each kind of notification. It is delivered once with no retry; a failure is written to the server log.
- The secret is stored encrypted when
CREDENTIALS_ENCRYPTION_KEYis set. The Send test button on the channel sends a one-off message through that channel only. - Which messages exist, and when each is sent, is in NOTIFICATIONS.md.
Troubleshooting
| Symptom | Cause |
|---|---|
Test says timed out |
The receiver did not answer in five seconds, or is not reachable from the app (containers: localhost is the container itself). |
Test says 401/403 |
Your receiver rejected it — often a signature check against the wrong secret, or one computed over parsed rather than raw JSON. |
Test says 301/302/308 |
Redirects are not followed. Use the final address (https:// instead of http://, the trailing slash). |
| Test works but real events never arrive | The webhook is switched off, or does not have that event ticked, or the alert is under maintenance / folded under a host (not announced until it stands alone). |
| The same alert arrives twice | A retry — the id is the same. Handle each id once. |
| An alert I snoozed or that is flapping still arrived | By design: machines hear every raise and clear. |
| A secret about to expire / a stale backup did not produce an event | Summaries are not events. Read them from /api/v1 or /metrics. |
| I lost the secret | It cannot be shown again. Remove the webhook and make a new one. |