Files
Homelab-manager/docs/WEBHOOKS.md
T
bobbanandClaude Sonnet 5.5 c2e76262ca Documentation: guides in docs/, and the reference docs brought up to date
- New docs/ folder: SSO, API, Prometheus and Grafana, webhooks, heartbeats and calendar, how monitoring works, and an index; the old root API.md is split across them
- Read API: alerts carry snoozedUntil
- DATABASE.md: table and migration counts, indexes, relations, integration config, settings and internal flags, delete rules, retention
- ROLES.md: the sidebar table rebuilt from the menu
- NOTIFICATIONS.md, INTEGRATIONS.md, BACKUP.md, AGENT_UPDATE.md: what had drifted
- README: links to the guides; accounts cannot be deleted in the app

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 00:09:43 +02:00

15 KiB

Webhooks

Homelab Manager can tell other systems when something happens — Home Assistant, n8n, a status board, your own script — instead of making them ask. There are two different things called a webhook here, and they are easy to mix up:

Event webhooks The notification webhook
For Machines: a structured, signed JSON message per event People: the same sentence a human would read, posted to a chat-style endpoint
Where Settings → API and webhooks → Event webhooks Settings → Notifications → Channels → Webhook
Sent when An alert opens or resolves, a maintenance window starts or ends Whenever a notification is sent to a channel (it is one of the five channels)
Body { event, id, createdAt, source, data } { "content": "**Title**\nmessage" }
Authenticity X-HLM-Signature — an HMAC-SHA256 of the body a shared secret in X-Webhook-Secret (not a signature)
Held back by routing, quiet hours, flapping, snooze No (it is a separate way of being told) Yes (it is a notification like the others)

Event webhooks

Setting one up

Administrators: Settings → API and webhooks → Event webhooks → Add a webhook.

Field Notes
Name up to 60 characters; what the list shows
Address http:// or https://, no user name or password in it. The address is not restricted to public hosts — a receiver on your LAN is the normal case.
Secret optional, up to 200 characters. With one, every message is signed. Stored encrypted when CREDENTIALS_ENCRYPTION_KEY is set; never shown again after saving.
Events at least one of the four below
  • Up to 20 webhooks. Each can be switched off and on, removed, and tested. The secret cannot be edited afterwards: to change it, remove the webhook and make a new one.
  • Test sends {"event": "test", …} right away, signed like the real thing, whether or not the webhook is switched on or wants that event, and says how it went (200, or what went wrong).
  • The list shows when the last message was sent and its result — a status code, timed out, or the error.
  • Making, switching and removing a webhook are in the audit log with its name and events — never its address or secret (the address can carry a token of its own).

Events

Event Sent when
alert.opened A problem begins.
alert.resolved A problem ends.
maintenance.started A maintenance window begins — by hand, in bulk, from a recurring schedule, or mirrored from Uptime Kuma.
maintenance.ended A maintenance window finishes, on its own or ended early.

What counts as an alert here. alert.opened and alert.resolved are sent for the alerts that go through the health-style bookkeeping — server offline, disks, failed services, SMART and ZFS problems, new ports, network devices offline, your own alert rules, endpoint checks, heartbeats, and automation failures — on the same terms as a notification: a problem under a maintenance window, or folded under a server that is down, is not announced until it stands alone. Two things that do hold back a notification do not hold back the webhook: a problem being flapping, and one that is snoozed — a machine should hear every raise and clear. Summaries (an expiring secret, a stale backup, Docker updates, the weekly digest) are not events; read those from the read API or /metrics.

The message

POST <your address>
Content-Type: application/json
User-Agent: HomelabManager-Webhook
X-HLM-Event: alert.opened
X-HLM-Delivery: 5b0c3e0a-1f4e-4a55-9d0e-6c1c1c1c1c1c   (the same for a retry of the same message)
X-HLM-Signature: sha256=9f2c…                            (only if the webhook has a secret)
{
  "event": "alert.opened",
  "id": "5b0c3e0a-1f4e-4a55-9d0e-6c1c1c1c1c1c",
  "createdAt": "2026-10-17T21:04:11.482Z",
  "source": "homelab-manager",
  "data": {
    "alert": {
      "key": "offline:server:3",
      "source": "server",
      "severity": "critical",
      "message": "pve1 hasn't reported for 1 h — also takes down web and db"
    }
  }
}
data Fields
alert.opened alert.key (stable for the same problem — see Alert keys), alert.source (where it came from, e.g. server, proxmox:3:pve1, endpoint, heartbeat), alert.severity (critical or warning), alert.message
alert.resolved alert.key, alert.message (what it said when it opened)
maintenance.started window: id, targetType (server, integration, dns_provider, device), targetId, targetName, reason, startedAt, endsAt, source (null for one started by hand, schedule:… for a recurring one, kuma:… for one mirrored from Uptime Kuma)
maintenance.ended the same window, plus endedAt — one ended early has its endsAt brought forward
test message

A captured test message, as a receiver sees it (headers lower-cased by the receiving server):

{
  "headers": {
    "x-hlm-event": "test",
    "x-hlm-delivery": "72118185-66dd-49d8-8886-aed8bdc40d61",
    "x-hlm-signature": "sha256=c10f409aac74524e3bc84c0c062f536aa44b98c3bede082f970dfa496fb81f57",
    "user-agent": "HomelabManager-Webhook",
    "content-type": "application/json"
  },
  "body": {
    "event": "test",
    "id": "72118185-66dd-49d8-8886-aed8bdc40d61",
    "createdAt": "2026-10-20T21:46:21.349Z",
    "source": "homelab-manager",
    "data": { "message": "This is a test from Homelab Manager." }
  }
}

Checking the signature

With a secret, X-HLM-Signature is sha256= followed by the hex HMAC-SHA256 of the exact body bytes, with the secret as the key. Compute it over the raw body as received (not over parsed and re-serialised JSON), compare in constant time, and reject a message that does not match.

import { createHmac, timingSafeEqual } from "node:crypto";

export function isFromHomelabManager(secret, rawBody, signatureHeader) {
  const expected = Buffer.from(`sha256=${createHmac("sha256", secret).update(rawBody).digest("hex")}`);
  const given = Buffer.from(signatureHeader ?? "");
  return given.length === expected.length && timingSafeEqual(given, expected);
}
import hmac, hashlib

def is_from_homelab_manager(secret: str, raw_body: bytes, signature_header: str) -> bool:
    expected = "sha256=" + hmac.new(secret.encode(), raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, signature_header or "")
# Quick check at a shell, with the body saved to a file:
printf 'sha256=%s\n' "$(openssl dgst -sha256 -hmac "$SECRET" -hex < body.json | sed 's/^.* //')"

The signature proves the message came from someone with the secret; it does not stop a captured message being sent again later. If that matters, remember the id (or X-HLM-Delivery) of what you have handled and refuse repeats, and reject a createdAt that is far from your clock.

Delivery: what to rely on and what not to

  • Delivery is best-effort and never holds anything up. One try; if it does not get a 2xx answer, one more after two seconds (the same message, the same id and X-HLM-Delivery); then the result is written on the webhook and forgotten.
  • A request is given five seconds. Redirects are not followed (a redirect counts as a failure) — give the final address.
  • There is no queue and no replay. A receiver that is down for longer than those two tries misses the event. Messages for different events can arrive in any order. If you need to be certain, treat the webhook as a nudge and reconcile with the read API (GET /api/v1/alerts, /api/v1/maintenance), which always says what is true now.
  • Because a retry reuses the delivery id, handle each id once (idempotently).
  • Answer quickly (200 is enough) and do slow work afterwards.

Alert keys

alert.key is the stable identity of a problem: the same problem has the same key every time, so you can match an opened with its resolved. The shapes:

Key About
offline:server:<id> A server has stopped reporting
offline:device:<id> A UniFi device has been offline too long
disk:server:<id>:<mount> A disk is nearly full
disk:proxmox:<integration>:…, disk:synology:<integration>:<volume> A Proxmox storage or Synology volume is nearly full
synology-volume:<integration>:<volume>, synology-disk:<integration>:<disk> A Synology volume or disk is not healthy
failed-units:server:<id> Services have failed on a server
smart:server:<id>:<device>, zfs:server:<id>:<pool> A disk says it is failing, a pool is not online
new-port:server:<id>:<proto>:<port> A new TCP port started listening
rule:<rule id>:server:<id>[:<disk>] One of your own alert rules
endpoint:<check id>, endpoint-cert:<check id> An endpoint check is failing, or its certificate is running out
heartbeat:<id>, heartbeat-fail:<id> A job reported late, or said it failed
automation:semaphore:…, automation:gitea:… A Semaphore task or a Gitea Actions run failed

Treat unknown shapes as opaque; match on the whole key.

A small receiver

Node with Express — the one thing to get right is to verify the signature against the raw body:

import express from "express";
import { createHmac, timingSafeEqual } from "node:crypto";

const SECRET = process.env.HLM_WEBHOOK_SECRET;
const seen = new Set(); // use something persistent for real

const app = express();
app.post("/hlm", express.raw({ type: "application/json" }), (req, res) => {
  const expected = Buffer.from("sha256=" + createHmac("sha256", SECRET).update(req.body).digest("hex"));
  const given = Buffer.from(req.get("x-hlm-signature") ?? "");
  if (given.length !== expected.length || !timingSafeEqual(given, expected)) return res.sendStatus(401);

  const msg = JSON.parse(req.body.toString("utf8"));
  if (seen.has(msg.id)) return res.sendStatus(200);     // a retry of something already handled
  seen.add(msg.id);

  if (msg.event === "alert.opened") console.log(`[${msg.data.alert.severity}] ${msg.data.alert.message}`);
  else if (msg.event === "alert.resolved") console.log(`resolved: ${msg.data.alert.message}`);
  res.sendStatus(200);
});
app.listen(8080);

Python with Flask:

import hmac, hashlib, os
from flask import Flask, request, abort

SECRET = os.environ["HLM_WEBHOOK_SECRET"].encode()
app = Flask(__name__)

@app.post("/hlm")
def hook():
    raw = request.get_data()                      # the exact bytes
    expected = "sha256=" + hmac.new(SECRET, raw, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, request.headers.get("X-HLM-Signature", "")):
        abort(401)
    msg = request.get_json()
    if msg["event"] == "alert.opened":
        print(msg["data"]["alert"]["severity"], msg["data"]["alert"]["message"])
    return "", 200

Home Assistant and n8n

  • Home Assistant — an automation with a webhook trigger can take the message as it comes. Home Assistant's webhook URL has an unguessable id in it (/api/webhook/<id>), which is the access control, so a secret here is optional; if you want the signature, check it in a small receiver like the ones above and forward.

    automation:
      - alias: "Homelab Manager critical alert"
        trigger:
          - platform: webhook
            webhook_id: homelab_manager_alerts       # then the address is https://ha.example.lan/api/webhook/homelab_manager_alerts
            allowed_methods: [POST]
            local_only: true
        condition: "{{ trigger.json.event == 'alert.opened' and trigger.json.data.alert.severity == 'critical' }}"
        action:
          - service: notify.mobile_app_phone
            data:
              title: "Homelab alert"
              message: "{{ trigger.json.data.alert.message }}"
    
  • n8n — a Webhook node (method POST) gives you the address; add a Code or IF node on {{$json.body.event}}. To verify the signature in n8n, enable the node's Raw Body option and compute the HMAC in a Code node as above.

The notification webhook

The fifth notification channel (beside Gotify, ntfy, email and push) posts each notification to a URL you give it, as the sentence a person would read — handy for chat tools that accept an incoming-webhook post.

Settings → Notifications → Channels → Webhook: switch it on, give the address, and optionally a secret.

POST <your address>
Content-Type: application/json
X-Webhook-Secret: <the secret>         (only if one is set)

{ "content": "**Homelab Manager — Health Alert**\n1 new issue:\n\n⚠ pve1 hasn't reported for 1 h" }
  • The body is exactly { "content": "<title in bold>\n<message>" } — the shape Discord's incoming webhooks take (Slack and Mattermost expect text, so they need a small adapter in between).
  • X-Webhook-Secret is the secret as given, not a signature — it only lets a receiver reject strangers. For something verifiable, use an event webhook.
  • It follows everything a notification follows: the routing rules (which channels hear what), quiet hours, maintenance, flapping and snooze, and the toggle for each kind of notification. It is delivered once with no retry; a failure is written to the server log.
  • The secret is stored encrypted when CREDENTIALS_ENCRYPTION_KEY is set. The Send test button on the channel sends a one-off message through that channel only.
  • Which messages exist, and when each is sent, is in NOTIFICATIONS.md.

Troubleshooting

Symptom Cause
Test says timed out The receiver did not answer in five seconds, or is not reachable from the app (containers: localhost is the container itself).
Test says 401/403 Your receiver rejected it — often a signature check against the wrong secret, or one computed over parsed rather than raw JSON.
Test says 301/302/308 Redirects are not followed. Use the final address (https:// instead of http://, the trailing slash).
Test works but real events never arrive The webhook is switched off, or does not have that event ticked, or the alert is under maintenance / folded under a host (not announced until it stands alone).
The same alert arrives twice A retry — the id is the same. Handle each id once.
An alert I snoozed or that is flapping still arrived By design: machines hear every raise and clear.
A secret about to expire / a stale backup did not produce an event Summaries are not events. Read them from /api/v1 or /metrics.
I lost the secret It cannot be shown again. Remove the webhook and make a new one.