New UPS integration (NUT or SNMP v2c, read only). Monitoring -> UPS shows charge,
runtime, load and mains voltage with history and the times mains power was lost.
Mains lost and a nearly empty battery are critical alerts; battery to be replaced
and a heavy load are warnings. The UPS is a node in the dependency map and counts
as down while on battery, so servers that go offline with it are folded under its
alert; the page lists what is on it in the order to shut it down. /metrics series
and docs added.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A web check can read a value out of a JSON answer (power.now, state,
data.result[0].value[1]) and judge it: below, above, equal to, contains, is there.
The value read is shown on the Endpoints page, charted under History, exported as
homelab_endpoint_value and in the read API. A check can also send one header (a
token), stored encrypted, never returned, and not forwarded to another host on a
redirect.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Settings -> Data -> Storage measures the database, backups and sessions and every
table (size, rows, growth, how long it is kept, how far back it reaches), and flags
histories that are not being trimmed or have no limit.
Event webhooks now keep a delivery log (last 100 per webhook, 14 days): what was
sent, how it went, the message itself, and Send again, which resends it exactly as it
was (same id and signature). Resends are audited and in the Change log.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Monitoring -> Alert rules: CPU, memory, disk use and disk temperature limits with a time, severity and scope (all, one server, a tag); try before saving; findings are ordinary health conditions
- Proxmox storages and Synology volumes are written down on each health pass (hourly, 400 days) and forecast like server disks: badge and history chart on the pages, alert, digest line, /metrics gauge
- Snooze an alert until a time; held back like maintenance, messages held, announced once again if still there when it ends
- Docs updated
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Racks page and a network diagram (UniFi tree with servers, printable)
- Notes on servers and devices; To do list shown in Coming up and My work
- Agents report the operating system (Linux 7, Windows 6); end-of-life badge and alert
- Suggested dependencies from where a service is reached
- Nginx Proxy Manager integration: proxy hosts page, certificate and host alerts, suggested checks
- Home Assistant integration: watch chosen entities, history chart, limit and unavailable alerts
- TimeChart gets a "value" scale that fits negatives; docs updated
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
History: server load, memory and disks are also kept as hourly (four months) and
daily (over two years) averages with lowest and highest; endpoint answers as
hourly (four months) beside the daily figures. Charts ask for 24 hours to a year
and are drawn from whichever is fine enough. A disk that two weeks call flat is
looked at again over ninety days for the forecast. Built once from what is kept.
One chart: TimeChart replaces the sparkline and the multi-line chart: hover and
arrow-key readout, a lowest/highest band, gaps shown as gaps, the same ranges
everywhere and the choice remembered. Used for server trends, comparing servers,
a check's availability and answer times (Endpoints > History) and a server's
availability.
Read API: /api/v1 (servers, metrics, alerts, incidents, inventory, network,
endpoints and their history, maintenance) with personal read-only tokens made
under the user menu: shown once, hashed, 30/90/365 days or none, ten at most,
removed with their owner. GET only. Documented in API.md.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
UniFi: the controller's client list is matched to servers by IP address, so a
server is connected through a switch or access point (and port). That is a
dependency made by itself, shown on the server page, the Network devices page
and the Connections map. IP Addresses can sync the devices' addresses.
A server's page is split into Overview, Health, Network, History, Tasks and
Settings, by ?view=; alerts link to the section they are about.
My work lists what has your name on it: alerts you have, maintenance you
started (extend, end with undo) and update runs you started, with a badge.
Settings has a search box (and the command palette lists every setting).
Removing a dependency, admin link, runbook, hand-made IP address, port opening
or restore test is undone from the message instead of asked about first.
Two backup tests used a fixed date and failed as days passed; they follow the
real clock now.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A UniFi controller (API key on Network 9+, or a read-only local login on any
version) is read every five minutes. Its switches, access points and gateways
are kept with state, IP, uptime, firmware and uplink; its networks and Wi-Fi
names beside them (passphrases are never read). Nothing is written to the
controller.
Network > Network devices lists them. Devices are a new node type in the
dependency map: uplinks become dependencies by themselves, others can be
written by hand. A device offline past a limit is an offline alert, folds what
depends on it, can be silenced with maintenance and have runbooks.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Patch plans (Updates -> Patch plans): updates on a schedule, one server at a time, each by an
ordinary update run, stopping at the first failure by default, silencing what depends on a
server while it is updated, with one message at the end.
Monthly report (Operations -> Monthly report, and optionally a message on a day of the month):
availability, incidents, updates, restore tests, backups and renewals for a calendar month,
printable. Backed by daily endpoint figures and a record of renewals.
New ports: the first report with ports is the baseline; a TCP port that begins listening later and
is not noted on the server's Ports card is a warning, cleared by Mark as expected.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Heartbeats: a cron job or script calls /api/heartbeat/<token> (and /start, /fail); the app
alerts when the call is late or the job says it failed. Coming up: one agenda of everything
dated (expiries, warranties, end of support, planned and recurring maintenance, restore tests,
waiting updates), with a dashboard card and an iCalendar feed behind a share link. Suggested
checks: web ports and certificates the app already knows about, added with a click.
Menu: Incidents become Alerts -> History (old /incidents links redirect); new Monitoring group
(Endpoints, Heartbeats, Uptime Kuma); osTicket under Automation, Admin Links under
Infrastructure, Generator under Administration. Also makes the restore-test metrics test follow
the real clock.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Both sources are off until switched on under Settings, Sign-in. The server asks only
gravatar.com (by a hash of the email) and the identity provider's own host, accepts only
small PNG/JPEG/WebP/GIF, keeps each picture for a day and serves it from /api/avatars/<id>,
so browsers never contact either. Shown in the user menu and on the Users page, with initials
as the fallback; the Privacy page says when it is on.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Inventory: end of support and replacement cost per asset; a Lifecycle page showing what ages out year by year and what replacing it will cost; support ending within 90 days in the weekly digest
- Lists: a shared empty state saying what is missing, why, and the one thing to do; "nothing matches" with a clear button; skeleton rows instead of empty-looking lists while loading
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Endpoints: HTTP, TCP and TLS checks with answer time, uptime and certificate expiry; failures and expiring certificates become alerts and notifications; linked to a server for maintenance and tags
- Restore tests: record when a backup was restored and how it went; failed tests always alert, overdue ones with a limit set; shown on the Backup coverage page
- Change log: one filterable timeline made from the audit log and the incident record
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Agents (Linux 6, Windows 5) note each check they couldn't make and why; the server page turns that into tips and fixes, including when the agent is behind
- Linux --prepare leaves its notes for the report; a missing prepare step is itself noted
- Services to ignore are set per server from the Host health card (names or * patterns), applied on the server so alerts and /metrics drop them at once; saving alert limits keeps the list
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Routing: rules pick channels by kind, severity and server tags (first match wins, none = nobody); mixed batches split per route; recoveries follow their alert; quiet-hours digest keeps each notification's audience
- Push: each subscribed browser can ask for a minimum severity and some kinds
- Escalation: up to three timed steps re-notify alerts nobody has acknowledged, through chosen channels and an optional extra email; an acknowledgement stops them
- Fix ntfy: titles with an em dash made every ntfy send fail (header not Latin-1); now RFC 2047 encoded
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Dependencies: servers and integrations can depend on each other; alerts are folded under the root cause, and a failure explained by something down isn't announced
- Web push: hand-written VAPID and RFC 8291 encryption, per-device subscriptions, a fifth notification channel, service worker handlers
- Inventory: maker/model/serial from the agents fill blank fields, "Refresh from agent", QR label printing and ?asset= deep links
- Windows agent (v4) reports failed services, disk health, Storage Spaces pools and hardware identity; Linux agent (v5) reports hardware identity
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Share links can open a status page: an admin picks servers, integrations and Uptime Kuma
monitors and names them; the public sees only those names, Working / Not working /
Being worked on / Can't tell, an optional message and the 30-day figure (share_links.kind/
title/message and status_items, migration 0033). A status address doesn't open the wall and
the other way round; Uptime Kuma is asked at most every 30 seconds
- Grafana dashboard built from code (download in Settings -> API and webhooks); a test checks
it only uses series /metrics serves
- Tests and docs
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Alerts can be acknowledged (note, assignee, duration); acknowledged alerts are left out of what
needs attention (sidebar, dashboard strip, cards) but stay raised and counted by /metrics
- Incidents: each time an alert is raised and goes away is recorded (alert_acks, alert_incidents,
incident_notes, migration 0032), with a timeline of acks, maintenance, changes, update runs,
related alerts and notes; Incidents page and a digest section
- Backups of Homelab Manager's own database: daily checked VACUUM INTO copies, retention, download,
check, upload, delete, and restore staged and applied at start-up before the database opens
(pre-restore copy kept); alerts and /metrics for stale or failing backups
- Docs: README, BACKUP, DATABASE, NOTIFICATIONS, ROLES, API; tests
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Backups: a guest whose newest backup is older than its limit is an alert (backup kind),
in the daily backup message, the digest and /metrics (homelab_backup_guests, ..._last_age_seconds)
- Per-server thresholds (servers.thresholds, migration 0030): offline minutes, disk percent,
forecast window, backup age, and per-disk limits or "leave alone"; applied everywhere the
global ones were; saving clears the alerts cache
- Update runs (update_runs, migration 0031): Update... on the Updates page puts the server under
maintenance, runs a Semaphore playbook limited to it, waits for the host to report, and
reports what is left and whether a restart is due; poller survives restarts
- Notification toggle for update-run results; tests and docs
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Linux agent v4: failed systemd services, SMART disk health, ZFS pool state, pacman updates;
SMART/ZFS and self-update run in the ExecStartPre=-+ step outside the sandbox
- Server: hosts' health stored (servers.host_health, migration 0029), alerts under a new
"Failing services" kind plus disk/ZFS alerts, Host health card, Prometheus series
- Agent self-update (Linux v4, Windows v3): admin switch on the Servers page, checksum-verified
download, atomic replace; GET /api/agent/report/update
- homelab_server_disk_days_to_full in /metrics
- Tests (server, real PowerShell against a fake server) and docs
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Export CSV for the small tables that had none (reads the table as shown)
- Servers page shows a skeleton while loading instead of "No servers registered yet"
- Recurring maintenance schedules can be edited (PUT /schedules/:id)
- A server can be set to run on nothing (servers.no_parent, migration 0028)
- Linux agent v3: package-manager question runs in ExecStartPre=-+ outside the sandbox,
with --prepare-updates and --check-updates modes
- Docs: README, DATABASE, AGENT_UPDATE
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Runbooks: notes per server/integration, shown next to matching alerts
- Backup coverage page across Proxmox and PBS, with a digest section
- Hardware inventory with warranty, power and running cost
- Authentik group to role mapping with preview and safeguards
- /metrics (bearer token) and signed outgoing event webhooks
- Docs: README, DATABASE, NOTIFICATIONS, ROLES and new API.md
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Weekly digest through the notification channels: alerts now, a week of uptime, what happened, disks
heading for full, expiring secrets and domains, updates waiting, planned maintenance. Off until switched
on in Settings -> Notifications; Preview and Send it now there. Setting key 'digest'.
- Agents say their version in every report (Linux and Windows scripts are version 2); the server compares
it with the scripts it serves and marks agents that are behind on the Servers, server and Updates pages.
servers.agent_version, migration 0026.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Servers can say what they run on (or it is worked out from the Proxmox node); when the host is
down the guests' offline alerts fold under it: held back, not counted, not notified, still recorded.
- Recurring maintenance schedules made into ordinary windows a week ahead (maintenance_schedules).
- Run a Semaphore template limited to a server from its page or an alert, and watch the run.
- Agents report pending updates (apt, dnf, zypper, Windows Update); an Updates page lists them.
- Migrations 0024 and 0025.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Table search, status, tag and sort live in the address; Save view pins one as a favourite.
- Dashboard cards can be moved and hidden per user (users.dashboard_layout, migration 0022).
- Compare two to four servers' load, memory or disk on one chart.
- j/k move through table rows that open a detail panel.
- Share links open a read-only wall without signing in, numbers only or with names
(share_links, migration 0023); the wall is split into a shared view over two data sources.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Dashboard card on what changed since the user pressed Got it (users.last_looked_at,
migration 0021), folding each problem and its recovery into one line.
- Uptime ribbons on servers and integrations, built from the audit log; integrations'
failing/working-again moments are now recorded.
- Timeline of running and scheduled maintenance windows on the Maintenance page.
- TimeAgo component: relative times with the exact time in the chosen format on hover.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A star beside each page in the sidebar pins it under "Favourites" at the top of
the menu; the list is saved with the user (users.favorites, migration 0020), so
it follows them to any browser, is offered first in the search box, and can be
changed from there too. A button at the foot of the sidebar, or the [ key,
narrows it to a strip of icons, where a group's pages open in a small menu
beside its icon and the alert and maintenance counts stay visible; the choice
is kept per browser and applies only on wide screens. Menu text on a coloured
(environment) sidebar is full strength for contrast, and menu badges sit in
line on a phone.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Tables stack into labelled cards on a phone. Green, red and amber badges carry
a check, cross or triangle as well as their colour. Pages inside a menu group
show a breadcrumb, with the thing's name on a server's page. Integrations are
cards with an Open button and a live status (working, failing, not checked yet,
disabled). Agent-reported servers keep a short history of load, memory and disk
use (migration 0019, one sample per ten minutes for 35 days) and show it as a
Trends card with a days-until-full forecast for a filling disk.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Uptime Kuma often watches a host by another address than the one it was added
with here (a Proxmox host monitored over Tailscale, say). Address aliases on
the Maintenance page say that an address belongs to a server, integration or
DNS provider, so the sync puts the window there; matching stays exact.
Migration 0018 adds maintenance_aliases.
Also fixes monitors whose empty hostname/URL/port arrive from Kuma's metrics as
the text "null": they were taken as an address, hiding the real URL.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Uptime Kuma shares its maintenance windows only over its dashboard Socket.IO
connection, which takes a login. With a login username and password on the
integration, "Sync schedule" on the Maintenance page (and a 15-minute
background pass when switched on) reads each window's running and next time
slot and the monitors it covers, and makes a maintenance window on every
server those monitors match. Windows that haven't started are listed as
Scheduled and silence nothing until they begin; ones ended or cancelled by
hand stay ended, and ones removed in Kuma are removed here.
maintenance_windows gets a source column (migration 0017) and active windows
now require their start to have passed. Adds socket.io-client, and tests
against a stand-in Uptime Kuma speaking the same protocol.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The expiry date is now optional: no date means the secret never expires,
gets the status "no expiry" and never raises an alert or reminder. The
Secrets form has a "Never expires" checkbox. Certificates checked against a
host keep their live expiry and can't be set to never expire. Migration
0016 makes secrets.expiry_date nullable.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
With Settings -> Alert tickets switched on (off by default), every 15 minutes the problems on the
Alerts page are looked at and one that is new and has lasted gets one ticket in osTicket, through
its API (POST api/tickets.json, key in X-API-Key). One problem opens one ticket however many checks
see it; the app remembers what it has seen in a new alert_tickets table (migration 0015).
- Which: critical and warning by default, every kind; severities and kinds are settings.
- A problem must be seen in 2 checks in a row first (1-4, a setting), so a blip that fixes itself,
like one failed call to an integration, never opens a ticket.
- One ticket per occurrence; a problem that clears and returns within 60 minutes is the same one.
- More than 5 new problems in one check become a single summary ticket.
- Never for alerts under a maintenance window, nor about osTicket itself or its overdue tickets.
- Problems already there when it is switched on are left alone by default (the first check records
them as a baseline); can be set to ticket them too.
- If osTicket can't be reached nothing is recorded, so the next check retries; if a source couldn't
be read, a problem isn't treated as gone. Every ticket is audited.
The ticket API address, key (stored encrypted) and self-signed option are optional fields on the
osTicket integration, which otherwise still only reads the database. The settings page can send a
test ticket and run a check now; the Alerts page shows each alert's ticket number, linked to osTicket.
osTicket's API can't close or update a ticket, so a ticket opened here is the end of this app's part
in it. Tested against a stand-in osTicket, not a real one.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Summarizes every server's agent-reported listening ports in one
cross-server table (grouped by protocol+port, addresses merged,
loopback-only flagged) - previously this only existed per-server on
each server's own detail page.
Adds a second table for ports this app has no way to see on its own:
manually-recorded openings on a router, edge firewall, or cloud
security group, each with a label, external port/protocol, an optional
link to a tracked server (with its own internal port when NAT changes
it) or a freeform destination, a free-text source, and a comment.
Viewer-readable; adding/editing/deleting needs operator or admin.
The agent-port grouping logic (dedupe by protocol+port, detect
loopback-only sockets) was shared with the existing per-server Ports
card via a new agentPorts.ts service instead of duplicating it.
Verified with a real HTTP-level test: a genuine Express app with the
actual routers, a scratch SQLite DB, and forged admin/viewer sessions,
covering grouping correctness, the server-name join, input validation,
and role enforcement - the real dev DB was confirmed untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New Settings > Tags tab (admin) listing every tag -- ones servers use and
ones added ahead of time -- with how many servers carry each:
- Add a tag before anything uses it, optionally with a colour. Such tags
are offered as one-click "Add:" chips (and datalist suggestions) when
tagging a server, so the same word gets spelled the same way everywhere.
- Give any tag a colour of your choosing, in use or not, or reset it to the
automatic one. Changes are drafted with Save/Cancel rather than saved as
the picker drags. Colours show on the Servers page, its tag filter bar,
the detail page and the editor, and update everywhere without a reload
through one shared cached colour map.
- Rename a tag; every server that has it is rewritten. Renaming to a name
that already exists merges the two after a confirmation naming what will
happen; the target keeps its own colour unless it had none, and a server
carrying both ends up with one.
- Delete a tag, which removes it from every server that has it, with a
confirmation stating how many.
Tags still live on the servers (servers.tags); a new tag_definitions table
holds only what a server can't: existence before use, and a colour. A
defined tag stays listed until an admin deletes it, even with no servers.
Rename and delete change the servers and the catalogue in one transaction
so they can't disagree. Only servers that actually carry the tag are
rewritten and counted -- an earlier draft also counted servers whose tags
merely weren't in sorted order, which the tests caught.
Reading the list and colours is open to everyone signed in (needed to draw
tags anywhere); changing the catalogue is admin-only, while tagging a
server stays an operator action. Names go through the same normalisation
as before, colours must be #rrggbb, and every change is audit-logged.
New table tag_definitions (migration 0013).
Verified with 44 backend checks (list/counts, roles, create/adopt/
duplicate/rejects, colour set/reset, rename incl. defined, undefined and
unused tags, merge colour rules and both-sides servers, delete, audit, and
that normal tagging still works afterwards) and in a browser against the
real routers: add with colour, set/reset a colour and see it change on the
Servers page live, merge with confirmation, delete, and the error path.
Not clicked through: the quick-add chips inside the tag editor on a
server's detail page (typechecked; same colour code as the rest).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New Consistency page listing where the three places this app records
what lives at an address disagree:
- Address conflicts: the same address reported by more than one server.
- DNS out of date: a record named after a server (its hostname, or its
short name) that points at an address the server doesn't report.
- IPAM out of date: an entry labelled with a server's name at an
address the server doesn't report.
- Not in IPAM: addresses a server reports or DNS points at that IPAM
doesn't list, merged into one finding per address, with a one-click
"Add to IPAM" that pre-fills a label.
- No DNS record: server LAN addresses no cached A/AAAA record resolves to.
It compares data the app already holds and fetches nothing when opened,
so the page states how many servers had reported addresses and how many
DNS zones are synced (and how old the oldest sync is) -- DNS records are
only cached for zones that have been synced, and a report that silently
treated missing data as "no records" would mislead.
Rules chosen to keep it from crying wolf:
- Only private addresses are compared; public DNS records aren't expected
to be in IPAM.
- Servers with no reported addresses are never judged.
- Agents report IPv4 only, so records are only compared within an address
family (an AAAA record isn't "stale" for lacking an IPv6 address).
- Docker bridge networks (172.16/12) are ignored: shared ones aren't
conflicts, and they aren't listed unless someone put them in DNS.
- Tailscale addresses don't need DNS records (MagicDNS), and IPAM entries
kept current by the Tailscale/Proxmox syncs aren't second-guessed.
Findings anyone has decided are fine can be ignored (operators) with a
reason. An ignore is keyed on the finding's stable identity so it stays
ignored across runs, its stored text comes from the finding rather than
the request, and it is marked "no longer occurring" once the condition
goes away. Ignore/restore are audit-logged.
New table consistency_ignores (migration 0012). portScan's private-address
helper is now exported and shared.
Verified with 36 checks (each rule and its exclusions, address-family and
case/trailing-dot handling, IPv6 case, ordering, stable keys, the report
route including a garbled agent report, source counts, ignore/unignore
rules and audit entries) and by driving the page against the real routers
in a browser: Add to IPAM actually created the entry, ignore and restore,
severity filter, "show all", the viewer view, and narrow-width layout
(which found and fixed a squeezed badge and clipped buttons). Real dev
database mtime untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New Domains page listing when each domain registration expires, read from
the registry. Domains behind the DNS zones already synced are picked up
automatically; others can be added by hand. You're reminded daily from N
days before expiry (Settings > Notifications, default 30) until it's
renewed, and told when an expiry date hasn't been refreshable for several
days so a stale date isn't trusted silently.
RDAP alone would not have covered this homelab: .se, .nu, .io, .eu and .de
are not in IANA's RDAP bootstrap. Lookups therefore try RDAP where the TLD
publishes a server and fall back to WHOIS on port 43, found via IANA's
own referral, parsing the expiry line out of the free-text answer. Only
the expiry date and registrar are read or stored. Verified live against
the real registries: .se and .nu via WHOIS, .com/.org/.dev via RDAP.
Behaviour worth knowing:
- A DNS zone that is a subdomain (lab.example.se) resolves to the
registration that actually expires by trying the name and then its
parents, so no public-suffix list is needed. Zones already covered by a
tracked domain are not looked up again.
- "Couldn't ask" is never confused with "not registered": network errors,
rate limits and garbled answers are errors, and a transient error at any
level stops the walk from concluding the domain doesn't exist.
- A failed refresh keeps the last known expiry and records why, rather
than blanking a date that's still relied on.
- Zones that don't resolve to a real registration (.lan, .local, unregistered
names) simply get no row. Zone-derived rows disappear when their zone
does; manual rows stay. Zone-derived rows can't be deleted by hand.
- Registries that don't publish an expiry (.de, .eu) are tracked with a
note instead of a date.
- Input like "example.com/path" is refused rather than silently reduced
to its host.
- Runs on the daily secret-expiry schedule and reminder time, on demand
(Check all now / per domain), and once at startup if nothing has been
read in a day. Never blocks startup, one lookup at a time with a pause.
The warning window lives with the other thresholds in settings.
New table domains (migration 0011); two settings fields (toggle and
warning days).
Verified with 90 checks against fake RDAP/WHOIS backends (name
normalization, date formats, WHOIS parsing including rate-limit and
no-expiry answers, bootstrap and referral caching, stale-cache fallback,
parent walking, add/sync/check/refresh, concurrency guard, alert
selection and stale detection, the daily notification and its toggle,
role rules) plus a live smoke test against real registries and a browser
check of the page against the real router. Real dev database mtime
untouched. Not checked: a screenshot of the finished page (the capture
timed out); structure, sorting, errors and the viewer view were verified.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Servers can be tagged (prod, media, rack-1, ...) for grouping. Operators
and admins edit tags inline on a server's detail page; the input suggests
tags already used on other servers so the same word ends up spelled the
same way everywhere. Tags show as chips that keep one colour per tag, on
the server cards, in the Manage table (and its CSV export), and on the
detail page, where each chip links to the Servers list filtered by that
tag. The Servers page has a tag bar with counts; picking several tags
narrows to servers that have all of them. The filter lives in the URL, so
it survives a refresh and can be linked to. Tags are also matched by the
global search.
Tags are normalized on the server (trimmed, lowercased, spaces become "-",
duplicates merged, sorted); letters in any language are allowed, plus
digits and - _ . : /, at most 30 characters and 12 per server. Invalid
input is rejected with a message naming the offending tag, and nothing is
saved. Changes are audit-logged with the before and after lists.
Stored as a JSON column on servers (migration 0010). Every server response
now returns tags as an array, and the shared response shaping strips the
token hash in one place instead of five.
Verified with 27 backend checks (normalization edge cases including
Swedish letters, roles, validation, search, audit, PATCH/detail/list
shapes) and by driving the real Servers and detail pages against the real
router in a browser (filtering, editing, invalid tag, viewer view). Real
dev database mtime untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Each server's detail page now has a Ports card. "Scan…" runs a TCP connect
scan of a chosen range from the app and shows what's open, along with the
ranges that were actually confirmed free; clicking a free range starts a
reservation. Any port can carry a service name and a comment, so the page
also answers "what is this port for". A port with a note counts as taken
even when nothing is listening, which is what makes a reservation work.
Operators can scan and edit; everyone can read. Scans and note changes are
audit-logged.
Details that matter for correctness:
- "Free" means the host actively refused the connection AND nobody has
claimed the port. A port that never answers (firewall drop, host down)
is reported as not answering, not as free.
- A scan from elsewhere can't see services bound to localhost only, so the
agent now also reports what is bound on the host (ss -tulnp) and those
ports are treated as taken. They show as "local only". Existing agents
keep working; re-run the install one-liner to add this. The field is
validated leniently so one odd line can never cost an agent its whole
report, tasks included.
- If nothing answers at all during a scan, existing results are left
alone instead of being marked all-closed.
- Scan targets are limited to private addresses (RFC1918, Tailscale
100.64/10, link-local, IPv6 ULA/link-local); loopback and public
addresses are refused. Ranges are capped at 20,000 ports, and only one
scan runs per server at a time.
- Rows exist only while they carry information: an open port, or one with
a note. A closed port with no note disappears on the next scan; one with
a note stays as "reserved".
New table server_ports plus two columns on servers (migration 0009).
Verified with 76 backend checks (scanner open/refused/filtered, address
rules, agent report leniency, note/reserve/clear semantics, free-range
calculation including the localhost-only case, roles, concurrency lock,
no-response guard, audit entries, cascade delete) and by driving the real
component against the real router in a browser. Real dev database mtime
untouched.
Not verified: the agent's ss/awk/jq pipeline on a real host — the awk step
was checked against sample ss output and the script passes bash -n, but
jq isn't available here to run the whole thing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rebooting Proxmox or patching a server triggered failure/offline alerts
you then had to dismiss. A maintenance window silences alerts about one
server, integration, or DNS provider for a chosen time. New Maintenance
page (start with a duration and optional reason, end early, see what's
silenced and what isn't) and a banner in the app shell so every signed-in
user can see what is currently silenced. Starting/ending is operator-only
and audit-logged; starting one on a target that already has a window
restarts its clock instead of stacking.
Silenced for the target: server offline/disk alerts, Proxmox/Synology
storage and health alerts, Proxmox backup alerts, and "integration down"
alerts. Not silenced: expiry and update reminders, DNS change notices.
The design goal is that this cannot hide a real outage:
- Every window has a required end (5 min to 7 days); there is no
open-ended option, so a forgotten window expires by itself.
- A silenced problem is deliberately NOT recorded as "known". If it is
still present when the window ends it alerts then, as new. A problem
that was already alerted before the window stays known, so it isn't
repeated, and is reported cleared only after the window ends.
- Failure alerts keep counting failures during a window without marking
themselves alerted, so an outage that outlasts the window alerts on the
very next failed call.
Known limitation, stated on the page: integration-failure alerts are
tracked per service TYPE (all "proxmox"), not per configured instance, so
a window on one Proxmox integration also silences a failure on a second
Proxmox integration while it's open. Fixing that means threading the
integration id through every adapter and the diagnostic log, which is a
much larger change than this feature.
Also moved the API-error-message helper out of Secrets.tsx into a shared
util now that two pages use it. New table maintenance_windows (migration
0008).
Verified with 44 checks: the condition-key-to-subject mapping (including
server:3 vs server:33), the diff rules with silenced subjects (new problem
not recorded, alerts when the window ends; already-known one carried and
not repeated; clears only after the window), window expiry and
integration/DNS-provider source matching, the failure tracker end to end
against a webhook (silent during a window while an unrelated service still
alerts; outage that outlasts the window alerts on the next failure and
only once; fail-and-recover fully inside a window sends nothing), a full
health pass against a real window, and the real router with a stubbed
session (role rules, duration bounds including the missing-duration case,
extend-not-stack, 404s, deleted targets hidden, audit entries). Real dev
database mtime untouched.
Not done: I haven't clicked through the new page or banner in a browser
(they sit behind the Authentik login); it builds and the API behind it is
tested.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A certificate secret's expiry was only ever what someone typed in, so
a renewed cert (or a wrong date) meant the app's reminders were
silently wrong. A certificate secret can now be given a host:port; the
app opens a real TLS connection and reads the certificate's actual
expiry — on create/edit (if the host changes), daily, and via a
per-row "Check now" — and keeps expiryDate in sync. Because the daily
refresh runs before the existing expiry check, the reminder is always
computed from what's actually being served.
Verification is deliberately off for the connection: homelab services
routinely serve self-signed/internal-CA certs, and an already-expired
one is exactly the case worth reporting, which a verifying connection
would refuse before exposing the dates.
Failure handling avoids the silent-staleness this is meant to fix: a
failed check keeps the last known date, records why on the row (shown
as a "Check failed" badge), and is listed in the daily secrets
notification. Creating a monitored secret whose host can't be reached
and with no manual date is rejected with the reason rather than saved
blank. A non-TLS port (the likeliest typo) gets a plain-language
error instead of raw OpenSSL output.
Server-side connections to a user-supplied host:port need the same
operator role that already gates editing secrets (and running Semaphore
templates, which is strictly more powerful); the host is validated
against a strict character set before any connection is made.
New nullable secrets columns (check_host, check_port, last_checked_at,
last_check_error) via migration 0007; existing rows are unaffected.
Verified against real TLS servers (openssl-generated certs) and the
real secrets router with a stubbed session: a live 45-day cert read
back as the correct date via both an IP host (no SNI) and a hostname;
an already-expired cert reported its past date and shows as expired;
refused connections, a server that accepts but never answers (times
out), and a plain non-TLS server each produced a descriptive error
rather than a hang or crash. Through the router: create with a host
and no date reads the date; unreachable host with no date -> 400 with
the reason; unreachable with a manual date -> saved with the error
recorded; host on a non-certificate type and an invalid host string
-> 400; a hand-typed date on a monitored secret is ignored; changing
the host re-checks immediately; changing the type away from
certificate ends monitoring; a viewer gets 403 on Check now. 23 checks,
all passing (a first re-run showed 2 spurious failures that were leftover
rows from the previous run's scratch database, confirmed by a clean re-run).
Real dev database mtime untouched throughout.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Instant notifications (DNS changes, integration failure/recovery
alerts) had no way to avoid pinging overnight. Adds a "Quiet hours"
window under Settings -> Notifications: notifications that would fire
during the window are held in a new notification_queue table instead
of sent immediately, then delivered as one combined digest at the end
time (in the same timezone already used for the daily checks) via a
new scheduled flush job. The scheduled daily checks (secret expiry,
Tailscale key, Docker updates, Proxmox backups) already only fire
once at a chosen time, so this mainly matters for the instant ones.
Includes a live "N queued" indicator with a manual "Flush now" button
for visibility, and correctly falls back to sending immediately
whenever the feature is disabled (the default).
Gating lives at the single choke point every notification already
flows through (notify()), so no per-event-type wiring was needed.
Verified against a fake webhook receiver on an isolated scratch
database: exhaustively checked the midnight-wraparound window math
(9 cases including exact-boundary inclusive/exclusive edges) against
synthetic "now" values rather than depending on when the test
happens to run, then end-to-end through the real notify()/
flushQuietHoursQueue() functions with a window constructed around the
actual current time — confirmed a notification during the window
queues instead of sending, the flush produces one digest with the
original title/message intact and clears the queue, a notification
outside the window sends immediately, and disabling the feature
entirely sends immediately regardless of the window. Confirmed the
real dev database's mtime was untouched throughout.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Not every registered server is a Proxmox VM/LXC -- bare-metal boxes
and other hosts had no reason to show a "link to Proxmox" option, but
it appeared unconditionally on every server's detail page.
New hideProxmoxLink column on servers (default false, so existing
behavior is unchanged until someone opts in). A "Not a VM? Hide this"
link in the card's header sets it; once hidden, a small "+ Show
Proxmox link options" link takes its place so it's still reachable,
not buried in a settings form. The card always shows regardless of
this flag once a server IS actually linked, so unlinking never becomes
unreachable by hiding the card out from under an active link.
Verified against a temp SQLite DB with real migrations: a new server
defaults to false, and toggling true/false both persist correctly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Each server's detail page gets an "Admin Links" card for bookmarking
that host's own web UIs -- container managers, Webmin, Cockpit, or
anything else reachable by URL -- so there's a quick way to jump
there without hunting down the address each time.
New server_links table (serverId FK, label, url, ON DELETE CASCADE so
removing a server cleans up its links automatically) and three new
routes: POST/PATCH/DELETE /api/servers/:id/links, gated to operator+
like the rest of this page's editing actions; the existing GET
/:id/detail now includes the server's links alongside hardware/DNS
info. URLs are validated to start with http:// or https:// server-side
(rejecting e.g. a javascript: URL that would otherwise render as a
clickable link).
Rendered as a row of pill buttons (label opens the URL in a new tab),
with inline edit/remove controls next to each when the viewer can
edit, and an "Add link" form matching the existing task-form style on
the same page.
Verified against a temp SQLite DB with real migrations: create/list/
update/delete all work, and deleting the parent server cascades to
remove its links rather than leaving them orphaned.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sloth Manager tracked every API call made to DNS providers for
connectivity troubleshooting. Port that here, generalized to cover
every outbound integration this app makes, not just DNS -- Tailscale,
Proxmox, Synology, Semaphore, Gitea, and Dockhand calls now show up
too, since a broken API token or unreachable host on any of them is
just as worth diagnosing.
New services/diagLog.ts: a generic withDiagLogging(source, adapter)
wraps every async method of any adapter object with timing +
success/failure recording, without touching a single adapter's
request/error-handling internals -- every DNS and integration adapter
interface here is already just a flat set of async methods, so this
one wrapper works for all twelve of them. Applied it at each adapter
factory's own return statement (one line each) rather than at the
route layer, so background jobs that construct adapters directly
(the Tailscale key-expiry scheduler, IPAM sync, agent-driven Proxmox
lookups) get logged too, not just requests through routes/integrations.ts.
New diag_log table (ring-buffered to the last 500 rows, mirroring
Sloth Manager's approach -- this is for live troubleshooting, not a
durable record) and admin-only GET/DELETE /api/diag-log routes, source/
result filters, pagination.
New admin-only Diagnostic Log page: filterable, paginated table with a
Clear button. Also introduces the shared useSortable hook + SortableTh
component used here for the first time -- a follow-up commit applies
the same sorting (and CSV export) to the rest of the app's tables, per
the same request.
Verified end-to-end against a temp SQLite DB with real migrations: a
fake wrapped adapter's successful and failing calls both land correctly
in the log with the right source/operation/latency/error, and the
source/ok filters and clear-log operation all behave correctly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
IPAM was entirely manual — Tailscale device IPs never showed up there
even though the Tailscale integration already lists them. Add an
explicit sync action (matching this app's existing pattern of
user-triggered syncs rather than silent background polling).
- New source column on ipam_entries (null = manual, "tailscale" =
auto-synced) so a re-sync only ever touches rows it created itself —
a manually-entered IP that happens to collide with a tailnet address
is left untouched and reported back as skipped, never overwritten.
- POST /api/ipam/sync-tailscale pulls every enabled Tailscale
integration's device list, upserting by primary IP (label, OS in
notes, vendor "Tailscale"); one unreachable Tailscale integration
doesn't block others.
- New "Sync from Tailscale" button on the IP Addresses page, with a
small "synced" badge marking which rows came from it.
Also fixed a longstanding TODO found in the same file: the "DNS
records" column always showed "-" because matchingDnsRecords was
hardcoded to an empty array from before the DNS module existed. It
now does the same content-based reverse lookup against the DNS
module's record cache used elsewhere in the app.
Verified end-to-end by running the real server with Tailscale's fetch
call intercepted at the process level (its adapter hardcodes
api.tailscale.com with no configurable URL, so it can't be pointed at
a mock server the way Proxmox/Synology can): confirmed add, the
manual-entry skip/never-overwrite behavior, idempotent re-sync
(add -> update), and the DNS-matching fix, all against the real
route and adapter code.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Servers & Tasks only tracked scheduled tasks — there was no overview of
the servers themselves and no way to see CPU/RAM/disk/IP info. Add a
clickable server overview (visible to every role, not just admins) that
opens a per-server detail page.
- servers table gains an agent-reported hardware/network snapshot
(IPs, CPU model/cores/load, memory, disks) and an optional link to a
Proxmox VM/LXC (integration + node + guest type + vmid).
- Proxmox adapter gains getGuestDetail(): live cores/memory/disk from
/config, live cpu/mem/uptime from /status/current, and IPs (LXC net
config directly, QEMU via a best-effort guest-agent call that degrades
gracefully when the agent isn't installed).
- New GET /api/servers/:id/detail combines whichever hardware source
applies (live Proxmox vs. last agent report) with a DNS reverse-lookup
against the DNS module's own record cache, so matching hostnames show
up next to each IP. New PATCH /api/servers/:id manages the Proxmox
link.
- agent/linux/report-tasks.sh now also collects and reports IPs,
CPU/memory/disk info on every check-in (load-average-based CPU number,
not instantaneous, to keep the agent a cheap oneshot).
- Extracted the per-server task table into a shared ServerTaskTable
component so the all-servers view and the new detail page render
tasks identically.
Verified server-side end-to-end against the real dev server (agent
report -> detail endpoint -> DNS match) and the Proxmox adapter against
a mock HTTPS server covering LXC/QEMU config parsing and the
guest-agent-unavailable fallback.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ports zone/record management across Cloudflare, Loopia, Pi-hole, Azure DNS,
cPanel, and Technitium onto the new stack. Unlike the original (one instance
per provider configured via env vars), providers are now configured through
the UI and support multiple named instances per type, with API
credentials encrypted at rest via the integration_credentials table.
- server/src/dns/adapters/*: each provider ported to a config-based factory
(no more process.env reads), preserving each provider's original quirks
(Pi-hole session auth, Azure record-set merging, Loopia XML-RPC, cPanel
UAPI/API2 fallbacks, Technitium composite record IDs).
- server/src/routes/dns.ts: provider CRUD (admin), a "test connection"
endpoint, and zone/record browsing+sync+CRUD (operator+), all audit-logged.
- dns_zones_cache/dns_records_cache tables replace Sloth Manager's
dns-cache.json file, keeping the same "cache is the source of truth for
display, sync fetches fresh from the provider" behavior.
- web: a DNS page with provider management, zone browsing, and a record
editor, plus a reusable dynamic provider-config form.
Verified: full build (tsc + vite) passes; a scripted HTTP-layer test against
a running server exercises auth, role gating (403 for viewer), validation
(400 on missing config fields), provider CRUD, credential non-leakage in
list responses, and adapter error propagation (502 against an unreachable
host) — all passing. Real provider connectivity still needs to be checked
against the user's actual DNS accounts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Monorepo (Express+TS+Drizzle/libSQL server, React+Vite+Tabler web) matching
the stack used by ScheduleTaskManager and Sloth Manager. Includes Authentik
OIDC login with local admin/operator/viewer roles (first user becomes admin),
a generalized audit log, encrypted-at-rest storage for future integration API
tokens, the DB schema for all planned modules, and the Tabler-styled app
shell/nav. Also ports the Secrets (expiry tracker) and IP Addresses (IPAM)
modules from Sloth Manager onto the new stack.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>