Each server card shows its fullest disk over the last week under the load and
memory lines (a disk moves too slowly to show in a day), and the Manage
servers table has a Load and disk column with all three side by side.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A job whose last three runs in a row all finished with warnings is flagged and
raises an alert (on by default, switchable off per job). The size of the
destination, and its quota where the storage says, are read from the reports
and kept with each run; the Stored column shows the growth and, where the room
is known, when it will be full, with an alert inside the disk-forecast window.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
An administrator can make named displays (Hall TV, Office monitor) under
Settings -> Share links, each with an address of its own (/wall/hall-tv), the
pages it rotates through in the board's order, and a rotation speed. The
signed-in wall opens one from its address and can switch between them, and a
wall share link can open one too. The address still wins over the display's
speed, then the browser's choice.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Server cards show load and memory over the last 24 hours, endpoint checks show
their answer times with a tick where a check failed, and integrations that keep
a figure history show their headline figure over the last week. Each list asks
for all its lines in one request, drawn from the history already kept, and each
line has a sentence for screen readers.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The numbers integrations report (queries and blocked share, queue and missing
items, speeds, photos, documents, failed runs, firing rules, targets down,
monthly cost and more) are kept about every quarter of an hour for five weeks,
then as hours and days for up to two years, taken from each integration's
stored reading. Every card on the 14 integration pages gets a Show the
history button with a chart per figure, how it changed and its lowest and
highest. A stretch when the integration could not be read is a gap.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
An update run can name a second Semaphore playbook that restarts the server
once the updates are in and it says a restart is due. The run keeps the
maintenance window up, follows the restart, waits for the server to report
again and checks from its uptime (or its restart flag clearing) that it did
start again. A server that only needs a restart has a Restart button on the
Updates page, and a patch plan can carry the restart playbook too.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A guest's drawer on the Proxmox page now has a Snapshots section. Everyone
signed in sees them, operators take them, administrators roll back and delete
(each behind a confirmation). Proxmox does each as a task, followed for up to
20 seconds. Audit log and change log entries for all three. An http:// Proxmox
address is now allowed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Notifications are written down with the alerts they were about, and the
incident timeline lists each one: which channels took it, which refused and
what they said, or why it was held or not sent. New alerts, recovery notices,
reminders and escalations are all covered.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The typical draws in the Inventory, the price of a kWh and the currency set
there give a cost per device and in all, counted in the totals under an
Electricity category when there is a draw, a price and a three-letter currency,
and said plainly (with what is missing) when there is not.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each change a read of a Fragnet account sees, and each power signal sent from
here with who sent it, is written down (180 days). A server's details show the
share of the last 30 days it was on, starts and stops, a seven-day bar and the
latest changes, with who asked for them.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Every notification is written down with how it went through each channel
(or why it was not sent: held for quiet hours, a routing rule said nobody, no
channel on). Records -> Notifications lists them with a filter, a search and
Send again. Errors are cleaned of every secret in the settings; kept 30 days.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A read of a Fragnet account that shows a server on (running or starting) after
it was off, or off (stopped or stopping) after it was on, sends one message
through the configured channels and routing rules, a line for each server.
A server first seen, one whose state is unknown and one being installed or
moved say nothing. Switch: Settings -> Notifications -> What to send (on by
default).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Duplicati posts its message as a form body by default (message=<URL-encoded
text>). Unwrap it (any field name, other fields beside it), take the job and
operation from the 'Duplicati Backup report for <job>' line, read JSON under
that line, and read the Warnings and Errors lists of the text form.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each machine's card shows whether the last report was a backup, another
operation that was set aside, or one that could not be read (with the reason),
and counts of each since it was added. Administrators can see the start of an
unreadable report.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The first twenty warning messages of each run are kept with the run and shown
on the job and in its history. A job can be set to raise a warning-level alert
when its last run finished with warnings (off by default; a muted job raises
nothing; maintenance on the machine's server silences it).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reads /api/admin/cluster/state (Technitium 14+): a Cluster table on the page,
alerts for a node unreachable for ten minutes (primary critical) and for a
secondary that has stopped taking the primary's configuration, and three
/metrics series. A server in no cluster shows nothing; a token that may not
view administration is told so and the rest of the read is unaffected.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Three tick boxes on the Fragnet integration: do not alert about disk space,
memory or backups. The page still shows the figures (a switched-off bar is blue
and the backups tile says not judged); /metrics is unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Settings -> Display -> Menu: when on, the menu, command palette and shortcuts
omit the pages of integration kinds nobody has added (off by default, the same
for everyone). Pages useful without an integration always stay; adding an
integration brings its page back at once.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
One tile per server, time() - homelab_server_boot_time_seconds, in the Servers
row and following the Server picker.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
When each agent-reported server last started (last report minus uptime), so
uptime is time() - the metric and a reboot shows as it moving.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Linux agent 8 and Windows agent 7 say how long the machine has been up; the
server stores it (migration 0086) and shows it as Up for in the status strip
(left out while offline) and Uptime on the server page.
The Integrations header wraps and its button no longer squeezes, and
.table-responsive is positioned so hidden text in scrolled-off columns cannot
widen the page.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Both send to one fixed address each (api.telegram.org, api.pushover.net), take part in routing rules and the channel count, have a Send test that shows the service's own refusal, and keep their tokens and keys encrypted and out of errors and the audit log. A Telegram message carries one button, and a Pushover message one link, to the page where an alert can be acknowledged, silenced or snoozed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each cost item's name, amount, currency, period and kind are kept as versions (cost_item_versions), and Hetzner's estimate is written down once a day (cost_snapshots), so a past month's figure and the changes between can be worked out. History card with monthly bars per currency, month-by-month figures and a list of added, changed and removed items.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
POST /api/integrations/:id/read-now reads one integration at once (operators and administrators; not again within 20 s): a kind with a stored reading is read and judged again, a kind read live is asked whether it answers. The three per-kind lists in the integrations routes (add, change, delete) are now one map in services/integrationSyncs.ts. Button on the Integrations cards and the Media managers, Download clients and Hetzner Robot pages.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Read every 15 minutes (Robot allows 200 requests an hour; read-only): each server's product, addresses, status, rescue system and paid-until, and the storage boxes. Adapter, sync service, hetzner_robot_state table, /api/hetzner-robot, Infrastructure → Hetzner Robot page, metrics and docs.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each client is read every five minutes (read-only; qBittorrent's sign-in is the only POST): speed, queue and item states, network connection or paused queue, free space, SABnzbd's failures and warnings. Shared adapter, sync service, download_state table, /api/downloads, metrics and docs.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Alert settings keeps Alert rules, Service levels and Alert tickets; administrators get a link to Settings -> Notifications.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Old addresses (/alert-rules, /service-levels, /settings/notifications) lead to the right tab.
Notifications and Alert tickets stay administrator-only.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Backups → Proxmox backups has a Backup coverage tab and a Proxmox Backup
Server tab (?view=pbs), each keeping its own title and buttons. /pbs redirects
to the second tab so old links and favourites still work; the sidebar's two
entries become one, and the Backup Server's alert and integration links go
to the tab. Docs updated.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A Change link on the Identity card sets the hostname (a host name or an
address, written lower case without the trailing dot). It is then kept even
when the agent reports another — marked "set by hand", with what the agent
reports shown beside it — until "Use what the agent reports" gives it up. The
agent's own hostname is always remembered (reported_hostname). PUT and DELETE
/api/servers/:id/hostname, administrators only, audited with from and to.
Migration 0082 adds hostname_pinned and reported_hostname and fills the latter
for servers that have reported. It changes this app's name for the server, not
the machine's.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A new integration type (Network, next to AdGuard Home) reads Technitium's API
every two minutes with a non-expiring API token in the Authorization header:
version and uptime, whether blocking is on, block lists and their next update,
the last day's queries and how they ended, cache and zone counts, the top
clients and names, zone health (expired or unsynced secondary zones), DHCP
leases — and puts one real DNS question to it over UDP. A Technitium page
shows it; alerts: DNS not answering (critical), blocking left off, block lists
overdue, zones expired or not syncing; /metrics gets homelab_technitium_*.
Nothing is changed on the server. Technitium's API has no CPU, memory or disk
figures; the page says a server agent covers that. New table technitium_state
(migration 0081).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The health pass keeps the host ports Dockhand says running containers
publish (new table server_container_ports, migration 0080), matched to a
server by its Docker host's name. They name the port on the server's Ports
card and the network Ports table, ahead of a port preset and behind a
label somebody wrote, and in the new-port alert ("8096/tcp jellyfin
(container)"). Never written onto port notes, so a container's port is
still a new port; an unreadable host keeps its names, a day-old name is
dropped. Dockhand is now read on the health pass even with container
alerts off.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Lidarr and Readarr join the shared *arr reader. Integrations get an Inspect button that
runs the reader once and shows what each request returned as shapes (never values) and
where the result is empty. Operations -> App versions compares each application's version
with its newest GitHub release (lookup off until an admin switches it on; optional alert).
The Alerts page can group by source, kind of problem or type of alert. The shared HTTP
helper no longer reuses connections (stale-socket resets). Migration 0078; docs and tests
included.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
New integration kinds adguard (status, statistics, filter lists and a real DNS question
over UDP; alerts for DNS down/not running, protection left off, stale lists) and grafana
(alert rules firing or failing, data-source health). Optional warning for running Docker
images built more than N days ago (off by default). A server's overview cards can be
folded and arranged, kept per user. Migrations 0075-0077; docs, metrics and tests included.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A "What to do" note (text and/or link) can be written once for a whole kind of alert and
shows beside every alert of that kind; alerts now carry their kind and source type.
New optional setting: warn when a Docker host has N or more high vulnerabilities (off by
default). The wall display gains tiles (Hetzner and game servers running, critical
vulnerabilities, stuck downloads) and a summary line for each group of integrations,
also in share links. Migration 0074; docs and tests included.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
New integration kinds sonarr, radarr and prowlarr (shared adapter and sync, page under
Apps -> Media managers): own health findings, stuck downloads, nearly full disks.
Hetzner now reads load balancers (target health), floating IPs and certificates, includes
them in the cost estimate, and forecasts the month's traffic against the quota.
Migrations 0072 and 0073; docs, metrics and tests included.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
New integration kind "fragnet" reading Fragnet's Pterodactyl-based panel through the
client API (account key). Game servers page under Infrastructure with game, IP:port,
state, memory/disk, backups, schedules and databases; alerts for late backups and
nearly full disk/memory; operators can start/stop/restart/kill (audited). Metrics,
docs and tests included; migration 0071 adds fragnet_state.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Speedtest Tracker integration: charts, alerts for a slow line (median of the
last three tests under a share of what is paid for) and for failing tests.
- Costs page (Records): things paid for by hand plus Hetzner's estimate, totals
per currency, bills rolled forward into Coming up as Renewals.
- Status pages: a "Having trouble" state for things that answer but have an open
alert; Speedtest reads as the internet.
- Integrations page and kind picker grouped like the sidebar (one shared types
file); Notifications "What to send" shown one group at a time; Dashboard
integration sections follow the menu groups, with tiles for integrations that
have no widget.
- Docs, tests, and ROLES.md menu table brought up to date.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Closing several alerts together can now also close their osTicket tickets
(one by one, each written on its incident and in the audit log; one that
fails does not stop the rest; Undo does not reopen tickets). The single
and bulk close share one helper. Alerts moves to the top of the sidebar,
and Monthly report and Change log move to a new Records group. Docs updated.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reference holds Map, Inventory, Admin Links and Generator, leaving
Infrastructure with live systems only. Docs and UI text name the new
locations.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
One integration per project (read-only API token, read every 5 minutes).
Alerts for a server that was running and is off, stuck statuses, traffic
at 80%/100% of the monthly quota, rescue mode left on, and late backups.
The Hetzner page lists servers, volumes, snapshots and a cost estimate; a
server the app knows by its agent gets a Hetzner card on its own page.
/metrics and docs updated.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reads what Dockhand's scans have already stored (never starts one) every
15 minutes and raises a warning per Docker host with critical findings.
Unknown counts and stale readings are held, not treated as clean.
Settings -> Notifications has a switch (on by default); the Docker page
shows counts, scan coverage and the critical findings; /metrics and docs
updated.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Nextcloud: its own "Cron last run" setup check (GET /settings/setupcheck,
Nextcloud 28+, found by class name) raises an alert when background jobs have
stopped; other checks that report an error are one alert, warnings are shown
on the page. The core version Nextcloud says is available is shown too. Before
28, cron and the checks are unknown, never "fine".
Alerts page: bulk Close... (one reason, written on each incident, undoable) and
Open again. Bulk Wake now leaves closed alerts alone.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The Close drawer can also close the alert's osTicket ticket. osTicket's API
cannot, so this writes to its database with a separate, optional login: the
ticket is set to Closed, a note in its thread says who closed it and why, and
the closed event is recorded, in one transaction. Tables and columns are
checked first, only an open ticket is closed, and a ticket closed by somebody
else at the same moment is left alone. Summary tickets are not closed from here.
"Test closing" (Alert tickets settings) tries every write and rolls it back.
If the ticket cannot be closed the alert is still closed and the reason is
said, noted on the incident and audited.
Not run against a real osTicket: tested against a pretend database only.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Close... on the Alerts page hides an alert with no end, with an optional reason
that is also written on its incident. It opens again only if it gets more
serious than when it was closed (said once) or the problem has been gone for
half an hour; Open again brings it back at once.
Built on the snooze table (kind, severity and note columns), so everything that
holds back snoozed alerts holds back closed ones. Audit log, change log and
/api/v1 alerts carry it.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Three read-only integrations, each with a monitoring page, alerts, maintenance
silencing and held-when-unreadable handling.
Nextcloud: database needing upgrade (critical), maintenance mode, and the data
disk running low; users, files and app updates with an admin app password.
NetBox: no background worker running, counts, and a list of the servers this
app manages that have no address in NetBox (a list, not an alert).
n8n: an active workflow whose last run failed, clearing when it next succeeds.
Disk messages for Immich, Paperless and Nextcloud now use the same units as
the pages that show them.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Bulk snooze: Snooze... and Wake on the ticked alerts, with undo that puts back
each alert's earlier snooze exactly.
Alert rules are judged for a server right after its agent reports, instead of at
the next 15-minute pass; only that server's rule findings change, and the check
is serialized with the pass so neither overwrites the other's state.
Photos on server and device notes: up to 12 each, shrunk in the browser,
accepted by their own bytes (JPEG/PNG/GIF/WebP, 4 MB), kept in the database and
removed with the server or device.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Three read-only integrations, each with a monitoring page, alerts, maintenance
silencing and held-when-unreadable handling.
Prometheus: scrape targets that stop answering (5 minutes, one alert per job)
and alerts that Prometheus's own rules fire (critical when labelled so).
Immich: disk under the library filling, failed background jobs, counts.
Paperless-ngx: worker/database/Redis in error, documents that failed to
import, index and sanity check, disk. Immich and Paperless also in /metrics.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>