Add maintenance mode to silence alerts while working on a server or integration

Rebooting Proxmox or patching a server triggered failure/offline alerts
you then had to dismiss. A maintenance window silences alerts about one
server, integration, or DNS provider for a chosen time. New Maintenance
page (start with a duration and optional reason, end early, see what's
silenced and what isn't) and a banner in the app shell so every signed-in
user can see what is currently silenced. Starting/ending is operator-only
and audit-logged; starting one on a target that already has a window
restarts its clock instead of stacking.

Silenced for the target: server offline/disk alerts, Proxmox/Synology
storage and health alerts, Proxmox backup alerts, and "integration down"
alerts. Not silenced: expiry and update reminders, DNS change notices.

The design goal is that this cannot hide a real outage:
- Every window has a required end (5 min to 7 days); there is no
  open-ended option, so a forgotten window expires by itself.
- A silenced problem is deliberately NOT recorded as "known". If it is
  still present when the window ends it alerts then, as new. A problem
  that was already alerted before the window stays known, so it isn't
  repeated, and is reported cleared only after the window ends.
- Failure alerts keep counting failures during a window without marking
  themselves alerted, so an outage that outlasts the window alerts on the
  very next failed call.

Known limitation, stated on the page: integration-failure alerts are
tracked per service TYPE (all "proxmox"), not per configured instance, so
a window on one Proxmox integration also silences a failure on a second
Proxmox integration while it's open. Fixing that means threading the
integration id through every adapter and the diagnostic log, which is a
much larger change than this feature.

Also moved the API-error-message helper out of Secrets.tsx into a shared
util now that two pages use it. New table maintenance_windows (migration
0008).

Verified with 44 checks: the condition-key-to-subject mapping (including
server:3 vs server:33), the diff rules with silenced subjects (new problem
not recorded, alerts when the window ends; already-known one carried and
not repeated; clears only after the window), window expiry and
integration/DNS-provider source matching, the failure tracker end to end
against a webhook (silent during a window while an unrelated service still
alerts; outage that outlasts the window alerts on the next failure and
only once; fail-and-recover fully inside a window sends nothing), a full
health pass against a real window, and the real router with a stubbed
session (role rules, duration bounds including the missing-duration case,
extend-not-stack, 404s, deleted targets hidden, audit entries). Real dev
database mtime untouched.

Not done: I haven't clicked through the new page or banner in a browser
(they sit behind the Authentik login); it builds and the API behind it is
tested.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
bobbanandClaude Sonnet 5 committed 2026-09-26 02:23:06 +02:00
1 parent 1688de3ea2
commit aae4f0d74f
18 files changed
+1854 -26

No files matched your search

+41 -4
View File
@@ -1,5 +1,5 @@
import { useState, type ReactNode } from "react";
import { NavLink } from "react-router-dom";
import { useEffect, useState, type ReactNode } from "react";
import { Link, NavLink } from "react-router-dom";
import {
IconLayoutDashboard,
IconServer2,
@@ -22,8 +22,10 @@ import {
IconSun,
IconMoon,
IconWand,
IconTool,
} from "@tabler/icons-react";
import type { CurrentUser } from "../api/client";
import { api, type CurrentUser, type MaintenanceWindow } from "../api/client";
import { formatRemaining } from "../utils/duration";
import { getTheme, setTheme, type Theme } from "../utils/theme";
import CommandPalette from "../components/CommandPalette";
@@ -48,6 +50,7 @@ const NAV_ITEMS: NavItem[] = [
{ to: "/gitea", label: "Gitea", icon: <IconBrandGit size={20} /> },
{ to: "/integrations", label: "Integrations", icon: <IconPlugConnected size={20} /> },
{ to: "/generator", label: "Generator", icon: <IconWand size={20} /> },
{ to: "/maintenance", label: "Maintenance", icon: <IconTool size={20} /> },
{ to: "/users", label: "Users", icon: <IconUsers size={20} />, minRole: "admin" },
{ to: "/sessions", label: "Sessions", icon: <IconDevices size={20} />, minRole: "admin" },
{ to: "/audit-log", label: "Audit Log", icon: <IconHistory size={20} />, minRole: "operator" },
@@ -60,6 +63,28 @@ const roleRank: Record<CurrentUser["role"], number> = { viewer: 0, operator: 1,
export default function AppShell({ user, children }: { user: CurrentUser; children: ReactNode }) {
const [sidebarOpen, setSidebarOpen] = useState(false);
const [theme, setThemeState] = useState<Theme>(() => getTheme());
const [maintenance, setMaintenance] = useState<MaintenanceWindow[]>([]);
// Everyone sees what's currently silenced, so nobody is left wondering why an alert never came.
useEffect(() => {
let cancelled = false;
const load = () =>
api.maintenance
.list()
.then((res) => {
if (!cancelled) setMaintenance(res.windows);
})
.catch(() => {});
load();
const poll = setInterval(load, 60_000);
window.addEventListener("maintenance-changed", load);
return () => {
cancelled = true;
clearInterval(poll);
window.removeEventListener("maintenance-changed", load);
};
}, []);
const visibleItems = NAV_ITEMS.filter(
(item) => !item.minRole || roleRank[user.role] >= roleRank[item.minRole],
);
@@ -126,7 +151,19 @@ export default function AppShell({ user, children }: { user: CurrentUser; childr
<div className="page-wrapper">
<div className="page-body">
<div className="container-xl">{children}</div>
<div className="container-xl">
{maintenance.length > 0 && (
<div className="alert alert-warning d-flex align-items-center gap-2">
<IconTool size={18} />
<div>
Maintenance — alerts silenced for{" "}
{maintenance.map((w) => `${w.targetName} (${formatRemaining(w.endsAt)} left)`).join(", ")}.{" "}
<Link to="/maintenance">Manage</Link>
</div>
</div>
)}
{children}
</div>
</div>
</div>
</div>