Add maintenance mode to silence alerts while working on a server or integration

Rebooting Proxmox or patching a server triggered failure/offline alerts
you then had to dismiss. A maintenance window silences alerts about one
server, integration, or DNS provider for a chosen time. New Maintenance
page (start with a duration and optional reason, end early, see what's
silenced and what isn't) and a banner in the app shell so every signed-in
user can see what is currently silenced. Starting/ending is operator-only
and audit-logged; starting one on a target that already has a window
restarts its clock instead of stacking.

Silenced for the target: server offline/disk alerts, Proxmox/Synology
storage and health alerts, Proxmox backup alerts, and "integration down"
alerts. Not silenced: expiry and update reminders, DNS change notices.

The design goal is that this cannot hide a real outage:
- Every window has a required end (5 min to 7 days); there is no
  open-ended option, so a forgotten window expires by itself.
- A silenced problem is deliberately NOT recorded as "known". If it is
  still present when the window ends it alerts then, as new. A problem
  that was already alerted before the window stays known, so it isn't
  repeated, and is reported cleared only after the window ends.
- Failure alerts keep counting failures during a window without marking
  themselves alerted, so an outage that outlasts the window alerts on the
  very next failed call.

Known limitation, stated on the page: integration-failure alerts are
tracked per service TYPE (all "proxmox"), not per configured instance, so
a window on one Proxmox integration also silences a failure on a second
Proxmox integration while it's open. Fixing that means threading the
integration id through every adapter and the diagnostic log, which is a
much larger change than this feature.

Also moved the API-error-message helper out of Secrets.tsx into a shared
util now that two pages use it. New table maintenance_windows (migration
0008).

Verified with 44 checks: the condition-key-to-subject mapping (including
server:3 vs server:33), the diff rules with silenced subjects (new problem
not recorded, alerts when the window ends; already-known one carried and
not repeated; clears only after the window), window expiry and
integration/DNS-provider source matching, the failure tracker end to end
against a webhook (silent during a window while an unrelated service still
alerts; outage that outlasts the window alerts on the next failure and
only once; fail-and-recover fully inside a window sends nothing), a full
health pass against a real window, and the real router with a stubbed
session (role rules, duration bounds including the missing-duration case,
extend-not-stack, 404s, deleted targets hidden, audit entries). Real dev
database mtime untouched.

Not done: I haven't clicked through the new page or banner in a browser
(they sit behind the Authentik login); it builds and the API behind it is
tested.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
bobbanandClaude Sonnet 5 committed 2026-09-26 02:23:06 +02:00
1 parent 1688de3ea2
commit aae4f0d74f
18 files changed
+1854 -26

No files matched your search

+8
View File
@@ -142,6 +142,14 @@ raise one notification when the problem starts and one when it clears (both
thresholds are set under Settings → Notifications). Active problems are thresholds are set under Settings → Notifications). Active problems are
remembered across restarts, so a rebuild doesn't re-alert them. remembered across restarts, so a rebuild doesn't re-alert them.
**Maintenance mode** silences alerts about one server, integration, or DNS
provider while you work on it (server offline / disk, storage and Synology
health, Proxmox backup alerts, and "integration down" for that service type).
Every window has a fixed end (5 minutes to 7 days) and expires on its own, and
a problem that began during a window and is still present when it ends alerts
then — a forgotten window can't hide an outage. A banner shows what's currently
silenced to every signed-in user.
The app is installable as a PWA — "Install app" / "Add to Home Screen" from the The app is installable as a PWA — "Install app" / "Add to Home Screen" from the
browser gives it its own icon and a standalone window on phone or desktop. This browser gives it its own icon and a standalone window on phone or desktop. This
needs the site to be served over HTTPS (browsers only offer install on secure needs the site to be served over HTTPS (browsers only offer install on secure
+9
View File
@@ -0,0 +1,9 @@
CREATE TABLE `maintenance_windows` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`target_type` text NOT NULL,
`target_id` integer NOT NULL,
`reason` text,
`started_at` text NOT NULL,
`ends_at` text NOT NULL,
`created_by` text
);
File diff suppressed because it is too large. Load diff
+7
View File
@@ -57,6 +57,13 @@
"when": 1790372043652, "when": 1790372043652,
"tag": "0007_secret_madripoor", "tag": "0007_secret_madripoor",
"breakpoints": true "breakpoints": true
},
{
"idx": 8,
"version": "6",
"when": 1790381917814,
"tag": "0008_thin_boom_boom",
"breakpoints": true
} }
] ]
} }
+15
View File
@@ -59,6 +59,21 @@ export const notificationQueue = sqliteTable("notification_queue", {
.default(sql`(current_timestamp)`), .default(sql`(current_timestamp)`),
}); });
// ─── Maintenance windows — alerts about a target are silenced until endsAt ──
export const maintenanceTargetTypes = ["server", "integration", "dns_provider"] as const;
export type MaintenanceTargetType = (typeof maintenanceTargetTypes)[number];
export const maintenanceWindows = sqliteTable("maintenance_windows", {
id: integer("id").primaryKey({ autoIncrement: true }),
targetType: text("target_type").$type<MaintenanceTargetType>().notNull(),
targetId: integer("target_id").notNull(), // polymorphic — no FK; a deleted target's window is simply ignored
reason: text("reason"),
startedAt: text("started_at").notNull(), // ISO
endsAt: text("ends_at").notNull(), // ISO — required: a forgotten open-ended window would silence real problems forever
createdBy: text("created_by"),
});
// ─── Settings (key/value) ─────────────────────────────────────────────────── // ─── Settings (key/value) ───────────────────────────────────────────────────
export const settings = sqliteTable("settings", { export const settings = sqliteTable("settings", {
+2
View File
@@ -23,6 +23,7 @@ import { integrationsRouter } from "./routes/integrations.js";
import { settingsRouter } from "./routes/settings.js"; import { settingsRouter } from "./routes/settings.js";
import { searchRouter } from "./routes/search.js"; import { searchRouter } from "./routes/search.js";
import { sessionsRouter } from "./routes/sessions.js"; import { sessionsRouter } from "./routes/sessions.js";
import { maintenanceRouter } from "./routes/maintenance.js";
import { initSecretExpiryScheduler } from "./services/secretExpiryScheduler.js"; import { initSecretExpiryScheduler } from "./services/secretExpiryScheduler.js";
import { initTailscaleKeyExpiryScheduler } from "./services/tailscaleKeyExpiryScheduler.js"; import { initTailscaleKeyExpiryScheduler } from "./services/tailscaleKeyExpiryScheduler.js";
import { initLogRetentionScheduler } from "./services/logRetentionScheduler.js"; import { initLogRetentionScheduler } from "./services/logRetentionScheduler.js";
@@ -90,6 +91,7 @@ app.use("/api/integrations", integrationsRouter);
app.use("/api/settings", settingsRouter); app.use("/api/settings", settingsRouter);
app.use("/api/search", searchRouter); app.use("/api/search", searchRouter);
app.use("/api/sessions", sessionsRouter); app.use("/api/sessions", sessionsRouter);
app.use("/api/maintenance", maintenanceRouter);
if (existsSync(webDist)) { if (existsSync(webDist)) {
app.use(express.static(webDist)); app.use(express.static(webDist));
+101
View File
@@ -0,0 +1,101 @@
import { Router } from "express";
import { eq, lt } from "drizzle-orm";
import { z } from "zod";
import { db } from "../db/client.js";
import { dnsProviders, integrations, maintenanceTargetTypes, maintenanceWindows, servers } from "../db/schema.js";
import { requireAuth, requireRole } from "../auth/middleware.js";
import { recordAudit } from "../services/audit.js";
import { describeTarget, listActiveWindows } from "../services/maintenance.js";
import { asyncHandler } from "../utils/asyncHandler.js";
export const maintenanceRouter = Router();
// Anyone signed in can see what's currently silenced (so nobody is surprised by missing alerts);
// starting and ending a window is an operator action, like the other things that change how the homelab behaves.
maintenanceRouter.use(requireAuth);
maintenanceRouter.get("/", asyncHandler(async (_req, res) => {
const windows = [];
for (const w of await listActiveWindows()) {
const target = await describeTarget(w.targetType, w.targetId);
if (!target) continue; // target was deleted — nothing left to silence
windows.push({ ...w, targetName: target.name, targetKind: target.kind });
}
windows.sort((a, b) => a.endsAt.localeCompare(b.endsAt));
const [serverRows, integrationRows, providerRows] = await Promise.all([
db.select({ id: servers.id, name: servers.name }).from(servers).orderBy(servers.name),
db.select({ id: integrations.id, name: integrations.name, type: integrations.type }).from(integrations).orderBy(integrations.name),
db.select({ id: dnsProviders.id, name: dnsProviders.name, providerType: dnsProviders.providerType }).from(dnsProviders).orderBy(dnsProviders.name),
]);
res.json({ windows, targets: { servers: serverRows, integrations: integrationRows, dnsProviders: providerRows } });
}));
const startSchema = z.object({
targetType: z.enum(maintenanceTargetTypes),
targetId: z.number().int().positive(),
// Required and capped: an open-ended window is the way this feature could quietly hide a real outage.
minutes: z.number().int().min(5).max(7 * 24 * 60),
reason: z.string().max(200).optional(),
});
maintenanceRouter.post("/", requireRole("operator"), asyncHandler(async (req, res) => {
const parsed = startSchema.safeParse(req.body);
if (!parsed.success) {
return res.status(400).json({ error: "invalid_body", details: parsed.error.flatten() });
}
const { targetType, targetId, minutes, reason } = parsed.data;
const target = await describeTarget(targetType, targetId);
if (!target) {
return res.status(404).json({ error: "not_found", message: "That target doesn't exist." });
}
const now = new Date();
const endsAt = new Date(now.getTime() + minutes * 60_000).toISOString();
const actor = req.currentUser!;
const createdBy = actor.name ?? actor.email ?? actor.oidcSub;
// Starting maintenance on something already in maintenance restarts its clock rather than stacking windows.
const existing = (await listActiveWindows(now)).find((w) => w.targetType === targetType && w.targetId === targetId);
let window;
if (existing) {
[window] = await db.update(maintenanceWindows).set({ endsAt, reason: reason ?? null, createdBy }).where(eq(maintenanceWindows.id, existing.id)).returning();
} else {
// Long-expired rows are just clutter; clear them out whenever a new one is added.
await db.delete(maintenanceWindows).where(lt(maintenanceWindows.endsAt, new Date(now.getTime() - 24 * 3600_000).toISOString()));
[window] = await db.insert(maintenanceWindows).values({ targetType, targetId, reason: reason ?? null, startedAt: now.toISOString(), endsAt, createdBy }).returning();
}
await recordAudit({
actor,
category: "maintenance",
action: existing ? "extend" : "start",
targetType,
targetId,
detail: { name: target.name, minutes, reason: reason ?? null },
});
res.status(201).json({ window: { ...window, targetName: target.name, targetKind: target.kind } });
}));
maintenanceRouter.delete("/:id", requireRole("operator"), asyncHandler(async (req, res) => {
const id = Number(req.params.id);
const [existing] = await db.select().from(maintenanceWindows).where(eq(maintenanceWindows.id, id)).limit(1);
if (!existing) {
return res.status(404).json({ error: "not_found" });
}
// Ending moves the end to now, so it stops applying immediately; the row is pruned later like any expired one.
await db.update(maintenanceWindows).set({ endsAt: new Date().toISOString() }).where(eq(maintenanceWindows.id, id));
const target = await describeTarget(existing.targetType, existing.targetId);
await recordAudit({
actor: req.currentUser!,
category: "maintenance",
action: "end",
targetType: existing.targetType,
targetId: existing.targetId,
detail: { name: target?.name ?? null },
});
res.status(204).end();
}));
+17 -5
View File
@@ -6,6 +6,7 @@ import { createProxmoxAdapter, type ProxmoxNodeStats } from "../integrations/pro
import { createSynologyAdapter, type SynologyStorageInfo } from "../integrations/synology/adapter.js"; import { createSynologyAdapter, type SynologyStorageInfo } from "../integrations/synology/adapter.js";
import { notifyHealthIssues, notifyHealthRecovered } from "./notify.js"; import { notifyHealthIssues, notifyHealthRecovered } from "./notify.js";
import { getInternalFlag, getSettings, setInternalFlag } from "./settingsStore.js"; import { getInternalFlag, getSettings, setInternalFlag } from "./settingsStore.js";
import { activeSubjects, subjectOfConditionKey } from "./maintenance.js";
const STATE_FLAG = "healthActiveConditions"; const STATE_FLAG = "healthActiveConditions";
/** After a restart the agents haven't had a chance to report yet (they run every 15 min), so server-derived conditions are held rather than judged. */ /** After a restart the agents haven't had a chance to report yet (they run every 15 min), so server-derived conditions are held rather than judged. */
@@ -216,15 +217,26 @@ export function diffConditions(
previous: ActiveState, previous: ActiveState,
current: HealthCondition[], current: HealthCondition[],
held: Set<string> = new Set(), held: Set<string> = new Set(),
/** Subjects ("server:3", "integration:1") under an active maintenance window. */
silenced: Set<string> = new Set(),
): { added: HealthCondition[]; resolved: { key: string; message: string }[]; next: ActiveState } { ): { added: HealthCondition[]; resolved: { key: string; message: string }[]; next: ActiveState } {
const next: ActiveState = {}; const isSilenced = (key: string) => {
for (const c of current) next[c.key] = { message: c.message, source: c.source }; const subject = subjectOfConditionKey(key);
return subject !== null && silenced.has(subject);
};
// A silenced problem is not recorded as "known": if it's still there when the window ends it must
// alert then, as new. (One that was already alerted before the window stays known, so it isn't repeated.)
const visible = current.filter((c) => !isSilenced(c.key));
const added = current.filter((c) => !(c.key in previous)); const next: ActiveState = {};
for (const c of visible) next[c.key] = { message: c.message, source: c.source };
const added = visible.filter((c) => !(c.key in previous));
const resolved: { key: string; message: string }[] = []; const resolved: { key: string; message: string }[] = [];
for (const [key, entry] of Object.entries(previous)) { for (const [key, entry] of Object.entries(previous)) {
if (key in next) continue; if (key in next) continue;
if (isHeld(entry.source, held)) { // Held (source unreadable) or silenced: leave it exactly as it was — neither cleared nor re-alerted.
if (isHeld(entry.source, held) || isSilenced(key)) {
next[key] = entry; next[key] = entry;
} else { } else {
resolved.push({ key, message: entry.message }); resolved.push({ key, message: entry.message });
@@ -296,7 +308,7 @@ export async function runHealthCheck(now: number = Date.now()): Promise<{ added:
if (inGrace) held.add("server"); if (inGrace) held.add("server");
const current = evaluateHealth(snapshot, healthChecks, now, { skipServers: inGrace }); const current = evaluateHealth(snapshot, healthChecks, now, { skipServers: inGrace });
const { added, resolved, next } = diffConditions(await loadState(), current, held); const { added, resolved, next } = diffConditions(await loadState(), current, held, await activeSubjects(new Date(now)));
await setInternalFlag(STATE_FLAG, JSON.stringify(next)); await setInternalFlag(STATE_FLAG, JSON.stringify(next));
// State is tracked even when the alert toggle is off (the notify functions check it), // State is tracked even when the alert toggle is off (the notify functions check it),
@@ -1,4 +1,5 @@
import { getSettings } from "./settingsStore.js"; import { getSettings } from "./settingsStore.js";
import { isSourceInMaintenance } from "./maintenance.js";
import { notifyIntegrationDown, notifyIntegrationRecovered } from "./notify.js"; import { notifyIntegrationDown, notifyIntegrationRecovered } from "./notify.js";
interface SourceHealth { interface SourceHealth {
@@ -31,8 +32,12 @@ export async function trackIntegrationHealth(source: string, ok: boolean): Promi
state.consecutiveFailures += 1; state.consecutiveFailures += 1;
const { notifications } = await getSettings(); const { notifications } = await getSettings();
if (notifications.integrationFailureAlerts && !state.alerted && state.consecutiveFailures >= notifications.integrationFailureThreshold) { if (notifications.integrationFailureAlerts && !state.alerted && state.consecutiveFailures >= notifications.integrationFailureThreshold) {
state.alerted = true; // During maintenance the failures keep being counted but `alerted` stays false, so if the service is
await notifyIntegrationDown(source, state.consecutiveFailures); // still failing once the window ends, the very next failed call alerts — a real outage isn't swallowed.
if (!(await isSourceInMaintenance(source))) {
state.alerted = true;
await notifyIntegrationDown(source, state.consecutiveFailures);
}
} }
health.set(source, state); health.set(source, state);
} }
+71
View File
@@ -0,0 +1,71 @@
import { eq, gt } from "drizzle-orm";
import { db } from "../db/client.js";
import { dnsProviders, integrations, maintenanceWindows, servers } from "../db/schema.js";
export type ActiveWindow = typeof maintenanceWindows.$inferSelect;
/** Windows whose end is still in the future. ISO strings compare correctly as text, so this is a plain string comparison. */
export async function listActiveWindows(now: Date = new Date()): Promise<ActiveWindow[]> {
return db.select().from(maintenanceWindows).where(gt(maintenanceWindows.endsAt, now.toISOString()));
}
/**
* The identity a health condition or alert belongs to, as "server:3" /
* "integration:1" / "dns_provider:2", so it can be matched against active windows.
* Derived from the condition key (which already encodes it) rather than stored,
* so conditions persisted by an earlier version still map correctly.
*/
export function subjectOfConditionKey(key: string): string | null {
let m = /^(?:offline|disk):server:(\d+)(?::|$)/.exec(key);
if (m) return `server:${m[1]}`;
m = /^disk:proxmox:(\d+):/.exec(key);
if (m) return `integration:${m[1]}`;
m = /^(?:synology-volume|synology-disk|disk:synology):(\d+):/.exec(key);
if (m) return `integration:${m[1]}`;
return null;
}
export async function activeSubjects(now: Date = new Date()): Promise<Set<string>> {
return new Set((await listActiveWindows(now)).map((w) => `${w.targetType}:${w.targetId}`));
}
export async function isInMaintenance(targetType: ActiveWindow["targetType"], targetId: number, now: Date = new Date()): Promise<boolean> {
return (await activeSubjects(now)).has(`${targetType}:${targetId}`);
}
/**
* Integration-failure alerts are tracked per service TYPE ("proxmox",
* "cloudflare", ...), not per configured instance, so a window on any
* integration or DNS provider of that type silences that type's failure alerts.
* (With two integrations of one type, a real failure on the un-windowed one is
* silenced too while the window is open — a known consequence of that
* granularity, stated on the Maintenance page.)
*/
export async function isSourceInMaintenance(source: string, now: Date = new Date()): Promise<boolean> {
const windows = await listActiveWindows(now);
if (windows.length === 0) return false;
for (const w of windows) {
if (w.targetType === "integration") {
const [row] = await db.select({ type: integrations.type }).from(integrations).where(eq(integrations.id, w.targetId)).limit(1);
if (row?.type === source) return true;
} else if (w.targetType === "dns_provider") {
const [row] = await db.select({ type: dnsProviders.providerType }).from(dnsProviders).where(eq(dnsProviders.id, w.targetId)).limit(1);
if (row?.type === source) return true;
}
}
return false;
}
/** Display name for a window's target, or null if the target no longer exists. */
export async function describeTarget(targetType: ActiveWindow["targetType"], targetId: number): Promise<{ name: string; kind: string } | null> {
if (targetType === "server") {
const [r] = await db.select({ name: servers.name }).from(servers).where(eq(servers.id, targetId)).limit(1);
return r ? { name: r.name, kind: "Server" } : null;
}
if (targetType === "integration") {
const [r] = await db.select({ name: integrations.name, type: integrations.type }).from(integrations).where(eq(integrations.id, targetId)).limit(1);
return r ? { name: r.name, kind: `Integration (${r.type})` } : null;
}
const [r] = await db.select({ name: dnsProviders.name, type: dnsProviders.providerType }).from(dnsProviders).where(eq(dnsProviders.id, targetId)).limit(1);
return r ? { name: r.name, kind: `DNS provider (${r.type})` } : null;
}
@@ -6,6 +6,7 @@ import { loadIntegrationConfig } from "../integrations/loadIntegration.js";
import { createProxmoxAdapter, guestsWithoutBackupCoverage, type ProxmoxBackupTask } from "../integrations/proxmox/adapter.js"; import { createProxmoxAdapter, guestsWithoutBackupCoverage, type ProxmoxBackupTask } from "../integrations/proxmox/adapter.js";
import { notifyProxmoxBackupFailure, notifyProxmoxUncoveredGuests } from "./notify.js"; import { notifyProxmoxBackupFailure, notifyProxmoxUncoveredGuests } from "./notify.js";
import { getSettings, getInternalFlag, setInternalFlag } from "./settingsStore.js"; import { getSettings, getInternalFlag, setInternalFlag } from "./settingsStore.js";
import { isInMaintenance } from "./maintenance.js";
const LAST_RUN_FLAG = "proxmoxBackupCheckLastRunDate"; const LAST_RUN_FLAG = "proxmoxBackupCheckLastRunDate";
@@ -19,6 +20,8 @@ async function checkProxmoxBackups(): Promise<void> {
const uncovered: { integrationName: string; guestName: string; vmid: number; node: string }[] = []; const uncovered: { integrationName: string; guestName: string; vmid: number; node: string }[] = [];
for (const row of rows) { for (const row of rows) {
// A host being worked on can't run or report backups; this check is daily, so tomorrow's pass covers it.
if (await isInMaintenance("integration", row.id)) continue;
try { try {
const loaded = await loadIntegrationConfig(row.id); const loaded = await loadIntegrationConfig(row.id);
if (!loaded) continue; if (!loaded) continue;
+2
View File
@@ -20,6 +20,7 @@ import Gitea from "./pages/Gitea";
import Proxmox from "./pages/Proxmox"; import Proxmox from "./pages/Proxmox";
import Synology from "./pages/Synology"; import Synology from "./pages/Synology";
import Generator from "./pages/Generator"; import Generator from "./pages/Generator";
import Maintenance from "./pages/Maintenance";
import Settings from "./pages/Settings"; import Settings from "./pages/Settings";
import NotificationSettings from "./pages/settings/NotificationSettings"; import NotificationSettings from "./pages/settings/NotificationSettings";
import BadgeSettings from "./pages/settings/BadgeSettings"; import BadgeSettings from "./pages/settings/BadgeSettings";
@@ -88,6 +89,7 @@ export default function App() {
<Route path="/secrets" element={<Secrets user={user} />} /> <Route path="/secrets" element={<Secrets user={user} />} />
<Route path="/integrations" element={<Integrations user={user} />} /> <Route path="/integrations" element={<Integrations user={user} />} />
<Route path="/generator" element={<Generator />} /> <Route path="/generator" element={<Generator />} />
<Route path="/maintenance" element={<Maintenance user={user} />} />
<Route path="/docker" element={<Docker user={user} />} /> <Route path="/docker" element={<Docker user={user} />} />
<Route path="/tailscale" element={<Tailscale user={user} />} /> <Route path="/tailscale" element={<Tailscale user={user} />} />
<Route path="/semaphore" element={<Semaphore user={user} />} /> <Route path="/semaphore" element={<Semaphore user={user} />} />
+26
View File
@@ -30,6 +30,26 @@ export interface SessionSummary {
expiresAt: string | null; expiresAt: string | null;
} }
export type MaintenanceTargetType = "server" | "integration" | "dns_provider";
export interface MaintenanceWindow {
id: number;
targetType: MaintenanceTargetType;
targetId: number;
targetName: string;
targetKind: string;
reason: string | null;
startedAt: string;
endsAt: string;
createdBy: string | null;
}
export interface MaintenanceTargets {
servers: { id: number; name: string }[];
integrations: { id: number; name: string; type: string }[];
dnsProviders: { id: number; name: string; providerType: string }[];
}
export interface AuditLogEntry { export interface AuditLogEntry {
id: number; id: number;
actorUserId: number | null; actorUserId: number | null;
@@ -649,6 +669,12 @@ export const api = {
body: JSON.stringify({ role }), body: JSON.stringify({ role }),
}), }),
}, },
maintenance: {
list: () => request<{ windows: MaintenanceWindow[]; targets: MaintenanceTargets }>("/api/maintenance"),
start: (data: { targetType: MaintenanceTargetType; targetId: number; minutes: number; reason?: string }) =>
request<{ window: MaintenanceWindow }>("/api/maintenance", { method: "POST", body: JSON.stringify(data) }),
end: (id: number) => request<void>(`/api/maintenance/${id}`, { method: "DELETE" }),
},
sessions: { sessions: {
list: () => request<{ sessions: SessionSummary[]; currentSessionId: string }>("/api/sessions"), list: () => request<{ sessions: SessionSummary[]; currentSessionId: string }>("/api/sessions"),
revoke: (id: string) => request<void>(`/api/sessions/${encodeURIComponent(id)}`, { method: "DELETE" }), revoke: (id: string) => request<void>(`/api/sessions/${encodeURIComponent(id)}`, { method: "DELETE" }),
+41 -4
View File
@@ -1,5 +1,5 @@
import { useState, type ReactNode } from "react"; import { useEffect, useState, type ReactNode } from "react";
import { NavLink } from "react-router-dom"; import { Link, NavLink } from "react-router-dom";
import { import {
IconLayoutDashboard, IconLayoutDashboard,
IconServer2, IconServer2,
@@ -22,8 +22,10 @@ import {
IconSun, IconSun,
IconMoon, IconMoon,
IconWand, IconWand,
IconTool,
} from "@tabler/icons-react"; } from "@tabler/icons-react";
import type { CurrentUser } from "../api/client"; import { api, type CurrentUser, type MaintenanceWindow } from "../api/client";
import { formatRemaining } from "../utils/duration";
import { getTheme, setTheme, type Theme } from "../utils/theme"; import { getTheme, setTheme, type Theme } from "../utils/theme";
import CommandPalette from "../components/CommandPalette"; import CommandPalette from "../components/CommandPalette";
@@ -48,6 +50,7 @@ const NAV_ITEMS: NavItem[] = [
{ to: "/gitea", label: "Gitea", icon: <IconBrandGit size={20} /> }, { to: "/gitea", label: "Gitea", icon: <IconBrandGit size={20} /> },
{ to: "/integrations", label: "Integrations", icon: <IconPlugConnected size={20} /> }, { to: "/integrations", label: "Integrations", icon: <IconPlugConnected size={20} /> },
{ to: "/generator", label: "Generator", icon: <IconWand size={20} /> }, { to: "/generator", label: "Generator", icon: <IconWand size={20} /> },
{ to: "/maintenance", label: "Maintenance", icon: <IconTool size={20} /> },
{ to: "/users", label: "Users", icon: <IconUsers size={20} />, minRole: "admin" }, { to: "/users", label: "Users", icon: <IconUsers size={20} />, minRole: "admin" },
{ to: "/sessions", label: "Sessions", icon: <IconDevices size={20} />, minRole: "admin" }, { to: "/sessions", label: "Sessions", icon: <IconDevices size={20} />, minRole: "admin" },
{ to: "/audit-log", label: "Audit Log", icon: <IconHistory size={20} />, minRole: "operator" }, { to: "/audit-log", label: "Audit Log", icon: <IconHistory size={20} />, minRole: "operator" },
@@ -60,6 +63,28 @@ const roleRank: Record<CurrentUser["role"], number> = { viewer: 0, operator: 1,
export default function AppShell({ user, children }: { user: CurrentUser; children: ReactNode }) { export default function AppShell({ user, children }: { user: CurrentUser; children: ReactNode }) {
const [sidebarOpen, setSidebarOpen] = useState(false); const [sidebarOpen, setSidebarOpen] = useState(false);
const [theme, setThemeState] = useState<Theme>(() => getTheme()); const [theme, setThemeState] = useState<Theme>(() => getTheme());
const [maintenance, setMaintenance] = useState<MaintenanceWindow[]>([]);
// Everyone sees what's currently silenced, so nobody is left wondering why an alert never came.
useEffect(() => {
let cancelled = false;
const load = () =>
api.maintenance
.list()
.then((res) => {
if (!cancelled) setMaintenance(res.windows);
})
.catch(() => {});
load();
const poll = setInterval(load, 60_000);
window.addEventListener("maintenance-changed", load);
return () => {
cancelled = true;
clearInterval(poll);
window.removeEventListener("maintenance-changed", load);
};
}, []);
const visibleItems = NAV_ITEMS.filter( const visibleItems = NAV_ITEMS.filter(
(item) => !item.minRole || roleRank[user.role] >= roleRank[item.minRole], (item) => !item.minRole || roleRank[user.role] >= roleRank[item.minRole],
); );
@@ -126,7 +151,19 @@ export default function AppShell({ user, children }: { user: CurrentUser; childr
<div className="page-wrapper"> <div className="page-wrapper">
<div className="page-body"> <div className="page-body">
<div className="container-xl">{children}</div> <div className="container-xl">
{maintenance.length > 0 && (
<div className="alert alert-warning d-flex align-items-center gap-2">
<IconTool size={18} />
<div>
Maintenance — alerts silenced for{" "}
{maintenance.map((w) => `${w.targetName} (${formatRemaining(w.endsAt)} left)`).join(", ")}.{" "}
<Link to="/maintenance">Manage</Link>
</div>
</div>
)}
{children}
</div>
</div> </div>
</div> </div>
</div> </div>
+249
View File
@@ -0,0 +1,249 @@
import { useEffect, useState } from "react";
import { api, type CurrentUser, type MaintenanceTargets, type MaintenanceTargetType, type MaintenanceWindow } from "../api/client";
import { formatDateTime } from "../utils/date";
import { formatRemaining } from "../utils/duration";
import { readableError } from "../utils/errors";
const DURATIONS: { minutes: number; label: string }[] = [
{ minutes: 30, label: "30 minutes" },
{ minutes: 60, label: "1 hour" },
{ minutes: 120, label: "2 hours" },
{ minutes: 240, label: "4 hours" },
{ minutes: 480, label: "8 hours" },
{ minutes: 1440, label: "24 hours" },
{ minutes: 4320, label: "3 days" },
];
// Tells the app shell's banner to refetch right away instead of waiting for its next poll.
function announceChange() {
window.dispatchEvent(new Event("maintenance-changed"));
}
export default function Maintenance({ user }: { user: CurrentUser }) {
const canEdit = user.role === "admin" || user.role === "operator";
const [windows, setWindows] = useState<MaintenanceWindow[] | null>(null);
const [targets, setTargets] = useState<MaintenanceTargets | null>(null);
const [error, setError] = useState<string | null>(null);
const [now, setNow] = useState(() => Date.now());
const [target, setTarget] = useState("");
const [minutes, setMinutes] = useState(60);
const [reason, setReason] = useState("");
const [busy, setBusy] = useState(false);
function load() {
api.maintenance
.list()
.then((res) => {
setWindows(res.windows);
setTargets(res.targets);
})
.catch((err) => setError(readableError(err)));
}
useEffect(() => {
load();
const tick = setInterval(() => setNow(Date.now()), 30_000);
return () => clearInterval(tick);
}, []);
async function start(e: React.FormEvent) {
e.preventDefault();
const [targetType, id] = target.split(":");
if (!targetType || !id) return;
setError(null);
setBusy(true);
try {
await api.maintenance.start({
targetType: targetType as MaintenanceTargetType,
targetId: Number(id),
minutes,
reason: reason.trim() || undefined,
});
setTarget("");
setReason("");
load();
announceChange();
} catch (err) {
setError(readableError(err));
} finally {
setBusy(false);
}
}
async function end(w: MaintenanceWindow) {
setError(null);
try {
await api.maintenance.end(w.id);
load();
announceChange();
} catch (err) {
setError(readableError(err));
}
}
return (
<>
<h2 className="page-title mb-3">Maintenance</h2>
<div className="text-secondary mb-3">
Silence alerts about one server or integration while you work on it. Every window ends on its own — anything
still wrong when it ends alerts then, so nothing stays hidden.
</div>
{error && <div className="alert alert-danger">{error}</div>}
{canEdit && (
<div className="card mb-3">
<div className="card-header">
<h3 className="card-title">Start maintenance</h3>
</div>
<form onSubmit={start}>
<div className="card-body row g-3">
<div className="col-md-4">
<label className="form-label">What are you working on?</label>
<select className="form-select" required value={target} onChange={(e) => setTarget(e.target.value)}>
<option value="">Choose…</option>
{targets && targets.servers.length > 0 && (
<optgroup label="Servers">
{targets.servers.map((s) => (
<option key={`s${s.id}`} value={`server:${s.id}`}>
{s.name}
</option>
))}
</optgroup>
)}
{targets && targets.integrations.length > 0 && (
<optgroup label="Integrations">
{targets.integrations.map((i) => (
<option key={`i${i.id}`} value={`integration:${i.id}`}>
{i.name} ({i.type})
</option>
))}
</optgroup>
)}
{targets && targets.dnsProviders.length > 0 && (
<optgroup label="DNS providers">
{targets.dnsProviders.map((p) => (
<option key={`d${p.id}`} value={`dns_provider:${p.id}`}>
{p.name} ({p.providerType})
</option>
))}
</optgroup>
)}
</select>
</div>
<div className="col-md-3">
<label className="form-label">For</label>
<select className="form-select" value={minutes} onChange={(e) => setMinutes(Number(e.target.value))}>
{DURATIONS.map((d) => (
<option key={d.minutes} value={d.minutes}>
{d.label}
</option>
))}
</select>
</div>
<div className="col-md-5">
<label className="form-label">Reason (optional)</label>
<input
className="form-control"
maxLength={200}
placeholder="e.g. kernel update and reboot"
value={reason}
onChange={(e) => setReason(e.target.value)}
/>
</div>
</div>
<div className="card-footer">
<button type="submit" className="btn btn-primary" disabled={busy || !target}>
{busy ? "Starting…" : "Start maintenance"}
</button>
</div>
</form>
</div>
)}
<div className="card mb-3">
<div className="card-header">
<h3 className="card-title">Active</h3>
</div>
<div className="table-responsive">
<table className="table table-vcenter card-table">
<thead>
<tr>
<th>Target</th>
<th>Reason</th>
<th>Ends</th>
<th>Started by</th>
{canEdit && <th className="w-1">Actions</th>}
</tr>
</thead>
<tbody>
{windows?.map((w) => (
<tr key={w.id}>
<td>
{w.targetName} <span className="text-secondary">· {w.targetKind}</span>
</td>
<td className="text-secondary">{w.reason ?? "—"}</td>
<td>
<span className="badge bg-yellow-lt text-yellow me-2">{formatRemaining(w.endsAt, now)} left</span>
<span className="text-secondary small">{formatDateTime(new Date(w.endsAt))}</span>
</td>
<td className="text-secondary">{w.createdBy ?? "—"}</td>
{canEdit && (
<td>
<button className="btn btn-sm btn-outline-secondary" onClick={() => end(w)}>
End now
</button>
</td>
)}
</tr>
))}
{windows?.length === 0 && (
<tr>
<td colSpan={canEdit ? 5 : 4} className="text-secondary text-center">
Nothing is in maintenance — all alerts are active.
</td>
</tr>
)}
</tbody>
</table>
</div>
</div>
<div className="card">
<div className="card-header">
<h3 className="card-title">What gets silenced</h3>
</div>
<div className="card-body">
<div className="row">
<div className="col-md-6">
<div className="fw-bold mb-1">Silenced for the target</div>
<ul className="text-secondary">
<li>Server offline and disk-full alerts (for a server).</li>
<li>Storage, volume and disk-health alerts (for a Proxmox or Synology integration).</li>
<li>Backup-failure and uncovered-guest alerts (for a Proxmox integration).</li>
<li>
"Integration down" alerts for that <em>type</em> of service — these are tracked per type (e.g. all
Proxmox), not per instance, so with two Proxmox integrations a failure on the other one is silenced
too while a window is open.
</li>
</ul>
</div>
<div className="col-md-6">
<div className="fw-bold mb-1">Not silenced</div>
<ul className="text-secondary">
<li>Secret and certificate expiry, Tailscale key expiry, Docker update reminders.</li>
<li>DNS record change notifications.</li>
</ul>
<div className="fw-bold mb-1">When a window ends</div>
<div className="text-secondary">
A problem that started during it and is still there alerts on the next check (within 15 minutes). One
that was already alerted before it began and has since cleared is reported as cleared; one that
started and cleared entirely inside the window is never reported.
</div>
</div>
</div>
</div>
</div>
</>
);
}
+1 -15
View File
@@ -3,6 +3,7 @@ import { useSearchParams } from "react-router-dom";
import { api, type CurrentUser, type SecretInput, type SecretRecord, type SecretStatus } from "../api/client"; import { api, type CurrentUser, type SecretInput, type SecretRecord, type SecretStatus } from "../api/client";
import { downloadCsv } from "../utils/csv"; import { downloadCsv } from "../utils/csv";
import { formatDateTime } from "../utils/date"; import { formatDateTime } from "../utils/date";
import { readableError } from "../utils/errors";
import { useSortable } from "../hooks/useSortable"; import { useSortable } from "../hooks/useSortable";
import SortableTh from "../components/SortableTh"; import SortableTh from "../components/SortableTh";
import { usePagination } from "../hooks/usePagination"; import { usePagination } from "../hooks/usePagination";
@@ -33,21 +34,6 @@ const emptyForm: SecretInput = {
checkPort: 443, checkPort: 443,
}; };
/** The API client throws `Request failed (400): {json}` — pull the server's own message out of that when there is one. */
function readableError(err: unknown): string {
const text = err instanceof Error ? err.message : String(err);
const match = text.match(/^Request failed \(\d+\): (.*)$/s);
if (match) {
try {
const body = JSON.parse(match[1]);
if (typeof body.message === "string") return body.message;
} catch {
// not JSON — fall through to the raw text
}
}
return text;
}
export default function Secrets({ user }: { user: CurrentUser }) { export default function Secrets({ user }: { user: CurrentUser }) {
const canEdit = user.role === "admin" || user.role === "operator"; const canEdit = user.role === "admin" || user.role === "operator";
const [secrets, setSecrets] = useState<SecretRecord[] | null>(null); const [secrets, setSecrets] = useState<SecretRecord[] | null>(null);
+9
View File
@@ -0,0 +1,9 @@
/** "45 min", "2 h 10 min", "1 d 3 h" until an ISO timestamp (never negative). */
export function formatRemaining(endsAtIso: string, now: number = Date.now()): string {
const minutes = Math.max(0, Math.ceil((new Date(endsAtIso).getTime() - now) / 60_000));
if (minutes < 60) return `${minutes} min`;
const hours = Math.floor(minutes / 60);
if (hours < 24) return minutes % 60 ? `${hours} h ${minutes % 60} min` : `${hours} h`;
const days = Math.floor(hours / 24);
return hours % 24 ? `${days} d ${hours % 24} h` : `${days} d`;
}
+14
View File
@@ -0,0 +1,14 @@
/** The API client throws `Request failed (400): {json}` — pull the server's own message out of that when there is one. */
export function readableError(err: unknown): string {
const text = err instanceof Error ? err.message : String(err);
const match = text.match(/^Request failed \(\d+\): (.*)$/s);
if (match) {
try {
const body = JSON.parse(match[1]);
if (typeof body.message === "string") return body.message;
} catch {
// not JSON — fall through to the raw text
}
}
return text;
}