Rebooting Proxmox or patching a server triggered failure/offline alerts you then had to dismiss. A maintenance window silences alerts about one server, integration, or DNS provider for a chosen time. New Maintenance page (start with a duration and optional reason, end early, see what's silenced and what isn't) and a banner in the app shell so every signed-in user can see what is currently silenced. Starting/ending is operator-only and audit-logged; starting one on a target that already has a window restarts its clock instead of stacking. Silenced for the target: server offline/disk alerts, Proxmox/Synology storage and health alerts, Proxmox backup alerts, and "integration down" alerts. Not silenced: expiry and update reminders, DNS change notices. The design goal is that this cannot hide a real outage: - Every window has a required end (5 min to 7 days); there is no open-ended option, so a forgotten window expires by itself. - A silenced problem is deliberately NOT recorded as "known". If it is still present when the window ends it alerts then, as new. A problem that was already alerted before the window stays known, so it isn't repeated, and is reported cleared only after the window ends. - Failure alerts keep counting failures during a window without marking themselves alerted, so an outage that outlasts the window alerts on the very next failed call. Known limitation, stated on the page: integration-failure alerts are tracked per service TYPE (all "proxmox"), not per configured instance, so a window on one Proxmox integration also silences a failure on a second Proxmox integration while it's open. Fixing that means threading the integration id through every adapter and the diagnostic log, which is a much larger change than this feature. Also moved the API-error-message helper out of Secrets.tsx into a shared util now that two pages use it. New table maintenance_windows (migration 0008). Verified with 44 checks: the condition-key-to-subject mapping (including server:3 vs server:33), the diff rules with silenced subjects (new problem not recorded, alerts when the window ends; already-known one carried and not repeated; clears only after the window), window expiry and integration/DNS-provider source matching, the failure tracker end to end against a webhook (silent during a window while an unrelated service still alerts; outage that outlasts the window alerts on the next failure and only once; fail-and-recover fully inside a window sends nothing), a full health pass against a real window, and the real router with a stubbed session (role rules, duration bounds including the missing-duration case, extend-not-stack, 404s, deleted targets hidden, audit entries). Real dev database mtime untouched. Not done: I haven't clicked through the new page or banner in a browser (they sit behind the Authentik login); it builds and the API behind it is tested. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
250 lines
9.4 KiB
TypeScript
250 lines
9.4 KiB
TypeScript
import { useEffect, useState } from "react";
|
|
import { api, type CurrentUser, type MaintenanceTargets, type MaintenanceTargetType, type MaintenanceWindow } from "../api/client";
|
|
import { formatDateTime } from "../utils/date";
|
|
import { formatRemaining } from "../utils/duration";
|
|
import { readableError } from "../utils/errors";
|
|
|
|
const DURATIONS: { minutes: number; label: string }[] = [
|
|
{ minutes: 30, label: "30 minutes" },
|
|
{ minutes: 60, label: "1 hour" },
|
|
{ minutes: 120, label: "2 hours" },
|
|
{ minutes: 240, label: "4 hours" },
|
|
{ minutes: 480, label: "8 hours" },
|
|
{ minutes: 1440, label: "24 hours" },
|
|
{ minutes: 4320, label: "3 days" },
|
|
];
|
|
|
|
// Tells the app shell's banner to refetch right away instead of waiting for its next poll.
|
|
function announceChange() {
|
|
window.dispatchEvent(new Event("maintenance-changed"));
|
|
}
|
|
|
|
export default function Maintenance({ user }: { user: CurrentUser }) {
|
|
const canEdit = user.role === "admin" || user.role === "operator";
|
|
const [windows, setWindows] = useState<MaintenanceWindow[] | null>(null);
|
|
const [targets, setTargets] = useState<MaintenanceTargets | null>(null);
|
|
const [error, setError] = useState<string | null>(null);
|
|
const [now, setNow] = useState(() => Date.now());
|
|
|
|
const [target, setTarget] = useState("");
|
|
const [minutes, setMinutes] = useState(60);
|
|
const [reason, setReason] = useState("");
|
|
const [busy, setBusy] = useState(false);
|
|
|
|
function load() {
|
|
api.maintenance
|
|
.list()
|
|
.then((res) => {
|
|
setWindows(res.windows);
|
|
setTargets(res.targets);
|
|
})
|
|
.catch((err) => setError(readableError(err)));
|
|
}
|
|
|
|
useEffect(() => {
|
|
load();
|
|
const tick = setInterval(() => setNow(Date.now()), 30_000);
|
|
return () => clearInterval(tick);
|
|
}, []);
|
|
|
|
async function start(e: React.FormEvent) {
|
|
e.preventDefault();
|
|
const [targetType, id] = target.split(":");
|
|
if (!targetType || !id) return;
|
|
setError(null);
|
|
setBusy(true);
|
|
try {
|
|
await api.maintenance.start({
|
|
targetType: targetType as MaintenanceTargetType,
|
|
targetId: Number(id),
|
|
minutes,
|
|
reason: reason.trim() || undefined,
|
|
});
|
|
setTarget("");
|
|
setReason("");
|
|
load();
|
|
announceChange();
|
|
} catch (err) {
|
|
setError(readableError(err));
|
|
} finally {
|
|
setBusy(false);
|
|
}
|
|
}
|
|
|
|
async function end(w: MaintenanceWindow) {
|
|
setError(null);
|
|
try {
|
|
await api.maintenance.end(w.id);
|
|
load();
|
|
announceChange();
|
|
} catch (err) {
|
|
setError(readableError(err));
|
|
}
|
|
}
|
|
|
|
return (
|
|
<>
|
|
<h2 className="page-title mb-3">Maintenance</h2>
|
|
<div className="text-secondary mb-3">
|
|
Silence alerts about one server or integration while you work on it. Every window ends on its own — anything
|
|
still wrong when it ends alerts then, so nothing stays hidden.
|
|
</div>
|
|
{error && <div className="alert alert-danger">{error}</div>}
|
|
|
|
{canEdit && (
|
|
<div className="card mb-3">
|
|
<div className="card-header">
|
|
<h3 className="card-title">Start maintenance</h3>
|
|
</div>
|
|
<form onSubmit={start}>
|
|
<div className="card-body row g-3">
|
|
<div className="col-md-4">
|
|
<label className="form-label">What are you working on?</label>
|
|
<select className="form-select" required value={target} onChange={(e) => setTarget(e.target.value)}>
|
|
<option value="">Choose…</option>
|
|
{targets && targets.servers.length > 0 && (
|
|
<optgroup label="Servers">
|
|
{targets.servers.map((s) => (
|
|
<option key={`s${s.id}`} value={`server:${s.id}`}>
|
|
{s.name}
|
|
</option>
|
|
))}
|
|
</optgroup>
|
|
)}
|
|
{targets && targets.integrations.length > 0 && (
|
|
<optgroup label="Integrations">
|
|
{targets.integrations.map((i) => (
|
|
<option key={`i${i.id}`} value={`integration:${i.id}`}>
|
|
{i.name} ({i.type})
|
|
</option>
|
|
))}
|
|
</optgroup>
|
|
)}
|
|
{targets && targets.dnsProviders.length > 0 && (
|
|
<optgroup label="DNS providers">
|
|
{targets.dnsProviders.map((p) => (
|
|
<option key={`d${p.id}`} value={`dns_provider:${p.id}`}>
|
|
{p.name} ({p.providerType})
|
|
</option>
|
|
))}
|
|
</optgroup>
|
|
)}
|
|
</select>
|
|
</div>
|
|
<div className="col-md-3">
|
|
<label className="form-label">For</label>
|
|
<select className="form-select" value={minutes} onChange={(e) => setMinutes(Number(e.target.value))}>
|
|
{DURATIONS.map((d) => (
|
|
<option key={d.minutes} value={d.minutes}>
|
|
{d.label}
|
|
</option>
|
|
))}
|
|
</select>
|
|
</div>
|
|
<div className="col-md-5">
|
|
<label className="form-label">Reason (optional)</label>
|
|
<input
|
|
className="form-control"
|
|
maxLength={200}
|
|
placeholder="e.g. kernel update and reboot"
|
|
value={reason}
|
|
onChange={(e) => setReason(e.target.value)}
|
|
/>
|
|
</div>
|
|
</div>
|
|
<div className="card-footer">
|
|
<button type="submit" className="btn btn-primary" disabled={busy || !target}>
|
|
{busy ? "Starting…" : "Start maintenance"}
|
|
</button>
|
|
</div>
|
|
</form>
|
|
</div>
|
|
)}
|
|
|
|
<div className="card mb-3">
|
|
<div className="card-header">
|
|
<h3 className="card-title">Active</h3>
|
|
</div>
|
|
<div className="table-responsive">
|
|
<table className="table table-vcenter card-table">
|
|
<thead>
|
|
<tr>
|
|
<th>Target</th>
|
|
<th>Reason</th>
|
|
<th>Ends</th>
|
|
<th>Started by</th>
|
|
{canEdit && <th className="w-1">Actions</th>}
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
{windows?.map((w) => (
|
|
<tr key={w.id}>
|
|
<td>
|
|
{w.targetName} <span className="text-secondary">· {w.targetKind}</span>
|
|
</td>
|
|
<td className="text-secondary">{w.reason ?? "—"}</td>
|
|
<td>
|
|
<span className="badge bg-yellow-lt text-yellow me-2">{formatRemaining(w.endsAt, now)} left</span>
|
|
<span className="text-secondary small">{formatDateTime(new Date(w.endsAt))}</span>
|
|
</td>
|
|
<td className="text-secondary">{w.createdBy ?? "—"}</td>
|
|
{canEdit && (
|
|
<td>
|
|
<button className="btn btn-sm btn-outline-secondary" onClick={() => end(w)}>
|
|
End now
|
|
</button>
|
|
</td>
|
|
)}
|
|
</tr>
|
|
))}
|
|
{windows?.length === 0 && (
|
|
<tr>
|
|
<td colSpan={canEdit ? 5 : 4} className="text-secondary text-center">
|
|
Nothing is in maintenance — all alerts are active.
|
|
</td>
|
|
</tr>
|
|
)}
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
|
|
<div className="card">
|
|
<div className="card-header">
|
|
<h3 className="card-title">What gets silenced</h3>
|
|
</div>
|
|
<div className="card-body">
|
|
<div className="row">
|
|
<div className="col-md-6">
|
|
<div className="fw-bold mb-1">Silenced for the target</div>
|
|
<ul className="text-secondary">
|
|
<li>Server offline and disk-full alerts (for a server).</li>
|
|
<li>Storage, volume and disk-health alerts (for a Proxmox or Synology integration).</li>
|
|
<li>Backup-failure and uncovered-guest alerts (for a Proxmox integration).</li>
|
|
<li>
|
|
"Integration down" alerts for that <em>type</em> of service — these are tracked per type (e.g. all
|
|
Proxmox), not per instance, so with two Proxmox integrations a failure on the other one is silenced
|
|
too while a window is open.
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
<div className="col-md-6">
|
|
<div className="fw-bold mb-1">Not silenced</div>
|
|
<ul className="text-secondary">
|
|
<li>Secret and certificate expiry, Tailscale key expiry, Docker update reminders.</li>
|
|
<li>DNS record change notifications.</li>
|
|
</ul>
|
|
<div className="fw-bold mb-1">When a window ends</div>
|
|
<div className="text-secondary">
|
|
A problem that started during it and is still there alerts on the next check (within 15 minutes). One
|
|
that was already alerted before it began and has since cleared is reported as cleared; one that
|
|
started and cleared entirely inside the window is never reported.
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</>
|
|
);
|
|
}
|