Acknowledge and hand over alerts, incident timelines, and built-in database backups

- Alerts can be acknowledged (note, assignee, duration); acknowledged alerts are left out of what
  needs attention (sidebar, dashboard strip, cards) but stay raised and counted by /metrics
- Incidents: each time an alert is raised and goes away is recorded (alert_acks, alert_incidents,
  incident_notes, migration 0032), with a timeline of acks, maintenance, changes, update runs,
  related alerts and notes; Incidents page and a digest section
- Backups of Homelab Manager's own database: daily checked VACUUM INTO copies, retention, download,
  check, upload, delete, and restore staged and applied at start-up before the database opens
  (pre-restore copy kept); alerts and /metrics for stale or failing backups
- Docs: README, BACKUP, DATABASE, NOTIFICATIONS, ROLES, API; tests

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
bobbanandClaude Sonnet 5.5 committed 2026-10-18 02:21:08 +02:00
1 parent 7a5b623444
commit 2744a84719
41 files changed
+6573 -26

No files matched your search

+57
View File
@@ -572,6 +572,63 @@ export const runbooks = sqliteTable(
(t) => [index("runbooks_target_idx").on(t.targetType, t.targetId)],
);
// ─── Acknowledgements and incidents ─────────────────────────────────────────
// "I'm on it" for one alert — see services/alertAcks.ts. Keyed by the alert's stable id (not a foreign key to anything); gone once the alert has been.
export const alertAcks = sqliteTable(
"alert_acks",
{
id: integer("id").primaryKey({ autoIncrement: true }),
alertKey: text("alert_key").notNull(),
note: text("note").notNull().default(""),
assigneeUserId: integer("assignee_user_id").references(() => users.id, { onDelete: "set null" }),
assigneeName: text("assignee_name"), // as it was when handed over, so it still reads if the user is removed
ackedBy: text("acked_by").notNull(),
ackedAt: text("acked_at").notNull(), // ISO
until: text("until"), // ISO; null = until the alert is gone
lastSeenAt: text("last_seen_at").notNull(), // ISO; the last time the alert was there
},
(t) => [uniqueIndex("alert_acks_key_idx").on(t.alertKey)],
);
// One time an alert was raised and later went away — see services/incidents.ts. When the same alert comes back it is a new row.
export const alertIncidents = sqliteTable(
"alert_incidents",
{
id: integer("id").primaryKey({ autoIncrement: true }),
alertKey: text("alert_key").notNull(),
severity: text("severity").notNull(), // as it is now (or was when it ended)
peakSeverity: text("peak_severity").notNull(),
category: text("category").notNull(),
source: text("source").notNull(),
message: text("message").notNull(), // the latest wording
link: text("link"),
targetType: text("target_type"), // server | integration | dns_provider, when the alert is about one of those
targetId: integer("target_id"),
targetName: text("target_name"),
causedBy: text("caused_by"), // the host whose outage this was folded under
openedAt: text("opened_at").notNull(), // ISO
resolvedAt: text("resolved_at"), // ISO; null while it is open
lastSeenAt: text("last_seen_at").notNull(), // ISO
missingSince: text("missing_since"), // ISO; set while it has been absent from the last look(s), before it is called over
},
(t) => [index("alert_incidents_key_idx").on(t.alertKey), index("alert_incidents_opened_idx").on(t.openedAt), index("alert_incidents_target_idx").on(t.targetType, t.targetId)],
);
export const incidentNotes = sqliteTable(
"incident_notes",
{
id: integer("id").primaryKey({ autoIncrement: true }),
incidentId: integer("incident_id")
.notNull()
.references(() => alertIncidents.id, { onDelete: "cascade" }),
author: text("author").notNull(),
body: text("body").notNull(),
createdAt: text("created_at").notNull(), // ISO
},
(t) => [index("incident_notes_incident_idx").on(t.incidentId)],
);
// ─── Update runs: apply a server's updates and see it through ──────────────
// A playbook run in Semaphore for one server, wrapped in a maintenance window and followed until the server reports again — see