Heartbeats, Coming up and suggested checks; Operations menu tidied

Heartbeats: a cron job or script calls /api/heartbeat/<token> (and /start, /fail); the app
alerts when the call is late or the job says it failed. Coming up: one agenda of everything
dated (expiries, warranties, end of support, planned and recurring maintenance, restore tests,
waiting updates), with a dashboard card and an iCalendar feed behind a share link. Suggested
checks: web ports and certificates the app already knows about, added with a click.

Menu: Incidents become Alerts -> History (old /incidents links redirect); new Monitoring group
(Endpoints, Heartbeats, Uptime Kuma); osTicket under Automation, Admin Links under
Infrastructure, Generator under Administration. Also makes the restore-test metrics test follow
the real clock.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
bobbanandClaude Sonnet 5.5 committed 2026-10-19 21:42:34 +02:00
1 parent c05c847d17
commit 4e0c6927b2
44 files changed
+7092 -75

No files matched your search

+41 -2
View File
@@ -192,6 +192,44 @@ export const restoreTests = sqliteTable(
(t) => [index("restore_tests_subject_idx").on(t.subjectKey, t.testedAt)],
);
// A job that reports in: a cron job or backup script calls its address when it finishes, and the app speaks up when the call doesn't come in time —
// see services/heartbeats.ts. The token is part of the address (/api/heartbeat/<token>) and is all the access a ping needs.
export const heartbeats = sqliteTable("heartbeats", {
id: integer("id").primaryKey({ autoIncrement: true }),
name: text("name").notNull(),
token: text("token").notNull().unique(),
periodMinutes: integer("period_minutes").notNull(), // how often it should report
graceMinutes: integer("grace_minutes").notNull().default(10), // how much later than that is still fine
serverId: integer("server_id"), // soft reference: the server it runs on, so its maintenance silences the alerts
enabled: integer("enabled", { mode: "boolean" }).notNull().default(true),
notes: text("notes"),
lastPingAt: text("last_ping_at"), // ISO; the last report of an end (success or failure)
lastStatus: text("last_status"), // ok | fail
lastMessage: text("last_message"),
lastDurationMs: integer("last_duration_ms"), // when it said it had started first
lastStartAt: text("last_start_at"), // ISO; the last "started" call
pingCount: integer("ping_count").notNull().default(0),
createdAt: text("created_at")
.notNull()
.default(sql`(current_timestamp)`),
});
// The recent calls to each heartbeat, for its history; kept two weeks.
export const heartbeatPings = sqliteTable(
"heartbeat_pings",
{
id: integer("id").primaryKey({ autoIncrement: true }),
heartbeatId: integer("heartbeat_id")
.notNull()
.references(() => heartbeats.id, { onDelete: "cascade" }),
at: text("at").notNull(), // ISO
kind: text("kind").notNull(), // ok | fail | start
durationMs: integer("duration_ms"),
message: text("message"),
},
(t) => [index("heartbeat_pings_idx").on(t.heartbeatId, t.at)],
);
// "A needs B", for things that can be down: a server or an integration each. Polymorphic like maintenance_windows (no foreign key); an edge whose
// end was deleted is ignored. When B is down, A's alert is folded under B's — see services/dependencies.ts.
export const dependencies = sqliteTable(
@@ -628,8 +666,9 @@ export const alertTickets = sqliteTable(
export const shareLinkDetails = ["summary", "names"] as const;
export type ShareLinkDetail = (typeof shareLinkDetails)[number];
// What a link opens: the wall display (how things are, for a TV), or a status page (is each of the things I chose working, for people at home).
export const shareLinkKinds = ["wall", "status"] as const;
// What a link opens: the wall display (how things are, for a TV), a status page (is each of the things I chose working, for people at home), or
// a calendar feed (what is coming up, for a phone's calendar to subscribe to).
export const shareLinkKinds = ["wall", "status", "calendar"] as const;
export type ShareLinkKind = (typeof shareLinkKinds)[number];
// A link to the wall display that needs no sign-in, for a TV in the hall or someone who only wants to know if things are up. Only a hash