From c79b2eda0d70ba71056b1323d4d247c951152451 Mon Sep 17 00:00:00 2001 From: Bobban Rydh Date: Tue, 3 Nov 2026 23:54:11 +0100 Subject: [PATCH] Show new ports across every server on the Ports page, and mark them as expected there A New badge, a notice with Show only new and Mark all as expected, and Mark as expected on each row. Uses the same rule as the alert and the server's Ports card. Co-Authored-By: Claude Sonnet 5.5 --- README.md | 2 +- ROLES.md | 2 +- docs/MONITORING.md | 2 +- server/src/routes/ports.ts | 22 ++++++++- server/test/newPorts.test.ts | 30 ++++++++++-- web/src/api/client.ts | 2 + web/src/pages/Ports.tsx | 92 ++++++++++++++++++++++++++++++++++-- 7 files changed, 139 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index efa0f49..45dfa0b 100644 --- a/README.md +++ b/README.md @@ -891,7 +891,7 @@ minimum in both themes. Motion is turned down for those who ask the system to. **New ports.** When an agent first reports what a server is listening on, that is taken as normal (the baseline). After that, a **TCP port that begins listening**, that is reachable from outside the machine, and that nobody has noted is a warning on the Alerts page (kind *Monitoring*) and a notification, with what is listening on it: "nas: a new port is listening — 8080/tcp (node)". UDP (clients and discovery open them all the time), ports from 49152 up (where programs pick a free one) and anything bound to localhost only are left out. The port's row on the server's **Ports** card says *New* and has **Mark as expected**, which notes it and ends the alert; so does a label or comment you write on -it yourself. It also ends when the port stops listening, and maintenance on the server holds it back like its other alerts. It can be switched off under Settings → Notifications (*Servers and disks*); servers get their baseline from their next report after an upgrade, so nothing alerts for what was already there. +it yourself. **Network → Ports** shows the same across every server: a *New* badge on each such port, a notice that counts them (with **Show only new**), **Mark as expected** on a row and **Mark all as expected** (which asks first, naming the servers) — so there is no need to open each server. It also ends when the port stops listening, and maintenance on the server holds it back like its other alerts. It can be switched off under Settings → Notifications (*Servers and disks*); servers get their baseline from their next report after an upgrade, so nothing alerts for what was already there. **Broken containers.** With Dockhand connected, the app now says when a **container is broken**, not only when an image has an update: one that **keeps restarting** (critical, with its exit code), one whose **own health check fails**, one that **stopped when nobody meant it to**, one **created and never started**, and a **Docker host Dockhand can no longer reach**. A stop is only news when it was seen happen — a stack switched off weeks ago stays quiet — and an alert stays until the container runs again. The Docker page marks each problem and has a **Leave alone** button; diff --git a/ROLES.md b/ROLES.md index b219c7c..33f4fc7 100644 --- a/ROLES.md +++ b/ROLES.md @@ -180,7 +180,7 @@ even further, to admin only: are going; starting an update run (**Update…**) or stopping one needs operator. - **Patch plans** (Updates → Patch plans) can be seen by everyone, with how they went; making, changing, switching off and removing one needs administrator (it will run on its own, at night); **Run now** and **Stop** need operator, like an update run. The **Monthly report** page is for everyone signed in; sending it now, and its schedule (Settings), are administrators'. - **Mark as expected** on a server's Ports card (which ends a new-port alert) needs operator, like any note on a port. + **Mark as expected** on a server's Ports card, and on the Ports page under Network (one port, or **Mark all as expected**), which ends a new-port alert, needs operator, like any note on a port; everyone signed in sees which ports are *New*. - **Acknowledging an alert, handing it over, and taking it back** need operator (the person it is handed to must be an operator or administrator); everyone sees who has what. **Incidents** (Alerts → History) are readable by everyone signed in; writing a note needs operator, and a note can be removed by its author or an administrator. diff --git a/docs/MONITORING.md b/docs/MONITORING.md index c6e9ba3..c7d8d0f 100644 --- a/docs/MONITORING.md +++ b/docs/MONITORING.md @@ -137,7 +137,7 @@ Every threshold here is a default; where it can be changed, the place is named. | **ZFS / storage pool** | A ZFS pool (or Windows Storage Spaces pool) is not ONLINE | | not configurable | | **Synology volume / disk** | A volume's status is not "normal"; a disk's status or SMART result is not normal, or it is over the bad-sector or under the remaining-life limit | | not configurable | | **Unusual for the time** | CPU load or memory use over the last hour is far above what is normal for this hour of this weekday ([below](#what-is-normal-for-the-time)) | on · warning | Settings → Notifications | -| **New port** | A TCP port starts listening that was not there when the agent first reported and that nobody noted on the server's *Ports* card (the container that publishes it, or else a port preset, names it in the message: "8096/tcp jellyfin (container)", "3389/tcp RDP") | on | Settings → Notifications | +| **New port** | A TCP port starts listening that was not there when the agent first reported and that nobody noted on the server's *Ports* card or marked as expected on **Network → Ports** (which lists every server's new ports in one place, with *Show only new*, *Mark as expected* and *Mark all as expected*) (the container that publishes it, or else a port preset, names it in the message: "8096/tcp jellyfin (container)", "3389/tcp RDP") | on | Settings → Notifications | | **Network device offline** | A UniFi device has been offline too long | 10 min · critical | Settings → Notifications | | **Operating system support** | A server's OS has passed, or is about to pass, its end of support | within 90 days · note; warning once ended | Settings → Notifications | | **Updates waiting** | The agent reports updates waiting (and, where it can say, security ones and a restart needed) | | Updates page | diff --git a/server/src/routes/ports.ts b/server/src/routes/ports.ts index 30695f5..245a3e8 100644 --- a/server/src/routes/ports.ts +++ b/server/src/routes/ports.ts @@ -2,11 +2,13 @@ import { Router } from "express"; import { eq } from "drizzle-orm"; import { z } from "zod"; import { db } from "../db/client.js"; -import { servers, portForwards } from "../db/schema.js"; +import { servers, serverPorts, portForwards } from "../db/schema.js"; import { requireAuth, requireRole } from "../auth/middleware.js"; import { recordAudit } from "../services/audit.js"; import { type AgentPort, groupAgentPorts, parseJson, splitPortKey } from "../services/agentPorts.js"; import { listPresets, serviceName, serviceNames } from "../services/portPresets.js"; +import { newListeningPorts, parseBaseline, portKey } from "../services/newPorts.js"; +import { getSettings } from "../services/settingsStore.js"; import { containerPortName, containerPortNames, type ContainerPortName } from "../services/containerPorts.js"; import { asyncHandler } from "../utils/asyncHandler.js"; @@ -30,19 +32,34 @@ export interface AgentPortRow { /** The container that publishes it, as Dockhand says, when one does. */ container: ContainerPortName | null; lastSeenAt: string | null; + /** Began listening after the server's first report and nobody has noted it: a "new port" alert is on until it is noted or stops listening. */ + isNew: boolean; } portsRouter.get("/agent", asyncHandler(async (_req, res) => { const rows = await db - .select({ id: servers.id, name: servers.name, hostname: servers.hostname, listeningPorts: servers.listeningPorts, lastSeenAt: servers.lastSeenAt }) + .select({ id: servers.id, name: servers.name, hostname: servers.hostname, listeningPorts: servers.listeningPorts, portBaseline: servers.portBaseline, lastSeenAt: servers.lastSeenAt }) .from(servers); + // Which are new: the rule the alert and the server's Ports card use — listening, reachable from outside, not in the baseline, and nobody has given the port a label or a comment. + const alerting = (await getSettings()).healthChecks.newPortAlerts; + const expectedByServer = new Map>(); + if (alerting) { + for (const r of await db.select().from(serverPorts)) { + if (!r.label && !r.comment) continue; + const set = expectedByServer.get(r.serverId) ?? new Set(); + set.add(portKey(r.protocol, r.port)); + expectedByServer.set(r.serverId, set); + } + } + const names = serviceNames(await listPresets()); const containers = await containerPortNames(); const out: AgentPortRow[] = []; for (const s of rows) { const raw = parseJson(s.listeningPorts, null); if (!raw) continue; + const fresh = new Set(alerting ? newListeningPorts(raw, parseBaseline(s.portBaseline), expectedByServer.get(s.id) ?? new Set()).map((p) => portKey(p.protocol, p.port)) : []); for (const [key, grouped] of groupAgentPorts(raw)) { const { protocol, port } = splitPortKey(key); out.push({ @@ -57,6 +74,7 @@ portsRouter.get("/agent", asyncHandler(async (_req, res) => { service: serviceName(names, protocol, port), container: containerPortName(containers, s.id, protocol, port), lastSeenAt: s.lastSeenAt, + isNew: fresh.has(key), }); } } diff --git a/server/test/newPorts.test.ts b/server/test/newPorts.test.ts index ed03056..a914a31 100644 --- a/server/test/newPorts.test.ts +++ b/server/test/newPorts.test.ts @@ -30,6 +30,7 @@ const { startOrExtendWindow } = await import("../src/services/maintenance.js"); const { metaOfConditionKey } = await import("../src/services/notifyRouting.js"); const { subjectOfConditionKey } = await import("../src/services/maintenance.js"); const { serverPortsRouter } = await import("../src/routes/serverPorts.js"); +const { portsRouter } = await import("../src/routes/ports.js"); const { updateSettings } = await import("../src/services/settingsStore.js"); const { eq } = await import("drizzle-orm"); @@ -94,6 +95,7 @@ describe("the health check, the alert and the routing", () => { describe("from a report to the Ports card", () => { let app: Server; let base = ""; + let origin = ""; let serverId = 0; let who = "operator"; const report = (ports: ReturnType[]) => ({ tasks: [], system: { listening_ports: ports } }); @@ -113,10 +115,12 @@ describe("from a report to the Ports card", () => { // The real servers router puts a signed-in user on the request before this; here the role is read the same way the middleware does. const { requireAuth } = await import("../src/auth/middleware.js"); a.use("/api/servers/:id/ports", requireAuth, serverPortsRouter); + a.use("/api/ports", portsRouter); app = await new Promise((resolve) => { const s = a.listen(0, "127.0.0.1", () => resolve(s)); }); - base = `http://127.0.0.1:${(app.address() as AddressInfo).port}/api/servers/${serverId}/ports`; + origin = `http://127.0.0.1:${(app.address() as AddressInfo).port}`; + base = `${origin}/api/servers/${serverId}/ports`; }); after(() => { @@ -175,13 +179,33 @@ describe("from a report to the Ports card", () => { assert.deepEqual(await newKeys(), []); }); + test("the Ports page, across every server, marks the same ports new — and noting one there is the same note", async () => { + await syncServerTasks(serverId, report([sock(22), sock(443), sock(8080), sock(9999, { process: "python" })])); + const [other] = await db.insert(servers).values({ name: "web", apiTokenHash: "w", apiTokenPrefix: "w" }).returning({ id: servers.id }); + await syncServerTasks(other.id, report([sock(80)])); + await syncServerTasks(other.id, report([sock(80), sock(3000), sock(5353, { protocol: "udp" })])); + who = "viewer"; + type Row = { serverId: number; port: number; protocol: string; isNew: boolean }; + const overview = async () => ((await (await fetch(`${origin}/api/ports/agent`)).json()).ports as Row[]); + const fresh = async () => (await overview()).filter((p) => p.isNew).map((p) => `${p.serverId}:${p.protocol}/${p.port}`).sort(); + assert.deepEqual(await fresh(), [`${serverId}:tcp/9999`, `${other.id}:tcp/3000`].sort(), "what the baseline had, UDP and noted ports are not new"); + assert.equal((await overview()).filter((p) => p.serverId === serverId).length, 4, "every port is still listed, new or not"); + + who = "operator"; + const res = await fetch(base, { method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify({ port: 9999, protocol: "tcp", label: "Expected" }) }); + assert.equal(res.status, 200); + assert.deepEqual(await fresh(), [`${other.id}:tcp/3000`]); + assert.deepEqual(await newKeys(), [], "and the alert ends with it"); + }); + test("it can be switched off, and then nothing is new", async () => { - await syncServerTasks(serverId, report([sock(22), sock(443), sock(8080), sock(9999)])); - assert.deepEqual(await newKeys(), [9999]); + await syncServerTasks(serverId, report([sock(22), sock(443), sock(8080), sock(9998)])); + assert.deepEqual(await newKeys(), [9998]); await updateSettings({ healthChecks: { newPortAlerts: false } }); assert.deepEqual(await newKeys(), []); who = "viewer"; assert.equal(((await (await fetch(base)).json()).ports as { isNew: boolean }[]).some((p) => p.isNew), false); + assert.equal(((await (await fetch(`${origin}/api/ports/agent`)).json()).ports as { isNew: boolean }[]).some((p) => p.isNew), false, "nor on the Ports page"); await updateSettings({ healthChecks: { newPortAlerts: true } }); }); }); diff --git a/web/src/api/client.ts b/web/src/api/client.ts index 5003123..1ef43ee 100644 --- a/web/src/api/client.ts +++ b/web/src/api/client.ts @@ -400,6 +400,8 @@ export interface AgentPortRow { service: string | null; /** The Docker container that publishes it, as Dockhand says; null when none does. */ container: PortContainer | null; + /** Began listening after the server's first report and nobody has noted it: a "new port" alert is on until it is noted or stops listening. */ + isNew: boolean; } export interface PortContainer { diff --git a/web/src/pages/Ports.tsx b/web/src/pages/Ports.tsx index 528f264..9f5193f 100644 --- a/web/src/pages/Ports.tsx +++ b/web/src/pages/Ports.tsx @@ -20,6 +20,8 @@ import { ErrorNotice } from "../components/Notice"; import PortPresets from "../components/PortPresets"; import ViewTabs from "../components/ViewTabs"; import { portName } from "../utils/portName"; +import { confirmDialog } from "../utils/dialogs"; +import { readableError } from "../utils/errors"; import { EmptyRow, LoadingRow } from "../components/EmptyState"; const emptyForm: PortForwardInput = { @@ -42,6 +44,9 @@ export default function Ports({ user }: { user: CurrentUser }) { const [reportingServers, setReportingServers] = useState(0); const [totalServers, setTotalServers] = useState(0); const [agentError, setAgentError] = useState(null); + const [onlyNew, setOnlyNew] = useState(false); + /** Keys ("serverId:tcp:8080") of the ports being marked as expected right now. */ + const [marking, setMarking] = useState>(new Set()); const [servers, setServers] = useState(null); const [forwards, setForwards] = useState(null); @@ -79,8 +84,12 @@ export default function Ports({ user }: { user: CurrentUser }) { .catch(() => {}); }, []); + const newPorts = (agentPorts ?? []).filter((p) => p.isNew); + const newServers = new Set(newPorts.map((p) => p.serverId)).size; + // With none left, the filter has nothing to show: leave it rather than show an empty table that looks like a problem. + const showOnlyNew = onlyNew && newPorts.length > 0; const { sorted: sortedAgentPorts, sortKey: agentSortKey, sortDir: agentSortDir, requestSort: requestAgentSort } = useSortable( - agentPorts, + agentPorts === null ? null : showOnlyNew ? newPorts : agentPorts, "serverName", ); const { pageItems: agentPageItems, page: agentPage, setPage: setAgentPage, pageCount: agentPageCount, totalCount: agentTotalCount } = usePagination( @@ -92,10 +101,41 @@ export default function Ports({ user }: { user: CurrentUser }) { "externalPort", ); + const portId = (p: AgentPortRow) => `${p.serverId}:${p.protocol}:${p.port}`; + + /** "This is expected": the same note the server's own Ports card writes, so the new-port alert ends. One at a time, so a refusal names the port it stopped at. */ + async function markExpected(list: AgentPortRow[]) { + setAgentError(null); + setMarking((prev) => new Set([...prev, ...list.map(portId)])); + try { + for (const p of list) { + try { + await api.servers.ports.save(p.serverId, { port: p.port, protocol: p.protocol, label: "Expected", comment: "" }); + } catch (err) { + setAgentError(`${p.serverName} ${p.protocol}/${p.port}: ${readableError(err)}`); + break; + } + } + } finally { + setMarking(new Set()); + loadAgentPorts(); + } + } + + async function markAllExpected() { + const names = [...new Set(newPorts.map((p) => p.serverName))]; + const ok = await confirmDialog({ + title: "Mark every new port as expected", + message: `${newPorts.length} port${newPorts.length !== 1 ? "s" : ""} on ${names.length} server${names.length !== 1 ? "s" : ""} (${names.join(", ")}) will be noted as expected, and their alerts end. Only do this if you have looked at what is listening on each.`, + confirmLabel: "Mark as expected", + }); + if (ok) await markExpected(newPorts); + } + function exportAgentCsv() { downloadCsv( "agent-reported-ports.csv", - ["Server", "Hostname", "Protocol", "Port", "Service", "Addresses", "Process", "Local only"], + ["Server", "Hostname", "Protocol", "Port", "Service", "Addresses", "Process", "Local only", "New"], (sortedAgentPorts ?? []).map((p) => [ p.serverName, p.serverHostname ?? "", @@ -105,6 +145,7 @@ export default function Ports({ user }: { user: CurrentUser }) { p.addresses.join(", "), p.process ?? "", p.localOnly ? "yes" : "no", + p.isNew ? "yes" : "no", ]), ); } @@ -220,6 +261,27 @@ export default function Ports({ user }: { user: CurrentUser }) { + {newPorts.length > 0 && ( +
+
+ + {newPorts.length} new port{newPorts.length !== 1 ? "s" : ""} + {" "} + on {newServers} server{newServers !== 1 ? "s" : ""} began listening after that server's first report, and nobody has said {newPorts.length !== 1 ? "they are" : "it is"} expected. Each is a warning on the{" "} + Alerts page until it is marked as expected or stops listening. +
+
+ + {canEdit && ( + + )} +
+
+ )}
@@ -231,6 +293,7 @@ export default function Ports({ user }: { user: CurrentUser }) { label="Last report" sortKeyName="lastSeenAt" activeKey={agentSortKey} direction={agentSortDir} onSort={requestAgentSort} /> + {canEdit && newPorts.length > 0 && @@ -241,7 +304,17 @@ export default function Ports({ user }: { user: CurrentUser }) { {p.serverHostname &&
{p.serverHostname}
} - + + {canEdit && newPorts.length > 0 && ( + + )} ))} - {agentPorts === null && } + {agentPorts === null && } {agentPorts !== null && agentPorts.length === 0 && ( - + )}
Address Process }
{p.protocol}{p.port} + {p.port} + {p.isNew && ( + + New + + )} + {portName(p)?.text ?? "—"} {p.addresses.join(", ")} @@ -249,11 +322,20 @@ export default function Ports({ user }: { user: CurrentUser }) { {p.process ?? "—"} + {p.isNew && ( + + )} +