diff --git a/DATABASE.md b/DATABASE.md index 32d7a38..acda120 100644 --- a/DATABASE.md +++ b/DATABASE.md @@ -413,7 +413,7 @@ records them; it can't check or change them. #### `port_presets` -A named list of ports ("Remote access": SSH, RDP, VNC). A port in any preset is named wherever it is shown, can be scanned for on many servers at once, and can have its name written onto the servers where it is open (`server_ports.label`). The ready-made ones are written once, into an empty table (flag `_internal:portPresetsSeeded` in `settings`), and are ordinary rows after that. The operating-system ones (*Windows client*, *Windows server*, *Debian / Ubuntu*) came later, with their own flag (`_internal:portPresetsOsSeeded`): once, into a table already in use they are added unless a preset of that name exists, and an emptied table stays empty. +A named list of ports ("Remote access": SSH, RDP, VNC). A port in any preset is named wherever it is shown, can be scanned for on many servers at once, and can have its name written onto the servers where it is open (`server_ports.label`). The ready-made ones are written once, into an empty table (flag `_internal:portPresetsSeeded` in `settings`), and are ordinary rows after that. Later batches each have their own flag — the operating-system ones (*Windows client*, *Windows server*, *Debian / Ubuntu*; `_internal:portPresetsOsSeeded`) and the network ones (*Network services*, *Tailscale*; `_internal:portPresetsNetworkSeeded`): once, into a table already in use, a batch is added unless a preset of that name exists, and an emptied table stays empty. | Column | Type | Notes | |---|---|---| diff --git a/README.md b/README.md index 9c5d0a6..f511756 100644 --- a/README.md +++ b/README.md @@ -1101,7 +1101,7 @@ install one-liner on a host to pick that up. **Ports named by their containers.** When a server's Docker host is in Dockhand (the Docker integration), the ports its running containers publish are named after the container — 8096 → *jellyfin* — on the server's Ports card, in the agent table on the Network → Ports page (hover for the image) and in the new-port alert ("a new port is listening — 8096/tcp jellyfin (container)"). It is automatic: nothing to set up beyond the Docker host in Dockhand having the same name as the server (the short name, the hostname or an address — the match Uptime Kuma targets use), and it is only a name beside the port. A label you wrote wins; a container's name wins over a port preset's; it is never written onto the port notes, so a published port is still a new port to be noted, and a container that starts publishing something is still news — now with its name. It is read on the health pass from the container list Dockhand already gives (every 15 minutes, and at start-up, whether or not the container alerts are on); a Docker host Dockhand can't reach keeps its names, and names not refreshed for a day are dropped. Containers on the host network publish nothing Docker lists, so their ports aren't named. -**Port presets** (Network → Ports → the *Presets* tab). A preset is a named list of ports written once — *Remote access* (SSH, RDP, VNC, WinRM), *Databases* (MySQL, PostgreSQL, Redis, MongoDB …), *Web*, *File sharing*, *Homelab tools*, and what a machine has open out of the box — *Windows client*, *Windows server* (with the Active Directory ports) and *Debian / Ubuntu* — come ready-made, and are ordinary presets you can change or delete. The operating-system ones are added once to an install that already had presets (a preset of the same name is left alone, a deleted one is not put back). Add your own by typing one port to a line (`3389 RDP`, `53/udp DNS`). A preset does three things: it **names ports** — a port in any preset shows its name (3389 → RDP) on every server's Ports card, in the agent table and in the new-port alert ("a new port is listening — 3389/tcp RDP"), without a label typed for each host, and a name you write yourself always wins; it **checks hosts** — pick any number of servers and see in one table which of the preset's TCP ports are open on each, from a scan from the app (private addresses only, at the server's first known address; what it finds is remembered like any scan); and it **names the open ports on all of them** in one go, keeping names somebody already wrote unless you say to replace them. Note that a port with a name counts as noted, so writing names also quiets the new-port alert for those ports. A scan only looks at TCP; a UDP port in a preset is only given its name. Operators change presets; everyone sees them. +**Port presets** (Network → Ports → the *Presets* tab). A preset is a named list of ports written once — *Remote access* (SSH, RDP, VNC, WinRM), *Databases* (MySQL, PostgreSQL, Redis, MongoDB …), *Web*, *File sharing*, *Homelab tools*, and what a machine has open out of the box — *Windows client*, *Windows server* (with the Active Directory ports) and *Debian / Ubuntu* — and for the network, *Network services* (DNS, DHCP, NTP, SNMP, syslog, RADIUS, VPNs, MikroTik, iperf3) and *Tailscale*, come ready-made, and are ordinary presets you can change or delete. The ones added later are added once to an install that already had presets (a preset of the same name is left alone, a deleted one is not put back). Add your own by typing one port to a line (`3389 RDP`, `53/udp DNS`). A preset does three things: it **names ports** — a port in any preset shows its name (3389 → RDP) on every server's Ports card, in the agent table and in the new-port alert ("a new port is listening — 3389/tcp RDP"), without a label typed for each host, and a name you write yourself always wins; it **checks hosts** — pick any number of servers and see in one table which of the preset's TCP ports are open on each, from a scan from the app (private addresses only, at the server's first known address; what it finds is remembered like any scan); and it **names the open ports on all of them** in one go, keeping names somebody already wrote unless you say to replace them. Note that a port with a name counts as noted, so writing names also quiets the new-port alert for those ports. A scan only looks at TCP; a UDP port in a preset is only given its name. Operators change presets; everyone sees them. **Network → Ports** is the cross-server counterpart: one page listing every port every agent currently reports as listening, across all servers at once diff --git a/server/src/services/portPresets.ts b/server/src/services/portPresets.ts index a408022..aadedf8 100644 --- a/server/src/services/portPresets.ts +++ b/server/src/services/portPresets.ts @@ -25,7 +25,6 @@ export interface PortPreset { export const MAX_PRESET_PORTS = 60; const SEEDED_FLAG = "portPresetsSeeded"; -const OS_SEEDED_FLAG = "portPresetsOsSeeded"; const tcp = (port: number, label: string): PresetPort => ({ port, protocol: "tcp", label }); const udp = (port: number, label: string): PresetPort => ({ port, protocol: "udp", label }); @@ -73,6 +72,29 @@ export const OS_PRESETS: { name: string; ports: PresetPort[] }[] = [ }, ]; +/** + * The network's own services, and Tailscale. Tailscale makes few ports of its own that a scan can find — it talks out, and on a port picked at run time — so what is named here is the direct + * WireGuard port an agent reports and the ports Serve and Funnel answer on. 8443 keeps the name the Web preset gives it: a name given here would be given to it on every host. + */ +export const NETWORK_PRESETS: { name: string; ports: PresetPort[] }[] = [ + { + name: "Network services", + ports: [ + tcp(22, "SSH"), tcp(23, "Telnet"), tcp(53, "DNS"), tcp(179, "BGP"), tcp(853, "DNS over TLS"), tcp(1723, "PPTP"), tcp(5201, "iperf3"), tcp(8291, "MikroTik Winbox"), tcp(8728, "MikroTik API"), tcp(8729, "MikroTik API (SSL)"), + udp(53, "DNS"), udp(67, "DHCP server"), udp(68, "DHCP client"), udp(69, "TFTP"), udp(123, "NTP"), udp(161, "SNMP"), udp(162, "SNMP trap"), udp(500, "IPsec (IKE)"), udp(514, "Syslog"), udp(520, "RIP"), + udp(546, "DHCPv6 client"), udp(547, "DHCPv6 server"), udp(1194, "OpenVPN"), udp(1701, "L2TP"), udp(1812, "RADIUS"), udp(1813, "RADIUS accounting"), udp(1900, "SSDP (UPnP discovery)"), udp(3478, "STUN"), + udp(4500, "IPsec (NAT traversal)"), udp(5201, "iperf3"), udp(5353, "mDNS"), udp(51820, "WireGuard"), + ], + }, + { name: "Tailscale", ports: [tcp(22, "SSH"), tcp(443, "HTTPS"), tcp(8443, "HTTPS (alternate)"), udp(3478, "STUN"), udp(41641, "Tailscale")] }, +]; + +/** The presets that came after the first five, each batch with the flag that says it has been put there. */ +const LATER_BATCHES: { flag: string; presets: { name: string; ports: PresetPort[] }[] }[] = [ + { flag: "portPresetsOsSeeded", presets: OS_PRESETS }, + { flag: "portPresetsNetworkSeeded", presets: NETWORK_PRESETS }, +]; + function parsePorts(stored: string): PresetPort[] { try { const v = JSON.parse(stored); @@ -88,25 +110,28 @@ function parsePorts(stored: string): PresetPort[] { const rowToPreset = (r: typeof portPresets.$inferSelect): PortPreset => ({ id: r.id, name: r.name, ports: parsePorts(r.ports), updatedAt: r.updatedAt }); /** - * Puts the ready-made presets there, once: the first time, and only into an empty list. The operating-system ones are put there once as well, and in a list that is already in use - * only the ones whose name is not taken (whatever its case) — an empty list stays empty, since somebody cleared it — so a preset deleted afterwards is not put back. + * Puts the ready-made presets there, once: the first time, and only into an empty list. The ones that came later (LATER_BATCHES) are put there once as well, and in a list that is already + * in use only those whose name is not taken (whatever its case) — an empty list stays empty, since somebody cleared it — so a preset deleted afterwards is not put back. */ async function ensureStarters(): Promise { const first = (await getInternalFlag(SEEDED_FLAG)) === null; - const osDone = (await getInternalFlag(OS_SEEDED_FLAG)) !== null; - if (!first && osDone) return; + const due = []; + for (const b of LATER_BATCHES) if ((await getInternalFlag(b.flag)) === null) due.push(b); + if (!first && due.length === 0) return; const now = new Date().toISOString(); const add = (p: { name: string; ports: PresetPort[] }) => db.insert(portPresets).values({ name: p.name, ports: JSON.stringify(p.ports), createdAt: now, updatedAt: now }); const existing = await db.select({ name: portPresets.name }).from(portPresets); if (first) { - if (existing.length === 0) for (const p of [...STARTER_PRESETS, ...OS_PRESETS]) await add(p); + if (existing.length === 0) for (const p of [...STARTER_PRESETS, ...LATER_BATCHES.flatMap((b) => b.presets)]) await add(p); await setInternalFlag(SEEDED_FLAG, now); - await setInternalFlag(OS_SEEDED_FLAG, now); + for (const b of LATER_BATCHES) await setInternalFlag(b.flag, now); return; } const taken = new Set(existing.map((e) => e.name.toLowerCase())); - if (existing.length > 0) for (const p of OS_PRESETS) if (!taken.has(p.name.toLowerCase())) await add(p); - await setInternalFlag(OS_SEEDED_FLAG, now); + for (const b of due) { + if (existing.length > 0) for (const p of b.presets) if (!taken.has(p.name.toLowerCase())) await add(p); + await setInternalFlag(b.flag, now); + } } /** Every preset, by name. */ diff --git a/server/test/portPresets.test.ts b/server/test/portPresets.test.ts index 712d419..fe2c269 100644 --- a/server/test/portPresets.test.ts +++ b/server/test/portPresets.test.ts @@ -150,18 +150,20 @@ describe("the routes", () => { test("the ready-made presets are there the first time, for everyone", async () => { const list = (await json<{ presets: Preset[] }>(await call("viewer", "GET", "/port-presets"))).presets; - assert.deepEqual(list.map((p) => p.name), ["Databases", "Debian / Ubuntu", "File sharing", "Homelab tools", "Remote access", "Web", "Windows client", "Windows server"]); + assert.deepEqual(list.map((p) => p.name), ["Databases", "Debian / Ubuntu", "File sharing", "Homelab tools", "Network services", "Remote access", "Tailscale", "Web", "Windows client", "Windows server"]); const remote = list.find((p) => p.name === "Remote access")!; assert.deepEqual(remote.ports.map((p) => [p.port, p.label]), [[22, "SSH"], [3389, "RDP"], [5900, "VNC"], [5985, "WinRM (HTTP)"], [5986, "WinRM (HTTPS)"]]); assert.ok(list.find((p) => p.name === "Databases")!.ports.some((p) => p.port === 5432 && p.label === "PostgreSQL")); }); - test("the operating-system presets name what a Windows client, a Windows server and a Debian or Ubuntu machine have open, and no port is called two things", async () => { + test("the operating-system, network and Tailscale presets name what a Windows client, a Windows server and a Debian or Ubuntu machine have open, what the network runs, and Tailscale — and no port is called two things", async () => { const list = (await json<{ presets: Preset[] }>(await call("viewer", "GET", "/port-presets"))).presets; const has = (name: string, protocol: string, port: number, label: string) => list.find((p) => p.name === name)!.ports.some((p) => p.protocol === protocol && p.port === port && p.label === label); assert.ok(has("Windows client", "tcp", 445, "SMB") && has("Windows client", "tcp", 49664, "RPC (dynamic)") && has("Windows client", "udp", 5353, "mDNS")); assert.ok(has("Windows server", "tcp", 389, "LDAP") && has("Windows server", "tcp", 9389, "AD Web Services") && has("Windows server", "udp", 67, "DHCP server")); assert.ok(has("Debian / Ubuntu", "tcp", 22, "SSH") && has("Debian / Ubuntu", "udp", 41641, "Tailscale")); + assert.ok(has("Network services", "udp", 161, "SNMP") && has("Network services", "udp", 51820, "WireGuard") && has("Network services", "tcp", 8291, "MikroTik Winbox")); + assert.ok(has("Tailscale", "udp", 41641, "Tailscale") && has("Tailscale", "udp", 3478, "STUN") && has("Tailscale", "tcp", 8443, "HTTPS (alternate)")); for (const p of list) assert.ok(p.ports.length <= pp.MAX_PRESET_PORTS, p.name); // Whichever preset names a port first (by name) is the one heard, so every preset that names the same port must say the same thing. const said = new Map(); @@ -339,29 +341,39 @@ describe("the routes", () => { }); }); -describe("the operating-system presets on a list that is already in use", () => { +describe("the presets that came later, on a list that is already in use", () => { const names = async () => (await pp.listPresets()).map((p) => p.name).sort(); + const forget = async (...which: string[]) => { + const { settings } = await import("../src/db/schema.js"); + for (const w of which) await db.delete(settings).where(eq(settings.key, `_internal:${w}`)); + }; test("they are added once, beside what is there: a preset of the same name is left as it is, a deleted one is not put back, and an emptied list stays empty", async () => { - const { settings } = await import("../src/db/schema.js"); - const forgetOsFlag = () => db.delete(settings).where(eq(settings.key, "_internal:portPresetsOsSeeded")); - // The list as an install from before them has it: five presets (one changed) and a flag for those five only. + // The list as an install from before them has it: two presets (one of them changed, and its name in another case) and a flag for the first five only. await db.delete(portPresets); const now = new Date().toISOString(); await db.insert(portPresets).values([ { name: "Remote access", ports: JSON.stringify([{ port: 22, protocol: "tcp", label: "SSH" }]), createdAt: now, updatedAt: now }, { name: "windows CLIENT", ports: JSON.stringify([{ port: 1, protocol: "tcp", label: "Mine" }]), createdAt: now, updatedAt: now }, ]); - await forgetOsFlag(); - assert.deepEqual(await names(), ["Debian / Ubuntu", "Remote access", "Windows server", "windows CLIENT"], "the missing ones came, the one with the same name (any case) did not"); + await forget("portPresetsOsSeeded", "portPresetsNetworkSeeded"); + assert.deepEqual(await names(), ["Debian / Ubuntu", "Network services", "Remote access", "Tailscale", "Windows server", "windows CLIENT"], "the missing ones came, the one with the same name (any case) did not"); assert.deepEqual((await pp.listPresets()).find((p) => p.name === "windows CLIENT")!.ports.map((p) => p.label), ["Mine"]); for (const p of (await pp.listPresets()).filter((x) => x.name === "Debian / Ubuntu")) await db.delete(portPresets).where(eq(portPresets.id, p.id)); - assert.deepEqual(await names(), ["Remote access", "Windows server", "windows CLIENT"], "deleted, and not put back"); + assert.deepEqual(await names(), ["Network services", "Remote access", "Tailscale", "Windows server", "windows CLIENT"], "deleted, and not put back"); await db.delete(portPresets); - await forgetOsFlag(); + await forget("portPresetsOsSeeded", "portPresetsNetworkSeeded"); assert.deepEqual(await names(), [], "somebody emptied the list, so it stays empty"); assert.deepEqual(await names(), [], "and the next look is the same"); }); + + test("an install that already has the operating-system presets gets only the network ones, and not the ones it deleted", async () => { + await db.delete(portPresets); + const now = new Date().toISOString(); + await db.insert(portPresets).values({ name: "Remote access", ports: JSON.stringify([{ port: 22, protocol: "tcp", label: "SSH" }]), createdAt: now, updatedAt: now }); + await forget("portPresetsNetworkSeeded"); + assert.deepEqual(await names(), ["Network services", "Remote access", "Tailscale"]); + }); });