Dockhand vulnerability scans: warn about critical findings, with a setting to turn it off
Reads what Dockhand's scans have already stored (never starts one) every 15 minutes and raises a warning per Docker host with critical findings. Unknown counts and stale readings are held, not treated as clean. Settings -> Notifications has a switch (on by default); the Docker page shows counts, scan coverage and the critical findings; /metrics and docs updated. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
65214f6f0f
commit
5a2b320175
25 files changed
+8527
-12
No files matched your search
+19
-2
@@ -23,7 +23,7 @@ what the app actually creates.
|
|||||||
|---|---|
|
|---|---|
|
||||||
| **File** | `DATABASE_PATH`, default `../data/homelab-manager.sqlite` (relative to `server/`; `/app/data/...` in Docker). The folder is created if missing. |
|
| **File** | `DATABASE_PATH`, default `../data/homelab-manager.sqlite` (relative to `server/`; `/app/data/...` in Docker). The folder is created if missing. |
|
||||||
| **Foreign keys** | Switched on for every connection (`PRAGMA foreign_keys = ON`), so the `ON DELETE` rules below are enforced. |
|
| **Foreign keys** | Switched on for every connection (`PRAGMA foreign_keys = ON`), so the `ON DELETE` rules below are enforced. |
|
||||||
| **Migrations** | SQL files in [`server/drizzle/`](server/drizzle) (`0000` … `0066`), applied automatically when the server starts. Which ones have run is recorded in the table `__drizzle_migrations`. |
|
| **Migrations** | SQL files in [`server/drizzle/`](server/drizzle) (`0000` … `0067`), applied automatically when the server starts. Which ones have run is recorded in the table `__drizzle_migrations`. |
|
||||||
| **Tables** | 78 application tables, plus drizzle's own `__drizzle_migrations`. |
|
| **Tables** | 78 application tables, plus drizzle's own `__drizzle_migrations`. |
|
||||||
|
|
||||||
**Not in the database, but beside it in the data folder:** the backups of the database itself (`backups/homelab-manager-….sqlite`, taken daily and on request; see [BACKUP.md](BACKUP.md)), and, only while a restore is waiting or has just happened, `restore-pending.sqlite` / `restore-pending.json` and `restore-result.json`.
|
**Not in the database, but beside it in the data folder:** the backups of the database itself (`backups/homelab-manager-….sqlite`, taken daily and on request; see [BACKUP.md](BACKUP.md)), and, only while a restore is waiting or has just happened, `restore-pending.sqlite` / `restore-pending.json` and `restore-result.json`.
|
||||||
@@ -73,6 +73,7 @@ erDiagram
|
|||||||
integrations ||--o| prometheus_state : "cascade"
|
integrations ||--o| prometheus_state : "cascade"
|
||||||
integrations ||--o| immich_state : "cascade"
|
integrations ||--o| immich_state : "cascade"
|
||||||
integrations ||--o| nextcloud_state : "cascade"
|
integrations ||--o| nextcloud_state : "cascade"
|
||||||
|
integrations ||--o{ dockhand_vulnerabilities : "cascade"
|
||||||
integrations ||--o| netbox_state : "cascade"
|
integrations ||--o| netbox_state : "cascade"
|
||||||
integrations ||--o| n8n_state : "cascade"
|
integrations ||--o| n8n_state : "cascade"
|
||||||
integrations ||--o| paperless_state : "cascade"
|
integrations ||--o| paperless_state : "cascade"
|
||||||
@@ -889,6 +890,22 @@ How a Nextcloud (an integration of type `nextcloud`) is right now, one row each,
|
|||||||
| `setup_issues` | text | JSON: name, severity (`error` or `warning`), text — its other setup checks that report a problem, at most 40. Null when the checks could not be asked. |
|
| `setup_issues` | text | JSON: name, severity (`error` or `warning`), text — its other setup checks that report a problem, at most 40. Null when the checks could not be asked. |
|
||||||
| `updated_at` | text NOT NULL | ISO. |
|
| `updated_at` | text NOT NULL | ISO. |
|
||||||
|
|
||||||
|
#### `dockhand_vulnerabilities`
|
||||||
|
|
||||||
|
What Dockhand's own stored vulnerability scans add up to for each of its Docker hosts (its *environments*), as last read — one row for each Dockhand integration and environment, replaced at every read (every fifteen minutes; a host Dockhand no longer lists is dropped). `integration_id` is a foreign key with **cascade**; the primary key is (`integration_id`, `environment_id`). Nothing is scanned by this app — it reads only what Dockhand kept.
|
||||||
|
|
||||||
|
| Column | Type | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `integration_id` | integer PK | |
|
||||||
|
| `environment_id` | integer PK | Dockhand's own id for the Docker host. |
|
||||||
|
| `environment_name` | text NOT NULL | |
|
||||||
|
| `critical`, `high`, `medium`, `low` | integer | The counts across the host's scanned images. **Null when the count could not be read** (an older Dockhand, a host it cannot reach) — which is not the same as 0. |
|
||||||
|
| `images_scanned`, `total_images` | integer | How many of the host's images have a scan; the rest are not known to be clean. |
|
||||||
|
| `findings` | text NOT NULL, default `[]` | JSON: the critical findings (CVE, package, installed and fixed version, image, the containers that use it), at most 100. |
|
||||||
|
| `updated_at` | text NOT NULL | ISO. |
|
||||||
|
|
||||||
|
How the last read went is kept in `settings`, key `_internal:dockerVulnSync`.
|
||||||
|
|
||||||
#### `netbox_state`
|
#### `netbox_state`
|
||||||
|
|
||||||
How a NetBox (an integration of type `netbox`) is right now, one row each, replaced at every read (every five minutes). `integration_id` is the primary key and a foreign key with **cascade**.
|
How a NetBox (an integration of type `netbox`) is right now, one row each, replaced at every read (every five minutes). `integration_id` is the primary key and a foreign key with **cascade**.
|
||||||
@@ -1349,7 +1366,7 @@ that were never changed aren't stored; the app fills in defaults when reading.
|
|||||||
| `inventory` | `currency` (a short name, may be blank) and `pricePerKwh` (null) — what the inventory's running cost is worked out with. |
|
| `inventory` | `currency` (a short name, may be blank) and `pricePerKwh` (null) — what the inventory's running cost is worked out with. |
|
||||||
| `monthlyReport` | The monthly report: `enabled` (false), `day` (1–28, default 1) and `time` (`HH:MM`, default `08:00`), in the notification time zone; `quarterly` and `annual` (both false) add the quarter's report after March, June, September and December, and the year's after December. |
|
| `monthlyReport` | The monthly report: `enabled` (false), `day` (1–28, default 1) and `time` (`HH:MM`, default `08:00`), in the notification time zone; `quarterly` and `annual` (both false) add the quarter's report after March, June, September and December, and the year's after December. |
|
||||||
| `digest` | The weekly digest: `enabled` (false), `day` (0 = Sunday … 6; default 1, Monday) and `time` (`HH:MM`, default `08:00`), in the notification time zone. |
|
| `digest` | The weekly digest: `enabled` (false), `day` (0 = Sunday … 6; default 1, Monday) and `time` (`HH:MM`, default `08:00`), in the notification time zone. |
|
||||||
| `healthChecks` | `restoreTestDays` (0 = off — how long a guest's last passing restore test counts; a failed one is always raised), `serverOfflineMinutes` (60), `diskUsagePercent` (90), `domainWarnDays` (30), `diskForecastDays` (14; 0 = off — how far ahead a filling disk is flagged on the Alerts page), `backupStaleHours` (48; 0 = off — how old a Proxmox guest's newest backup may be), `newPortAlerts` (true — warn when a server starts listening on a TCP port that is not in its baseline and not noted), `flapRaises` (3; 0 = off — raises of one alert within an hour that make it "flapping" so its messages are held; see NOTIFICATIONS.md), `deviceOfflineMinutes` (10 — how long a UniFi device may be offline before it is raised), `osSupportWarnDays` (90; 0 = off), `baselineAlerts` (true — warn when CPU or memory over the last hour is far above what the server's own history says is normal for this hour of the week; see docs/MONITORING.md), `containerAlerts` (true — warn when a Docker container is broken) and `containerIgnore` (a list of container names or `host/name`, `*` allowed, to leave alone). The baselines themselves are not stored: they are worked out from `server_metrics_rollup` (hourly rows, `grain = 'h'`) when needed. A server can override each of these (see `servers.thresholds`). |
|
| `healthChecks` | `restoreTestDays` (0 = off — how long a guest's last passing restore test counts; a failed one is always raised), `serverOfflineMinutes` (60), `diskUsagePercent` (90), `domainWarnDays` (30), `diskForecastDays` (14; 0 = off — how far ahead a filling disk is flagged on the Alerts page), `backupStaleHours` (48; 0 = off — how old a Proxmox guest's newest backup may be), `newPortAlerts` (true — warn when a server starts listening on a TCP port that is not in its baseline and not noted), `flapRaises` (3; 0 = off — raises of one alert within an hour that make it "flapping" so its messages are held; see NOTIFICATIONS.md), `deviceOfflineMinutes` (10 — how long a UniFi device may be offline before it is raised), `osSupportWarnDays` (90; 0 = off), `baselineAlerts` (true — warn when CPU or memory over the last hour is far above what the server's own history says is normal for this hour of the week; see docs/MONITORING.md), `containerAlerts` (true — warn when a Docker container is broken), `vulnerabilityAlerts` (true — warn when Dockhand's scans found critical vulnerabilities; an older stored setting without it counts as on) and `containerIgnore` (a list of container names or `host/name`, `*` allowed, to leave alone). The baselines themselves are not stored: they are worked out from `server_metrics_rollup` (hourly rows, `grain = 'h'`) when needed. A server can override each of these (see `servers.thresholds`). |
|
||||||
| `logRetention` | `enabled`, `retentionDays` (90), `intervalHours` (24). |
|
| `logRetention` | `enabled`, `retentionDays` (90), `intervalHours` (24). |
|
||||||
| `display` | `dateFormat`, `timeFormat`, `pageSize`. |
|
| `display` | `dateFormat`, `timeFormat`, `pageSize`. |
|
||||||
| `providerColors`, `integrationColors` | Badge colours, by provider / integration type. |
|
| `providerColors`, `integrationColors` | Badge colours, by provider / integration type. |
|
||||||
|
|||||||
@@ -59,6 +59,7 @@ the dashboard views themselves load fine. (UniFi, Nginx Proxy Manager, Home Assi
|
|||||||
- **Suggested checks:** each container's image and published ports are also used to suggest an endpoint check on the app's own health address ([MONITORING.md](docs/MONITORING.md#endpoint-checks-and-heartbeats)). Ports are read from whatever shape Dockhand's container list carries them in (Docker's own, `"0.0.0.0:8080->80/tcp"` strings or inspect's map) — **not checked against a live Dockhand**; if none is read, the Docker host's listening ports from its agent stand in.
|
- **Suggested checks:** each container's image and published ports are also used to suggest an endpoint check on the app's own health address ([MONITORING.md](docs/MONITORING.md#endpoint-checks-and-heartbeats)). Ports are read from whatever shape Dockhand's container list carries them in (Docker's own, `"0.0.0.0:8080->80/tcp"` strings or inspect's map) — **not checked against a live Dockhand**; if none is read, the Docker host's listening ports from its agent stand in.
|
||||||
- **Container health:** on the 15-minute health pass the app also reads every container's state and status text and raises alerts for what is broken — see [Docker containers that are broken](docs/MONITORING.md#docker-containers-that-are-broken). A Docker host that Dockhand
|
- **Container health:** on the 15-minute health pass the app also reads every container's state and status text and raises alerts for what is broken — see [Docker containers that are broken](docs/MONITORING.md#docker-containers-that-are-broken). A Docker host that Dockhand
|
||||||
cannot read (its Hawser agent is down) is an alert too, and its containers' alerts are held, not cleared. The Docker page shows each container's problem and has a **Leave alone** button.
|
cannot read (its Hawser agent is down) is an alert too, and its containers' alerts are held, not cleared. The Docker page shows each container's problem and has a **Leave alone** button.
|
||||||
|
- **Vulnerability scans:** every fifteen minutes the app also reads what Dockhand's scans have stored — `GET /api/vulnerabilities/count?env=` for each environment, and `GET /api/vulnerabilities?env=&severity=critical&limit=100` only where that count is above zero (needs the *images: view* permission). Nothing is scanned from here. **Critical** findings are a warning per Docker host (a setting turns it off — see [MONITORING.md](docs/MONITORING.md#docker-containers-that-are-broken)); the Docker page lists them. **Written from Dockhand's source and OpenAPI file, not tried against a live Dockhand** — its published docs show the lists at the top level of the answer, its code puts the counts under `summary`; the reader follows the code and reads nothing it does not find as *unknown*.
|
||||||
|
|
||||||
### Uptime Kuma
|
### Uptime Kuma
|
||||||
|
|
||||||
|
|||||||
@@ -127,8 +127,10 @@ All modules from the original plan are built:
|
|||||||
manages (one credential covers all of them), with
|
manages (one credential covers all of them), with
|
||||||
start/stop/restart actions, a host filter, image-update status per
|
start/stop/restart actions, a host filter, image-update status per
|
||||||
container (from Dockhand's own cached update check, plus a button
|
container (from Dockhand's own cached update check, plus a button
|
||||||
to trigger a fresh one), and a live running/total widget (with an
|
to trigger a fresh one), a live running/total widget (with an
|
||||||
updates-available count).
|
updates-available count), and the **vulnerabilities** Dockhand's own
|
||||||
|
scans found per host (critical ones listed; a warning is raised for
|
||||||
|
critical ones, and it can be switched off).
|
||||||
- **Uptime Kuma** — every monitor's status (up/down/pending/maintenance),
|
- **Uptime Kuma** — every monitor's status (up/down/pending/maintenance),
|
||||||
response time, and 24h/30d uptime, with certificate days remaining where
|
response time, and 24h/30d uptime, with certificate days remaining where
|
||||||
applicable. Each monitor is matched to one of your servers when its
|
applicable. Each monitor is matched to one of your servers when its
|
||||||
|
|||||||
@@ -124,6 +124,7 @@ even further, to admin only:
|
|||||||
**Settings search** is part of Settings, which is administrators'.
|
**Settings search** is part of Settings, which is administrators'.
|
||||||
- **Network devices** (Network → Network devices): everyone can see the devices, networks and Wi-Fi names read from UniFi controllers; **Read now** and putting a device under maintenance need operator; adding the UniFi
|
- **Network devices** (Network → Network devices): everyone can see the devices, networks and Wi-Fi names read from UniFi controllers; **Read now** and putting a device under maintenance need operator; adding the UniFi
|
||||||
integration itself (a key or login) is an administrator's job, like any integration.
|
integration itself (a key or login) is an administrator's job, like any integration.
|
||||||
|
- **Docker vulnerabilities** (Docker page): everyone signed in sees the counts and the critical findings from Dockhand's scans; nothing on the page starts a scan or changes anything. Switching the alert off is in Settings → Notifications (administrators).
|
||||||
- **Docker containers** (Docker page): everyone signed in sees each container and what is wrong with it; start, stop and restart need operator; **Leave alone** (no alerts for a container) and the list in Settings are administrators'.
|
- **Docker containers** (Docker page): everyone signed in sees each container and what is wrong with it; start, stop and restart need operator; **Leave alone** (no alerts for a container) and the list in Settings are administrators'.
|
||||||
- **Away mode** (Settings → Notifications → When): administrators start, plan, change and end it, since it decides who is told what; everyone signed in sees the banner that says someone is away and how many notifications are held.
|
- **Away mode** (Settings → Notifications → When): administrators start, plan, change and end it, since it decides who is told what; everyone signed in sees the banner that says someone is away and how many notifications are held.
|
||||||
- **Buttons on notifications** (Settings → Notifications → Who hears what): switching them on is administrator-only. A link in a notification needs **no role at all** — the signed token is the access, for the alerts in that message, for 24 hours, to acknowledge, silence for an hour or snooze, and nothing else.
|
- **Buttons on notifications** (Settings → Notifications → Who hears what): switching them on is administrator-only. A link in a notification needs **no role at all** — the signed token is the access, for the alerts in that message, for 24 hours, to acknowledge, silence for an hour or snooze, and nothing else.
|
||||||
|
|||||||
@@ -147,6 +147,7 @@ Every threshold here is a default; where it can be changed, the place is named.
|
|||||||
| **Jellyfin needs a look** | Jellyfin wants a restart, a scheduled task (a library scan) failed, or 10 or more sign-ins failed in a day | on, with the integration | Jellyfin page; [below](#jellyfin) |
|
| **Jellyfin needs a look** | Jellyfin wants a restart, a scheduled task (a library scan) failed, or 10 or more sign-ins failed in a day | on, with the integration | Jellyfin page; [below](#jellyfin) |
|
||||||
| **Duplicati backup failed / overdue** | A backup's last run failed (critical from the second in a row), or it has not succeeded for longer than it should | per job (2 days by default) | Duplicati page; [below](#duplicati-backups) |
|
| **Duplicati backup failed / overdue** | A backup's last run failed (critical from the second in a row), or it has not succeeded for longer than it should | per job (2 days by default) | Duplicati page; [below](#duplicati-backups) |
|
||||||
| **Docker container broken** | A container keeps restarting (critical), is unhealthy, stopped unexpectedly, or was created and never started; or Dockhand cannot reach a Docker host | on | Settings → Notifications; [below](#docker-containers-that-are-broken) |
|
| **Docker container broken** | A container keeps restarting (critical), is unhealthy, stopped unexpectedly, or was created and never started; or Dockhand cannot reach a Docker host | on | Settings → Notifications; [below](#docker-containers-that-are-broken) |
|
||||||
|
| **Docker image has critical vulnerabilities** | Dockhand's own scans of a Docker host's images found one or more *critical* findings (high, medium and low are only shown) | on | Settings → Notifications; [below](#docker-containers-that-are-broken) |
|
||||||
| **Automation failed** | A Semaphore template's or Gitea repository's latest run failed (a run still going or cancelled does not count) | | not configurable |
|
| **Automation failed** | A Semaphore template's or Gitea repository's latest run failed (a run still going or cancelled does not count) | | not configurable |
|
||||||
| **Integration failing** | An integration's calls have failed several times in a row ([Integrations](#integrations)) | 3 calls | Settings → Notifications |
|
| **Integration failing** | An integration's calls have failed several times in a row ([Integrations](#integrations)) | 3 calls | Settings → Notifications |
|
||||||
| **Endpoint check failing / certificate running out** | See [below](#endpoint-checks-and-heartbeats) | | per check |
|
| **Endpoint check failing / certificate running out** | See [below](#endpoint-checks-and-heartbeats) | | per check |
|
||||||
@@ -348,6 +349,9 @@ long stopped. The scanner containers Dockhand starts (`dockhand-grype-*`, `dockh
|
|||||||
Maintenance on the Docker integration holds these alerts like its other ones. If Dockhand itself cannot be read the containers' alerts are held rather than cleared (and the integration is raised as failing, as any other). Switch the whole thing off with *Warn when a
|
Maintenance on the Docker integration holds these alerts like its other ones. If Dockhand itself cannot be read the containers' alerts are held rather than cleared (and the integration is raised as failing, as any other). Switch the whole thing off with *Warn when a
|
||||||
Docker container is broken*. What the app has seen of each container is kept between looks (in `settings`, key `_internal:containerMemory`).
|
Docker container is broken*. What the app has seen of each container is kept between looks (in `settings`, key `_internal:containerMemory`).
|
||||||
|
|
||||||
|
**Critical vulnerabilities.** Dockhand can scan images (Grype or Trivy) — by hand, after a pull, or for a whole host. Every fifteen minutes the app reads **what Dockhand has already stored** (never starting a scan, so it costs Dockhand nothing) and raises a *warning* for each Docker host whose scanned images have one or more **critical** findings, naming the images and how many of the findings have a fix. High, medium and low findings are shown on the Docker page (*Vulnerabilities*, with the critical ones listed: CVE, package, installed and fixed version, image, and the containers that use it) but are never alerted. An image that was never scanned is not known to be clean — the page says how many of a host's images have a scan. A host whose count cannot be read is *unknown*, not clean, and says nothing. A reading older than 45 minutes is not believed: the alert is neither raised nor cleared and the Alerts page says it could not check. Maintenance on the Docker integration holds it, and an alert can be closed or snoozed like any other.
|
||||||
|
**To switch it off:** Settings → Notifications → What to send → *Warn about critical vulnerabilities in Docker images* (on by default). Dockhand can also send its own vulnerability notifications; with both on you hear twice. `/metrics` has `homelab_docker_vulnerabilities` (by `severity`) and `homelab_docker_images_scanned` for each host.
|
||||||
|
|
||||||
## Endpoint checks and heartbeats
|
## Endpoint checks and heartbeats
|
||||||
|
|
||||||
**Endpoint checks** (Monitoring → Endpoints) are for the simple "is it up?" that does not need a monitoring server of its own. Three kinds:
|
**Endpoint checks** (Monitoring → Endpoints) are for the simple "is it up?" that does not need a monitoring server of its own. Three kinds:
|
||||||
|
|||||||
@@ -299,6 +299,8 @@ All gauges. The same table is checked by a test against what `/metrics` really s
|
|||||||
| `homelab_nextcloud_cron_ok` | `name` | 1 if Nextcloud's own check says its background jobs ran within the last hour, else 0; absent when that could not be asked (no login, or before Nextcloud 28). |
|
| `homelab_nextcloud_cron_ok` | `name` | 1 if Nextcloud's own check says its background jobs ran within the last hour, else 0; absent when that could not be asked (no login, or before Nextcloud 28). |
|
||||||
| `homelab_nextcloud_maintenance_mode`, `homelab_nextcloud_app_updates` | `name` | 1 if Nextcloud is in maintenance mode, else 0; and how many apps have an update waiting (left out when not asked). Left out for a Nextcloud not read in the last twenty minutes. |
|
| `homelab_nextcloud_maintenance_mode`, `homelab_nextcloud_app_updates` | `name` | 1 if Nextcloud is in maintenance mode, else 0; and how many apps have an update waiting (left out when not asked). Left out for a Nextcloud not read in the last twenty minutes. |
|
||||||
| `homelab_nextcloud_free_bytes`, `homelab_nextcloud_active_users_24h` | `name` | Bytes free where the data is kept, and the users active in the last day (both need an administrator's app password). |
|
| `homelab_nextcloud_free_bytes`, `homelab_nextcloud_active_users_24h` | `name` | Bytes free where the data is kept, and the users active in the last day (both need an administrator's app password). |
|
||||||
|
| `homelab_docker_vulnerabilities` | `integration`, `host`, `severity` | The vulnerabilities in the scanned images of a Docker host, by severity (`critical`, `high`, `medium`, `low`), from Dockhand's stored scans. Left out for a host whose count could not be read, and when the reading is older than 45 minutes. |
|
||||||
|
| `homelab_docker_images_scanned` | `integration`, `host` | How many of a Docker host's images have a scan in Dockhand (the rest are not known to be clean). |
|
||||||
| `homelab_netbox_workers`, `homelab_netbox_devices`, `homelab_netbox_ip_addresses` | `name` | The background workers running, and the devices and IP addresses NetBox holds; left out for a NetBox not read in the last twenty minutes. |
|
| `homelab_netbox_workers`, `homelab_netbox_devices`, `homelab_netbox_ip_addresses` | `name` | The background workers running, and the devices and IP addresses NetBox holds; left out for a NetBox not read in the last twenty minutes. |
|
||||||
| `homelab_n8n_workflows`, `homelab_n8n_workflows_active`, `homelab_n8n_workflows_failing` | `name` | How many workflows n8n has, how many are switched on, and how many of those failed on their last run (absent when the runs could not be read). |
|
| `homelab_n8n_workflows`, `homelab_n8n_workflows_active`, `homelab_n8n_workflows_failing` | `name` | How many workflows n8n has, how many are switched on, and how many of those failed on their last run (absent when the runs could not be read). |
|
||||||
| `homelab_immich_photos`, `homelab_immich_videos`, `homelab_immich_library_bytes` | `name` | How many photos and videos Immich holds and what the library takes; left out for an Immich not read in the last twenty minutes. |
|
| `homelab_immich_photos`, `homelab_immich_videos`, `homelab_immich_library_bytes` | `name` | How many photos and videos Immich holds and what the library takes; left out for an Immich not read in the last twenty minutes. |
|
||||||
|
|||||||
@@ -0,0 +1,15 @@
|
|||||||
|
CREATE TABLE `dockhand_vulnerabilities` (
|
||||||
|
`integration_id` integer NOT NULL,
|
||||||
|
`environment_id` integer NOT NULL,
|
||||||
|
`environment_name` text NOT NULL,
|
||||||
|
`critical` integer,
|
||||||
|
`high` integer,
|
||||||
|
`medium` integer,
|
||||||
|
`low` integer,
|
||||||
|
`images_scanned` integer,
|
||||||
|
`total_images` integer,
|
||||||
|
`findings` text DEFAULT '[]' NOT NULL,
|
||||||
|
`updated_at` text NOT NULL,
|
||||||
|
PRIMARY KEY(`integration_id`, `environment_id`),
|
||||||
|
FOREIGN KEY (`integration_id`) REFERENCES `integrations`(`id`) ON UPDATE no action ON DELETE cascade
|
||||||
|
);
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -470,6 +470,13 @@
|
|||||||
"when": 1792961992575,
|
"when": 1792961992575,
|
||||||
"tag": "0066_careless_roulette",
|
"tag": "0066_careless_roulette",
|
||||||
"breakpoints": true
|
"breakpoints": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"idx": 67,
|
||||||
|
"version": "6",
|
||||||
|
"when": 1792963314439,
|
||||||
|
"tag": "0067_shallow_mastermind",
|
||||||
|
"breakpoints": true
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
@@ -1169,6 +1169,30 @@ export const haReadings = sqliteTable(
|
|||||||
(t) => [index("ha_readings_lookup_idx").on(t.integrationId, t.entityId, t.at)],
|
(t) => [index("ha_readings_lookup_idx").on(t.integrationId, t.entityId, t.at)],
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// ─── Docker vulnerability scans (from Dockhand) ─────────────────────────────
|
||||||
|
|
||||||
|
// What Dockhand's own stored scans add up to for one of its environments (a Docker host), as last read: one row for each (Dockhand integration, environment), replaced at every read.
|
||||||
|
// A null count means the environment's summary could not be read — which is not the same as no vulnerabilities. Read by services/dockerVulnSync.ts.
|
||||||
|
export const dockhandVulnerabilities = sqliteTable(
|
||||||
|
"dockhand_vulnerabilities",
|
||||||
|
{
|
||||||
|
integrationId: integer("integration_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => integrations.id, { onDelete: "cascade" }),
|
||||||
|
environmentId: integer("environment_id").notNull(), // Dockhand's own id for the Docker host
|
||||||
|
environmentName: text("environment_name").notNull(),
|
||||||
|
critical: integer("critical"),
|
||||||
|
high: integer("high"),
|
||||||
|
medium: integer("medium"),
|
||||||
|
low: integer("low"),
|
||||||
|
imagesScanned: integer("images_scanned"),
|
||||||
|
totalImages: integer("total_images"),
|
||||||
|
findings: text("findings").notNull().default("[]"), // JSON: the critical findings (cve, package, installedVersion, fixedVersion, imageName, containers), at most 100
|
||||||
|
updatedAt: text("updated_at").notNull(), // ISO
|
||||||
|
},
|
||||||
|
(t) => [primaryKey({ columns: [t.integrationId, t.environmentId] })],
|
||||||
|
);
|
||||||
|
|
||||||
// ─── Nextcloud ──────────────────────────────────────────────────────────────
|
// ─── Nextcloud ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
// How a Nextcloud (an integration of type "nextcloud") is right now, one row each, replaced at every read: its version, whether it is in maintenance mode or needs its database
|
// How a Nextcloud (an integration of type "nextcloud") is right now, one row each, replaced at every read: its version, whether it is in maintenance mode or needs its database
|
||||||
|
|||||||
@@ -37,6 +37,7 @@ import { initPaperlessSync } from "./services/paperlessSync.js";
|
|||||||
import { initN8nSync } from "./services/n8nSync.js";
|
import { initN8nSync } from "./services/n8nSync.js";
|
||||||
import { initNetBoxSync } from "./services/netboxSync.js";
|
import { initNetBoxSync } from "./services/netboxSync.js";
|
||||||
import { initNextcloudSync } from "./services/nextcloudSync.js";
|
import { initNextcloudSync } from "./services/nextcloudSync.js";
|
||||||
|
import { initDockerVulnSync } from "./services/dockerVulnSync.js";
|
||||||
import { backfillMetricRollups } from "./services/serverMetrics.js";
|
import { backfillMetricRollups } from "./services/serverMetrics.js";
|
||||||
import { backfillEndpointHourly } from "./services/endpointChecks.js";
|
import { backfillEndpointHourly } from "./services/endpointChecks.js";
|
||||||
import { initSelfBackupScheduler } from "./services/selfBackupScheduler.js";
|
import { initSelfBackupScheduler } from "./services/selfBackupScheduler.js";
|
||||||
@@ -95,6 +96,7 @@ initPaperlessSync();
|
|||||||
initN8nSync();
|
initN8nSync();
|
||||||
initNetBoxSync();
|
initNetBoxSync();
|
||||||
initNextcloudSync();
|
initNextcloudSync();
|
||||||
|
initDockerVulnSync();
|
||||||
// The hourly and daily figures are built from what is already kept, once, the first time after they were added (in the background: nothing waits for it).
|
// The hourly and daily figures are built from what is already kept, once, the first time after they were added (in the background: nothing waits for it).
|
||||||
void Promise.all([backfillMetricRollups(), backfillEndpointHourly()]).catch((err) => console.error("[history] couldn't build the hourly and daily figures:", err));
|
void Promise.all([backfillMetricRollups(), backfillEndpointHourly()]).catch((err) => console.error("[history] couldn't build the hourly and daily figures:", err));
|
||||||
|
|
||||||
|
|||||||
@@ -91,8 +91,75 @@ export interface DockhandContainerRead {
|
|||||||
unreachable: { environmentId: number; environmentName: string; error: string }[];
|
unreachable: { environmentId: number; environmentName: string; error: string }[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** What Dockhand's stored vulnerability scans add up to for one environment (GET /api/vulnerabilities/count). Counts are of findings, not images. */
|
||||||
|
export interface DockhandVulnSummary {
|
||||||
|
critical: number;
|
||||||
|
high: number;
|
||||||
|
medium: number;
|
||||||
|
low: number;
|
||||||
|
/** How many of the images on the host have a stored scan, and how many images there are: the rest have never been scanned, so nothing is known of them. */
|
||||||
|
imagesScanned: number | null;
|
||||||
|
totalImages: number | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One critical finding (GET /api/vulnerabilities), with the containers that use the image. */
|
||||||
|
export interface DockhandVulnFinding {
|
||||||
|
cve: string;
|
||||||
|
severity: string;
|
||||||
|
package: string | null;
|
||||||
|
installedVersion: string | null;
|
||||||
|
fixedVersion: string | null;
|
||||||
|
imageName: string;
|
||||||
|
containers: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface DockhandEnvironmentVulns {
|
||||||
|
environmentId: number;
|
||||||
|
environmentName: string;
|
||||||
|
/** Null when this environment's summary could not be read (an older Dockhand, a host it cannot reach) — which is not the same as no vulnerabilities. */
|
||||||
|
summary: DockhandVulnSummary | null;
|
||||||
|
/** The critical findings, at most MAX_VULN_FINDINGS; empty when there are none or the summary is unknown. */
|
||||||
|
critical: DockhandVulnFinding[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface DockhandVulnerabilityRead {
|
||||||
|
environments: DockhandEnvironmentVulns[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export const MAX_VULN_FINDINGS = 100;
|
||||||
|
|
||||||
|
const asObj = (v: unknown): Record<string, unknown> => (v !== null && typeof v === "object" && !Array.isArray(v) ? (v as Record<string, unknown>) : {});
|
||||||
|
const asStr = (v: unknown): string | null => (typeof v === "string" && v.trim() !== "" ? v.trim() : null);
|
||||||
|
const asCount = (v: unknown): number | null => (typeof v === "number" && Number.isFinite(v) && v >= 0 ? Math.floor(v) : null);
|
||||||
|
|
||||||
|
/** Pure: the severity summary from /api/vulnerabilities/count (the counts are under `summary`); null when it holds no critical count at all, which is not the same as zero. */
|
||||||
|
export function parseVulnSummary(raw: unknown): DockhandVulnSummary | null {
|
||||||
|
const root = asObj(raw);
|
||||||
|
const s = asObj(root.summary);
|
||||||
|
const critical = asCount(s.critical);
|
||||||
|
if (critical === null) return null;
|
||||||
|
return { critical, high: asCount(s.high) ?? 0, medium: asCount(s.medium) ?? 0, low: asCount(s.low) ?? 0, imagesScanned: asCount(s.imagesScanned), totalImages: asCount(s.totalImages) };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pure: the findings of one page of /api/vulnerabilities — each `{ cve, package, severity, installedVersion, fixedVersion, imageName, containers: [{ id, name }] }`. */
|
||||||
|
export function parseVulnFindings(raw: unknown): DockhandVulnFinding[] {
|
||||||
|
const list = asObj(raw).findings;
|
||||||
|
const out: DockhandVulnFinding[] = [];
|
||||||
|
for (const item of Array.isArray(list) ? list : []) {
|
||||||
|
const f = asObj(item);
|
||||||
|
const cve = asStr(f.cve) ?? asStr(f.id);
|
||||||
|
const imageName = asStr(f.imageName) ?? asStr(f.image);
|
||||||
|
if (!cve || !imageName) continue;
|
||||||
|
const containers = (Array.isArray(f.containers) ? f.containers : []).map((c) => (typeof c === "string" ? asStr(c) : asStr(asObj(c).name))).filter((n): n is string => n !== null);
|
||||||
|
out.push({ cve, severity: (asStr(f.severity) ?? "critical").toLowerCase(), package: asStr(f.package), installedVersion: asStr(f.installedVersion) ?? asStr(f.version), fixedVersion: asStr(f.fixedVersion), imageName, containers });
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
export interface DockhandAdapter {
|
export interface DockhandAdapter {
|
||||||
ping(): Promise<{ ok: boolean; latencyMs?: number; error?: string }>;
|
ping(): Promise<{ ok: boolean; latencyMs?: number; error?: string }>;
|
||||||
|
/** What Dockhand's own stored scans say, per environment. Reads only: no scan is started. */
|
||||||
|
readVulnerabilities(): Promise<DockhandVulnerabilityRead>;
|
||||||
listContainers(): Promise<DockhandContainer[]>;
|
listContainers(): Promise<DockhandContainer[]>;
|
||||||
listContainersDetailed(): Promise<DockhandContainerRead>;
|
listContainersDetailed(): Promise<DockhandContainerRead>;
|
||||||
startContainer(environmentId: number, containerId: string): Promise<void>;
|
startContainer(environmentId: number, containerId: string): Promise<void>;
|
||||||
@@ -179,6 +246,29 @@ export function createDockhandAdapter(config: DockhandConfig): DockhandAdapter {
|
|||||||
return { containers: perEnv.flat(), environments, unreachable };
|
return { containers: perEnv.flat(), environments, unreachable };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function readVulnerabilities(): Promise<DockhandVulnerabilityRead> {
|
||||||
|
const environments = await listEnvironments();
|
||||||
|
const perEnv = await Promise.all(
|
||||||
|
environments.map(async (env): Promise<DockhandEnvironmentVulns> => {
|
||||||
|
const known = { environmentId: env.id, environmentName: env.name };
|
||||||
|
let summary: DockhandVulnSummary | null = null;
|
||||||
|
try {
|
||||||
|
summary = parseVulnSummary(await api("GET", `/api/vulnerabilities/count?env=${numericSegment(env.id, "environment ID")}`));
|
||||||
|
} catch {
|
||||||
|
return { ...known, summary: null, critical: [] }; // an older Dockhand, or a host it cannot reach: unknown, not clean
|
||||||
|
}
|
||||||
|
if (!summary || summary.critical === 0) return { ...known, summary, critical: [] };
|
||||||
|
try {
|
||||||
|
const page = await api("GET", `/api/vulnerabilities?env=${numericSegment(env.id, "environment ID")}&severity=critical&limit=${MAX_VULN_FINDINGS}`);
|
||||||
|
return { ...known, summary, critical: parseVulnFindings(page).slice(0, MAX_VULN_FINDINGS) };
|
||||||
|
} catch {
|
||||||
|
return { ...known, summary, critical: [] }; // the count is what alerts; the list is the detail
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
return { environments: perEnv };
|
||||||
|
}
|
||||||
|
|
||||||
async function listContainers(): Promise<DockhandContainer[]> {
|
async function listContainers(): Promise<DockhandContainer[]> {
|
||||||
return (await listContainersDetailed()).containers;
|
return (await listContainersDetailed()).containers;
|
||||||
}
|
}
|
||||||
@@ -224,5 +314,5 @@ export function createDockhandAdapter(config: DockhandConfig): DockhandAdapter {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return withDiagLogging("dockhand", { ping, listContainers, listContainersDetailed, startContainer, stopContainer, restartContainer, checkForUpdates });
|
return withDiagLogging("dockhand", { ping, readVulnerabilities, listContainers, listContainersDetailed, startContainer, stopContainer, restartContainer, checkForUpdates });
|
||||||
}
|
}
|
||||||
@@ -2,7 +2,7 @@ import { Router, type Request, type Response } from "express";
|
|||||||
import { eq } from "drizzle-orm";
|
import { eq } from "drizzle-orm";
|
||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
import { db } from "../db/client.js";
|
import { db } from "../db/client.js";
|
||||||
import { integrations, integrationCredentials, integrationTypes, servers, type IntegrationType } from "../db/schema.js";
|
import { dockhandVulnerabilities, integrations, integrationCredentials, integrationTypes, servers, type IntegrationType } from "../db/schema.js";
|
||||||
import { requireAuth, requireRole } from "../auth/middleware.js";
|
import { requireAuth, requireRole } from "../auth/middleware.js";
|
||||||
import { recordAudit } from "../services/audit.js";
|
import { recordAudit } from "../services/audit.js";
|
||||||
import { encryptSecret } from "../crypto.js";
|
import { encryptSecret } from "../crypto.js";
|
||||||
@@ -37,6 +37,7 @@ import { forgetPaperless, syncPaperless } from "../services/paperlessSync.js";
|
|||||||
import { forgetN8n, syncN8n } from "../services/n8nSync.js";
|
import { forgetN8n, syncN8n } from "../services/n8nSync.js";
|
||||||
import { forgetNetBox, syncNetBox } from "../services/netboxSync.js";
|
import { forgetNetBox, syncNetBox } from "../services/netboxSync.js";
|
||||||
import { forgetNextcloud, syncNextcloud } from "../services/nextcloudSync.js";
|
import { forgetNextcloud, syncNextcloud } from "../services/nextcloudSync.js";
|
||||||
|
import { dockerVulnSyncStatuses, findingsOf, forgetDockerVulns, isDockerVulnFresh, syncDockerVulns } from "../services/dockerVulnSync.js";
|
||||||
import { invalidateAlertsCache } from "../services/alerts.js";
|
import { invalidateAlertsCache } from "../services/alerts.js";
|
||||||
import { getSettings, updateSettings } from "../services/settingsStore.js";
|
import { getSettings, updateSettings } from "../services/settingsStore.js";
|
||||||
import { asyncHandler } from "../utils/asyncHandler.js";
|
import { asyncHandler } from "../utils/asyncHandler.js";
|
||||||
@@ -147,6 +148,7 @@ integrationsRouter.post("/", requireRole("admin"), asyncHandler(async (req, res)
|
|||||||
if (type === "n8n") void syncN8n(created.id).then(() => invalidateAlertsCache());
|
if (type === "n8n") void syncN8n(created.id).then(() => invalidateAlertsCache());
|
||||||
if (type === "netbox") void syncNetBox(created.id).then(() => invalidateAlertsCache());
|
if (type === "netbox") void syncNetBox(created.id).then(() => invalidateAlertsCache());
|
||||||
if (type === "nextcloud") void syncNextcloud(created.id).then(() => invalidateAlertsCache());
|
if (type === "nextcloud") void syncNextcloud(created.id).then(() => invalidateAlertsCache());
|
||||||
|
if (type === "dockhand") void syncDockerVulns(created.id).then(() => invalidateAlertsCache());
|
||||||
|
|
||||||
res.status(201).json({
|
res.status(201).json({
|
||||||
integration: {
|
integration: {
|
||||||
@@ -258,6 +260,7 @@ integrationsRouter.patch("/:id", requireRole("admin"), asyncHandler(async (req,
|
|||||||
if (updated.type === "n8n" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncN8n(updated.id).then(() => invalidateAlertsCache());
|
if (updated.type === "n8n" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncN8n(updated.id).then(() => invalidateAlertsCache());
|
||||||
if (updated.type === "netbox" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncNetBox(updated.id).then(() => invalidateAlertsCache());
|
if (updated.type === "netbox" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncNetBox(updated.id).then(() => invalidateAlertsCache());
|
||||||
if (updated.type === "nextcloud" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncNextcloud(updated.id).then(() => invalidateAlertsCache());
|
if (updated.type === "nextcloud" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncNextcloud(updated.id).then(() => invalidateAlertsCache());
|
||||||
|
if (updated.type === "dockhand" && updated.enabled && (parsed.data.config || parsed.data.enabled === true)) void syncDockerVulns(updated.id).then(() => invalidateAlertsCache());
|
||||||
|
|
||||||
res.json({
|
res.json({
|
||||||
integration: {
|
integration: {
|
||||||
@@ -345,6 +348,7 @@ integrationsRouter.delete("/:id", requireRole("admin"), asyncHandler(async (req,
|
|||||||
if (existing.type === "n8n") await forgetN8n(id);
|
if (existing.type === "n8n") await forgetN8n(id);
|
||||||
if (existing.type === "netbox") await forgetNetBox(id);
|
if (existing.type === "netbox") await forgetNetBox(id);
|
||||||
if (existing.type === "nextcloud") await forgetNextcloud(id);
|
if (existing.type === "nextcloud") await forgetNextcloud(id);
|
||||||
|
if (existing.type === "dockhand") await forgetDockerVulns(id);
|
||||||
await db.delete(integrations).where(eq(integrations.id, id));
|
await db.delete(integrations).where(eq(integrations.id, id));
|
||||||
if (existing.credentialId) {
|
if (existing.credentialId) {
|
||||||
await db.delete(integrationCredentials).where(eq(integrationCredentials.id, existing.credentialId));
|
await db.delete(integrationCredentials).where(eq(integrationCredentials.id, existing.credentialId));
|
||||||
@@ -575,6 +579,24 @@ async function requireDockhandAdapter(req: Request, res: Response) {
|
|||||||
return { integration: loaded.integration, adapter: createDockhandAdapter(loaded.config as any) };
|
return { integration: loaded.integration, adapter: createDockhandAdapter(loaded.config as any) };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// What Dockhand's own vulnerability scans add up to for each of its Docker hosts, as last read (nothing is asked of Dockhand here, and nothing is scanned).
|
||||||
|
integrationsRouter.get("/:id/dockhand/vulnerabilities", asyncHandler(async (req, res) => {
|
||||||
|
const id = Number(req.params.id);
|
||||||
|
const loaded = await loadIntegrationConfig(id);
|
||||||
|
if (!loaded) return res.status(404).json({ error: "not_found" });
|
||||||
|
if (loaded.integration.type !== "dockhand") return res.status(400).json({ error: "wrong_type" });
|
||||||
|
const [rows, statuses, settings] = await Promise.all([db.select().from(dockhandVulnerabilities).where(eq(dockhandVulnerabilities.integrationId, id)), dockerVulnSyncStatuses(), getSettings()]);
|
||||||
|
const status = statuses[id] ?? null;
|
||||||
|
res.json({
|
||||||
|
alerts: settings.healthChecks.vulnerabilityAlerts,
|
||||||
|
status,
|
||||||
|
fresh: isDockerVulnFresh(status ?? undefined, Date.now()),
|
||||||
|
environments: rows
|
||||||
|
.sort((a, b) => a.environmentName.localeCompare(b.environmentName))
|
||||||
|
.map((r) => ({ environmentId: r.environmentId, environmentName: r.environmentName, critical: r.critical, high: r.high, medium: r.medium, low: r.low, imagesScanned: r.imagesScanned, totalImages: r.totalImages, findings: findingsOf(r), updatedAt: r.updatedAt })),
|
||||||
|
});
|
||||||
|
}));
|
||||||
|
|
||||||
integrationsRouter.get("/:id/dockhand/containers", asyncHandler(async (req, res) => {
|
integrationsRouter.get("/:id/dockhand/containers", asyncHandler(async (req, res) => {
|
||||||
const found = await requireDockhandAdapter(req, res);
|
const found = await requireDockhandAdapter(req, res);
|
||||||
if (!found) return;
|
if (!found) return;
|
||||||
|
|||||||
@@ -144,7 +144,7 @@ const updateSchema = z.object({
|
|||||||
.partial()
|
.partial()
|
||||||
.optional(),
|
.optional(),
|
||||||
healthChecks: z
|
healthChecks: z
|
||||||
.object({ serverOfflineMinutes: z.number().int().min(15).max(10080), diskUsagePercent: z.number().int().min(50).max(99), domainWarnDays: z.number().int().min(1).max(365), diskForecastDays: z.number().int().min(0).max(90), backupStaleHours: z.number().int().min(0).max(2160), restoreTestDays: z.number().int().min(0).max(730), newPortAlerts: z.boolean(), flapRaises: z.number().int().min(0).max(20), deviceOfflineMinutes: z.number().int().min(5).max(1440), osSupportWarnDays: z.number().int().min(0).max(365), baselineAlerts: z.boolean(), containerAlerts: z.boolean(), containerIgnore: z.array(z.string().trim().min(1).max(100)).max(200) })
|
.object({ serverOfflineMinutes: z.number().int().min(15).max(10080), diskUsagePercent: z.number().int().min(50).max(99), domainWarnDays: z.number().int().min(1).max(365), diskForecastDays: z.number().int().min(0).max(90), backupStaleHours: z.number().int().min(0).max(2160), restoreTestDays: z.number().int().min(0).max(730), newPortAlerts: z.boolean(), flapRaises: z.number().int().min(0).max(20), deviceOfflineMinutes: z.number().int().min(5).max(1440), osSupportWarnDays: z.number().int().min(0).max(365), baselineAlerts: z.boolean(), containerAlerts: z.boolean(), vulnerabilityAlerts: z.boolean().default(true), containerIgnore: z.array(z.string().trim().min(1).max(100)).max(200) })
|
||||||
.partial()
|
.partial()
|
||||||
.optional(),
|
.optional(),
|
||||||
quietHours: z
|
quietHours: z
|
||||||
|
|||||||
@@ -119,6 +119,8 @@ function originOfConditionKey(key: string): { type: string; id: number | null; c
|
|||||||
if (m) return { type: "slo", id: Number(m[1]), category: "monitoring" };
|
if (m) return { type: "slo", id: Number(m[1]), category: "monitoring" };
|
||||||
m = /^container-(?:restarting|unhealthy|stopped|created|host):(\d+):/.exec(key);
|
m = /^container-(?:restarting|unhealthy|stopped|created|host):(\d+):/.exec(key);
|
||||||
if (m) return { type: "dockhand", id: Number(m[1]), category: "services" };
|
if (m) return { type: "dockhand", id: Number(m[1]), category: "services" };
|
||||||
|
m = /^docker-vuln:(\d+):/.exec(key);
|
||||||
|
if (m) return { type: "dockhand", id: Number(m[1]), category: "services" };
|
||||||
m = /^duplicati-(?:failed|stale):(\d+)/.exec(key);
|
m = /^duplicati-(?:failed|stale):(\d+)/.exec(key);
|
||||||
if (m) return { type: "duplicati", id: Number(m[1]), category: "backup" };
|
if (m) return { type: "duplicati", id: Number(m[1]), category: "backup" };
|
||||||
m = /^nextcloud-(?:maintenance|upgrade|disk|cron|checks):(\d+)/.exec(key);
|
m = /^nextcloud-(?:maintenance|upgrade|disk|cron|checks):(\d+)/.exec(key);
|
||||||
@@ -272,7 +274,7 @@ export async function collectAlerts(): Promise<AlertsReport> {
|
|||||||
const m = /^(proxmox|synology|unifi|ups|dockhand|jellyfin|authentik|prometheus|immich|paperless|nextcloud|netbox|n8n):(\d+)(?::(.+))?$/.exec(h);
|
const m = /^(proxmox|synology|unifi|ups|dockhand|jellyfin|authentik|prometheus|immich|paperless|nextcloud|netbox|n8n):(\d+)(?::(.+))?$/.exec(h);
|
||||||
if (!m || (m[3] && held.has(`${m[1]}:${m[2]}`))) continue;
|
if (!m || (m[3] && held.has(`${m[1]}:${m[2]}`))) continue;
|
||||||
const row = intById.get(Number(m[2]));
|
const row = intById.get(Number(m[2]));
|
||||||
cannotCheck(`${SOURCES[m[1]].label} “${row?.name ?? `#${m[2]}`}”${m[3] ? ` (node ${m[3]})` : ""}`, "couldn't be read — see the Diagnostic Log", "Server and storage health");
|
cannotCheck(`${SOURCES[m[1]].label} “${row?.name ?? `#${m[2]}`}”${m[3] === "vuln" ? " (vulnerability scans)" : m[3] ? ` (node ${m[3]})` : ""}`, "couldn't be read — see the Diagnostic Log", "Server and storage health");
|
||||||
}
|
}
|
||||||
}).finally(() => {
|
}).finally(() => {
|
||||||
offlineKnown(new Set());
|
offlineKnown(new Set());
|
||||||
|
|||||||
@@ -0,0 +1,149 @@
|
|||||||
|
/**
|
||||||
|
* Keeps what Dockhand's vulnerability scans say up to date: every fifteen minutes each Dockhand integration is asked, for each of its Docker hosts, how many findings its stored scans
|
||||||
|
* add up to (and the critical ones, with the containers that use the image), and that is stored. Pages and alerts are built from what is stored, so none of them talks to Dockhand.
|
||||||
|
*
|
||||||
|
* Nothing here starts a scan: Dockhand scans when it is told to (by hand, after a pull, or for a whole host), and this only reads what it kept. So an image that was never scanned is
|
||||||
|
* not known to be clean — the page says how many of a host's images have a scan — and an environment whose summary could not be read is *unknown*, never *none*.
|
||||||
|
*
|
||||||
|
* What is raised is **critical** findings only (a host with one or more is a warning), and only while the setting for it is on (Settings → Notifications). Dockhand can also notify
|
||||||
|
* about vulnerabilities itself; if both are switched on you hear twice.
|
||||||
|
*/
|
||||||
|
import { and, eq, notInArray } from "drizzle-orm";
|
||||||
|
import { db } from "../db/client.js";
|
||||||
|
import { dockhandVulnerabilities, integrations } from "../db/schema.js";
|
||||||
|
import { createDockhandAdapter, type DockhandEnvironmentVulns, type DockhandVulnFinding } from "../integrations/dockhand/adapter.js";
|
||||||
|
import { loadIntegrationConfig } from "../integrations/loadIntegration.js";
|
||||||
|
import { getInternalFlag, setInternalFlag } from "./settingsStore.js";
|
||||||
|
|
||||||
|
export const SYNC_INTERVAL_MS = 15 * 60_000;
|
||||||
|
/** After this long without a good read, what is stored is no longer believed. */
|
||||||
|
export const STALE_AFTER_MS = 45 * 60_000;
|
||||||
|
const STATUS_FLAG = "dockerVulnSync";
|
||||||
|
|
||||||
|
export interface DockerVulnSyncStatus {
|
||||||
|
/** The last read that worked (kept when a later one failed). */
|
||||||
|
at: string;
|
||||||
|
ok: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readStatuses(): Promise<Record<string, DockerVulnSyncStatus>> {
|
||||||
|
try {
|
||||||
|
const raw = await getInternalFlag(STATUS_FLAG);
|
||||||
|
return raw ? (JSON.parse(raw) as Record<string, DockerVulnSyncStatus>) : {};
|
||||||
|
} catch {
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function dockerVulnSyncStatuses(): Promise<Record<number, DockerVulnSyncStatus>> {
|
||||||
|
return Object.fromEntries(Object.entries(await readStatuses()).map(([k, v]) => [Number(k), v]));
|
||||||
|
}
|
||||||
|
|
||||||
|
export const isDockerVulnFresh = (status: DockerVulnSyncStatus | undefined, nowMs: number) => !!status && status.ok && nowMs - Date.parse(status.at) <= STALE_AFTER_MS;
|
||||||
|
|
||||||
|
export type StateRow = typeof dockhandVulnerabilities.$inferSelect;
|
||||||
|
|
||||||
|
export const findingsOf = (row: Pick<StateRow, "findings">): DockhandVulnFinding[] => {
|
||||||
|
try {
|
||||||
|
const v = JSON.parse(row.findings);
|
||||||
|
return Array.isArray(v) ? (v as DockhandVulnFinding[]) : [];
|
||||||
|
} catch {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Writes down one read, replacing the last one for this integration (a host Dockhand no longer lists is dropped). */
|
||||||
|
export async function storeRead(integrationId: number, environments: DockhandEnvironmentVulns[], now: Date = new Date()): Promise<void> {
|
||||||
|
for (const e of environments) {
|
||||||
|
const values = {
|
||||||
|
environmentName: e.environmentName,
|
||||||
|
critical: e.summary?.critical ?? null,
|
||||||
|
high: e.summary?.high ?? null,
|
||||||
|
medium: e.summary?.medium ?? null,
|
||||||
|
low: e.summary?.low ?? null,
|
||||||
|
imagesScanned: e.summary?.imagesScanned ?? null,
|
||||||
|
totalImages: e.summary?.totalImages ?? null,
|
||||||
|
findings: JSON.stringify(e.critical),
|
||||||
|
updatedAt: now.toISOString(),
|
||||||
|
};
|
||||||
|
await db.insert(dockhandVulnerabilities).values({ integrationId, environmentId: e.environmentId, ...values }).onConflictDoUpdate({ target: [dockhandVulnerabilities.integrationId, dockhandVulnerabilities.environmentId], set: values });
|
||||||
|
}
|
||||||
|
const ids = environments.map((e) => e.environmentId);
|
||||||
|
await db.delete(dockhandVulnerabilities).where(ids.length > 0 ? and(eq(dockhandVulnerabilities.integrationId, integrationId), notInArray(dockhandVulnerabilities.environmentId, ids)) : eq(dockhandVulnerabilities.integrationId, integrationId));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Reads one Dockhand's scans and stores them. Never throws: how it went is the answer, and what was stored is left alone when it failed. */
|
||||||
|
export async function syncDockerVulns(integrationId: number, now: Date = new Date(), read?: (config: Record<string, unknown>) => Promise<{ environments: DockhandEnvironmentVulns[] }>): Promise<DockerVulnSyncStatus> {
|
||||||
|
let status: DockerVulnSyncStatus;
|
||||||
|
try {
|
||||||
|
const loaded = await loadIntegrationConfig(integrationId);
|
||||||
|
if (!loaded || loaded.integration.type !== "dockhand") throw new Error("not a Dockhand integration");
|
||||||
|
const result = await (read ? read(loaded.config) : createDockhandAdapter(loaded.config as never).readVulnerabilities());
|
||||||
|
await storeRead(integrationId, result.environments, now);
|
||||||
|
status = { at: now.toISOString(), ok: true, error: null };
|
||||||
|
} catch (err) {
|
||||||
|
const previous = (await readStatuses())[String(integrationId)];
|
||||||
|
status = { at: previous?.at ?? now.toISOString(), ok: false, error: err instanceof Error ? err.message : String(err) };
|
||||||
|
console.error(`[docker-vulns] couldn't read integration ${integrationId}:`, status.error);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const all = await readStatuses();
|
||||||
|
all[String(integrationId)] = status;
|
||||||
|
await setInternalFlag(STATUS_FLAG, JSON.stringify(all));
|
||||||
|
} catch (err) {
|
||||||
|
console.error("[docker-vulns] couldn't record the result:", err);
|
||||||
|
}
|
||||||
|
return status;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function syncAllDockerVulns(now: Date = new Date()): Promise<Record<number, DockerVulnSyncStatus>> {
|
||||||
|
const rows = await db.select({ id: integrations.id }).from(integrations).where(and(eq(integrations.type, "dockhand"), eq(integrations.enabled, true)));
|
||||||
|
const out: Record<number, DockerVulnSyncStatus> = {};
|
||||||
|
for (const r of rows) out[r.id] = await syncDockerVulns(r.id, now);
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function forgetDockerVulns(integrationId: number): Promise<void> {
|
||||||
|
const all = await readStatuses();
|
||||||
|
if (String(integrationId) in all) {
|
||||||
|
delete all[String(integrationId)];
|
||||||
|
await setInternalFlag(STATUS_FLAG, JSON.stringify(all));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── What is wrong ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
export interface DockerVulnProblem {
|
||||||
|
key: string;
|
||||||
|
message: string;
|
||||||
|
severity: "warning";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pure: a warning for each Docker host whose stored scans have critical findings. A host whose summary is unknown says nothing. */
|
||||||
|
export function dockerVulnProblems(integrationId: number, rows: Pick<StateRow, "environmentId" | "environmentName" | "critical" | "high" | "findings">[]): DockerVulnProblem[] {
|
||||||
|
const out: DockerVulnProblem[] = [];
|
||||||
|
for (const r of [...rows].sort((a, b) => a.environmentName.localeCompare(b.environmentName))) {
|
||||||
|
if (r.critical === null || r.critical <= 0) continue;
|
||||||
|
const findings = findingsOf(r);
|
||||||
|
const images = [...new Set(findings.map((f) => f.imageName))];
|
||||||
|
const fixable = findings.filter((f) => f.fixedVersion).length;
|
||||||
|
const where = images.length === 0 ? "" : ` in ${images.length === 1 ? "1 image" : `${images.length}${findings.length >= r.critical ? "" : "+"} images`}: ${images.slice(0, 3).join(", ")}${images.length > 3 ? ` and ${images.length - 3} more` : ""}`;
|
||||||
|
out.push({
|
||||||
|
key: `docker-vuln:${integrationId}:${r.environmentId}`,
|
||||||
|
severity: "warning",
|
||||||
|
message: `${r.environmentName}: ${r.critical} critical ${r.critical === 1 ? "vulnerability" : "vulnerabilities"}${where}${findings.length > 0 ? ` (${fixable} of the first ${findings.length} ${fixable === 1 ? "has" : "have"} a fix)` : ""}`,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
let timer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
|
||||||
|
export function initDockerVulnSync(): void {
|
||||||
|
if (timer) return;
|
||||||
|
const pass = () => void syncAllDockerVulns().catch((err) => console.error("[docker-vulns] pass failed:", err));
|
||||||
|
timer = setInterval(pass, SYNC_INTERVAL_MS);
|
||||||
|
setTimeout(pass, 27_000);
|
||||||
|
console.log(`[docker-vulns] reading Dockhand's vulnerability scans every ${SYNC_INTERVAL_MS / 60000} minutes`);
|
||||||
|
}
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
import { and, eq } from "drizzle-orm";
|
import { and, eq } from "drizzle-orm";
|
||||||
import { db } from "../db/client.js";
|
import { db } from "../db/client.js";
|
||||||
import { authentikState, n8nState, netboxState, nextcloudState, paperlessState, immichState, prometheusState, integrations, jellyfinState, networkDevices, serverPorts, servers, upsState } from "../db/schema.js";
|
import { authentikState, n8nState, netboxState, nextcloudState, paperlessState, immichState, prometheusState, integrations, jellyfinState, networkDevices, serverPorts, servers, upsState , dockhandVulnerabilities } from "../db/schema.js";
|
||||||
import { loadIntegrationConfig } from "../integrations/loadIntegration.js";
|
import { loadIntegrationConfig } from "../integrations/loadIntegration.js";
|
||||||
import { createProxmoxAdapter, type ProxmoxNodeStats } from "../integrations/proxmox/adapter.js";
|
import { createProxmoxAdapter, type ProxmoxNodeStats } from "../integrations/proxmox/adapter.js";
|
||||||
import { createSynologyAdapter, type SynologyStorageInfo } from "../integrations/synology/adapter.js";
|
import { createSynologyAdapter, type SynologyStorageInfo } from "../integrations/synology/adapter.js";
|
||||||
@@ -28,6 +28,7 @@ import { isPaperlessFresh, paperlessProblems, paperlessSyncStatuses, type Paperl
|
|||||||
import { isN8nFresh, n8nProblems, n8nSyncStatuses, type N8nProblem } from "./n8nSync.js";
|
import { isN8nFresh, n8nProblems, n8nSyncStatuses, type N8nProblem } from "./n8nSync.js";
|
||||||
import { isNetBoxFresh, netboxProblems, netboxSyncStatuses, type NetBoxProblem } from "./netboxSync.js";
|
import { isNetBoxFresh, netboxProblems, netboxSyncStatuses, type NetBoxProblem } from "./netboxSync.js";
|
||||||
import { isNextcloudFresh, nextcloudProblems, nextcloudSyncStatuses, type NextcloudProblem } from "./nextcloudSync.js";
|
import { isNextcloudFresh, nextcloudProblems, nextcloudSyncStatuses, type NextcloudProblem } from "./nextcloudSync.js";
|
||||||
|
import { dockerVulnProblems, dockerVulnSyncStatuses, isDockerVulnFresh, type DockerVulnProblem } from "./dockerVulnSync.js";
|
||||||
import { evaluateContainers, loadContainerMemory, saveContainerMemory, type ContainerMemory, type DockerSnapshot } from "./containerHealth.js";
|
import { evaluateContainers, loadContainerMemory, saveContainerMemory, type ContainerMemory, type DockerSnapshot } from "./containerHealth.js";
|
||||||
import { recordStorageSamples } from "./storageHistory.js";
|
import { recordStorageSamples } from "./storageHistory.js";
|
||||||
import { isUpsFresh, upsProblems, upsSyncStatuses, type UpsStateRow } from "./upsSync.js";
|
import { isUpsFresh, upsProblems, upsSyncStatuses, type UpsStateRow } from "./upsSync.js";
|
||||||
@@ -116,6 +117,8 @@ export interface HealthSnapshot {
|
|||||||
/** Each Docker integration's containers as last read, with what was remembered of them (services/containerHealth.ts). */
|
/** Each Docker integration's containers as last read, with what was remembered of them (services/containerHealth.ts). */
|
||||||
docker?: DockerSnapshot[];
|
docker?: DockerSnapshot[];
|
||||||
dockerMemory?: ContainerMemory;
|
dockerMemory?: ContainerMemory;
|
||||||
|
/** Critical vulnerabilities in the images of each Dockhand's Docker hosts, from its last scans, only while that reading is recent (services/dockerVulnSync.ts). */
|
||||||
|
dockerVulns?: { integrationId: number; problems: DockerVulnProblem[] }[];
|
||||||
/** Containers to leave alone. */
|
/** Containers to leave alone. */
|
||||||
dockerIgnore?: string[];
|
dockerIgnore?: string[];
|
||||||
/** The clock the container judgements are made on (the pass's, so a test can set it). */
|
/** The clock the container judgements are made on (the pass's, so a test can set it). */
|
||||||
@@ -254,6 +257,9 @@ export function evaluateHealth(snapshot: HealthSnapshot, thresholds: Thresholds,
|
|||||||
for (const c of evaluateContainers(snapshot.docker, snapshot.dockerMemory ?? {}, snapshot.dockerNow ?? now, snapshot.dockerIgnore ?? []).conditions) out.push(c);
|
for (const c of evaluateContainers(snapshot.docker, snapshot.dockerMemory ?? {}, snapshot.dockerNow ?? now, snapshot.dockerIgnore ?? []).conditions) out.push(c);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Critical vulnerabilities in the images Dockhand has scanned.
|
||||||
|
for (const v of snapshot.dockerVulns ?? []) for (const p of v.problems) out.push({ key: p.key, source: `dockhand:${v.integrationId}:vuln`, message: p.message, severity: p.severity });
|
||||||
|
|
||||||
// A service-level target that has used most of its month's error budget, or all of it.
|
// A service-level target that has used most of its month's error budget, or all of it.
|
||||||
for (const f of snapshot.levels ?? []) out.push({ key: levelKey(f), source: "slo", message: f.message, severity: f.severity });
|
for (const f of snapshot.levels ?? []) out.push({ key: levelKey(f), source: "slo", message: f.message, severity: f.severity });
|
||||||
|
|
||||||
@@ -646,6 +652,23 @@ export async function collectSnapshot(): Promise<{ snapshot: HealthSnapshot; hel
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Vulnerabilities Dockhand's scans found, as the sync pass stored them (only critical ones are raised). Switched off with the setting; a reading gone stale is held, not cleared.
|
||||||
|
if (checks.vulnerabilityAlerts) {
|
||||||
|
try {
|
||||||
|
const statuses = await dockerVulnSyncStatuses();
|
||||||
|
for (const row of await db.select({ id: integrations.id }).from(integrations).where(and(eq(integrations.type, "dockhand"), eq(integrations.enabled, true)))) {
|
||||||
|
const stored = await db.select().from(dockhandVulnerabilities).where(eq(dockhandVulnerabilities.integrationId, row.id));
|
||||||
|
if (stored.length === 0 || !isDockerVulnFresh(statuses[row.id], Date.now())) {
|
||||||
|
held.add(`dockhand:${row.id}:vuln`);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
(snapshot.dockerVulns ??= []).push({ integrationId: row.id, problems: dockerVulnProblems(row.id, stored) });
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
console.error("[health] couldn't judge the vulnerability scans:", err instanceof Error ? err.message : err);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Service-level targets: how much of the month's error budget is used. A failure here costs only this check.
|
// Service-level targets: how much of the month's error budget is used. A failure here costs only this check.
|
||||||
try {
|
try {
|
||||||
snapshot.levels = await levelFindings();
|
snapshot.levels = await levelFindings();
|
||||||
|
|||||||
@@ -31,6 +31,8 @@ export function subjectOfConditionKey(key: string): string | null {
|
|||||||
if (m) return `server:${m[1]}`;
|
if (m) return `server:${m[1]}`;
|
||||||
m = /^container-(?:restarting|unhealthy|stopped|created|host):(\d+):/.exec(key);
|
m = /^container-(?:restarting|unhealthy|stopped|created|host):(\d+):/.exec(key);
|
||||||
if (m) return `integration:${m[1]}`;
|
if (m) return `integration:${m[1]}`;
|
||||||
|
m = /^docker-vuln:(\d+):/.exec(key);
|
||||||
|
if (m) return `integration:${m[1]}`;
|
||||||
m = /^jellyfin-(?:restart|task|logins):(\d+)/.exec(key);
|
m = /^jellyfin-(?:restart|task|logins):(\d+)/.exec(key);
|
||||||
if (m) return `integration:${m[1]}`;
|
if (m) return `integration:${m[1]}`;
|
||||||
m = /^nextcloud-(?:maintenance|upgrade|disk|cron|checks):(\d+)/.exec(key);
|
m = /^nextcloud-(?:maintenance|upgrade|disk|cron|checks):(\d+)/.exec(key);
|
||||||
|
|||||||
@@ -5,12 +5,13 @@
|
|||||||
*/
|
*/
|
||||||
import { eq } from "drizzle-orm";
|
import { eq } from "drizzle-orm";
|
||||||
import { db } from "../db/client.js";
|
import { db } from "../db/client.js";
|
||||||
import { assets, endpointChecks, heartbeats, integrations, maintenanceWindows, secrets, servers, upsState, jellyfinState, authentikState, immichState, paperlessState, nextcloudState, netboxState, n8nState } from "../db/schema.js";
|
import { assets, endpointChecks, heartbeats, integrations, maintenanceWindows, secrets, servers, upsState, jellyfinState, authentikState, immichState, paperlessState, nextcloudState, netboxState, n8nState, dockhandVulnerabilities } from "../db/schema.js";
|
||||||
import { isUpsFresh, upsSyncStatuses } from "./upsSync.js";
|
import { isUpsFresh, upsSyncStatuses } from "./upsSync.js";
|
||||||
import { levelStatuses } from "./serviceLevels.js";
|
import { levelStatuses } from "./serviceLevels.js";
|
||||||
import { backupState, listAll as listDuplicati } from "./duplicati.js";
|
import { backupState, listAll as listDuplicati } from "./duplicati.js";
|
||||||
import { isJellyfinFresh, jellyfinSyncStatuses, streamsOf } from "./jellyfinSync.js";
|
import { isJellyfinFresh, jellyfinSyncStatuses, streamsOf } from "./jellyfinSync.js";
|
||||||
import { isNextcloudFresh, nextcloudSyncStatuses } from "./nextcloudSync.js";
|
import { isNextcloudFresh, nextcloudSyncStatuses } from "./nextcloudSync.js";
|
||||||
|
import { dockerVulnSyncStatuses, isDockerVulnFresh } from "./dockerVulnSync.js";
|
||||||
import { isNetBoxFresh, netboxSyncStatuses } from "./netboxSync.js";
|
import { isNetBoxFresh, netboxSyncStatuses } from "./netboxSync.js";
|
||||||
import { isN8nFresh, n8nSyncStatuses, workflowsOf as n8nWorkflowsOf } from "./n8nSync.js";
|
import { isN8nFresh, n8nSyncStatuses, workflowsOf as n8nWorkflowsOf } from "./n8nSync.js";
|
||||||
import { componentsOf as paperlessComponentsOf, failedOf as paperlessFailedOf, isPaperlessFresh, paperlessSyncStatuses } from "./paperlessSync.js";
|
import { componentsOf as paperlessComponentsOf, failedOf as paperlessFailedOf, isPaperlessFresh, paperlessSyncStatuses } from "./paperlessSync.js";
|
||||||
@@ -83,6 +84,22 @@ async function addNextcloudMetrics(out: Exposition, now: Date) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Each Docker host's vulnerability counts from Dockhand's scans, for /metrics. Only while the reading is recent, and only for hosts whose summary could be read. */
|
||||||
|
async function addDockerVulnMetrics(out: Exposition, now: Date) {
|
||||||
|
const statuses = await dockerVulnSyncStatuses();
|
||||||
|
const names = new Map((await db.select({ id: integrations.id, name: integrations.name, enabled: integrations.enabled }).from(integrations).where(eq(integrations.type, "dockhand"))).map((r) => [r.id, r]));
|
||||||
|
for (const s of await db.select().from(dockhandVulnerabilities)) {
|
||||||
|
const row = names.get(s.integrationId);
|
||||||
|
if (!row || !row.enabled || s.critical === null || !isDockerVulnFresh(statuses[s.integrationId], now.getTime())) continue;
|
||||||
|
const labels = { integration: row.name, host: s.environmentName };
|
||||||
|
out.add("homelab_docker_vulnerabilities", "gauge", "Vulnerabilities in the scanned images of a Docker host, by severity, from Dockhand's stored scans.", s.critical, { ...labels, severity: "critical" });
|
||||||
|
if (s.high !== null) out.add("homelab_docker_vulnerabilities", "gauge", "Vulnerabilities in the scanned images of a Docker host, by severity, from Dockhand's stored scans.", s.high, { ...labels, severity: "high" });
|
||||||
|
if (s.medium !== null) out.add("homelab_docker_vulnerabilities", "gauge", "Vulnerabilities in the scanned images of a Docker host, by severity, from Dockhand's stored scans.", s.medium, { ...labels, severity: "medium" });
|
||||||
|
if (s.low !== null) out.add("homelab_docker_vulnerabilities", "gauge", "Vulnerabilities in the scanned images of a Docker host, by severity, from Dockhand's stored scans.", s.low, { ...labels, severity: "low" });
|
||||||
|
if (s.imagesScanned !== null) out.add("homelab_docker_images_scanned", "gauge", "How many of a Docker host's images have a vulnerability scan in Dockhand.", s.imagesScanned, labels);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** Each NetBox, for /metrics: the workers running and what it holds. Only while the reading is recent. */
|
/** Each NetBox, for /metrics: the workers running and what it holds. Only while the reading is recent. */
|
||||||
async function addNetBoxMetrics(out: Exposition, now: Date) {
|
async function addNetBoxMetrics(out: Exposition, now: Date) {
|
||||||
const statuses = await netboxSyncStatuses();
|
const statuses = await netboxSyncStatuses();
|
||||||
@@ -388,6 +405,7 @@ export async function buildMetrics(now: Date = new Date()): Promise<string> {
|
|||||||
await addN8nMetrics(out, now);
|
await addN8nMetrics(out, now);
|
||||||
await addNetBoxMetrics(out, now);
|
await addNetBoxMetrics(out, now);
|
||||||
await addNextcloudMetrics(out, now);
|
await addNextcloudMetrics(out, now);
|
||||||
|
await addDockerVulnMetrics(out, now);
|
||||||
await addHeartbeatMetrics(out, now);
|
await addHeartbeatMetrics(out, now);
|
||||||
return out.toString();
|
return out.toString();
|
||||||
}
|
}
|
||||||
@@ -142,6 +142,8 @@ export interface HealthCheckSettings {
|
|||||||
baselineAlerts: boolean;
|
baselineAlerts: boolean;
|
||||||
/** A container that keeps restarting, is unhealthy, stopped unexpectedly or never started, and a Docker host Dockhand cannot reach, is an alert (services/containerHealth.ts). */
|
/** A container that keeps restarting, is unhealthy, stopped unexpectedly or never started, and a Docker host Dockhand cannot reach, is an alert (services/containerHealth.ts). */
|
||||||
containerAlerts: boolean;
|
containerAlerts: boolean;
|
||||||
|
/** Whether critical vulnerabilities found by Dockhand's scans are alerted (services/dockerVulnSync.ts). */
|
||||||
|
vulnerabilityAlerts: boolean;
|
||||||
/** Containers to leave alone: a name, or "host/name"; * stands for anything. */
|
/** Containers to leave alone: a name, or "host/name"; * stands for anything. */
|
||||||
containerIgnore: string[];
|
containerIgnore: string[];
|
||||||
}
|
}
|
||||||
@@ -327,7 +329,7 @@ const DEFAULTS: AppSettings = {
|
|||||||
selfBackup: { enabled: true, time: "03:30", keep: 14 },
|
selfBackup: { enabled: true, time: "03:30", keep: 14 },
|
||||||
profilePictures: { gravatar: false, identityProvider: false },
|
profilePictures: { gravatar: false, identityProvider: false },
|
||||||
signIn: { roleFromGroups: false, groupsClaim: "groups", adminGroups: [], operatorGroups: [], otherwise: "viewer" },
|
signIn: { roleFromGroups: false, groupsClaim: "groups", adminGroups: [], operatorGroups: [], otherwise: "viewer" },
|
||||||
healthChecks: { serverOfflineMinutes: 60, diskUsagePercent: 90, domainWarnDays: 30, diskForecastDays: 14, backupStaleHours: 48, restoreTestDays: 0, newPortAlerts: true, flapRaises: 3, deviceOfflineMinutes: 10, osSupportWarnDays: 90, baselineAlerts: true, containerAlerts: true, containerIgnore: [] },
|
healthChecks: { serverOfflineMinutes: 60, diskUsagePercent: 90, domainWarnDays: 30, diskForecastDays: 14, backupStaleHours: 48, restoreTestDays: 0, newPortAlerts: true, flapRaises: 3, deviceOfflineMinutes: 10, osSupportWarnDays: 90, baselineAlerts: true, containerAlerts: true, vulnerabilityAlerts: true, containerIgnore: [] },
|
||||||
// Docker's default bridge (172.17.0.0/16) and the pool it hands custom networks from (172.18–31) live here, and every
|
// Docker's default bridge (172.17.0.0/16) and the pool it hands custom networks from (172.18–31) live here, and every
|
||||||
// Docker host repeats them. Shown on the Consistency page, where it can be removed if 172.16/12 is used as a real LAN.
|
// Docker host repeats them. Shown on the Consistency page, where it can be removed if 172.16/12 is used as a real LAN.
|
||||||
consistency: { excludedRanges: ["172.16.0.0/12"] },
|
consistency: { excludedRanges: ["172.16.0.0/12"] },
|
||||||
|
|||||||
@@ -0,0 +1,254 @@
|
|||||||
|
/**
|
||||||
|
* Dockhand's vulnerability scans: reading what it stored (never starting a scan), what is kept, what is raised (critical only, and only while the setting is on), the health pass, the
|
||||||
|
* alerts, the page's route and /metrics.
|
||||||
|
*
|
||||||
|
* Run with: npm test --workspace server
|
||||||
|
*/
|
||||||
|
import { test, describe, before, after } from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { createServer as createHttpServer, type Server } from "node:http";
|
||||||
|
import { mkdtempSync, rmSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import type { AddressInfo } from "node:net";
|
||||||
|
|
||||||
|
const dir = mkdtempSync(join(tmpdir(), "hm-dockervulns-"));
|
||||||
|
process.env.DATABASE_PATH = join(dir, "test.sqlite");
|
||||||
|
process.env.SESSION_DIR = join(dir, "sessions");
|
||||||
|
process.env.CREDENTIALS_ENCRYPTION_KEY = "ab".repeat(32);
|
||||||
|
process.env.HEALTH_STARTUP_GRACE_MS = "0";
|
||||||
|
|
||||||
|
const express = (await import("express")).default;
|
||||||
|
const { db } = await import("../src/db/client.js");
|
||||||
|
const { users, integrations, dockhandVulnerabilities } = await import("../src/db/schema.js");
|
||||||
|
const { runMigrations } = await import("../src/db/migrate.js");
|
||||||
|
const adapter = await import("../src/integrations/dockhand/adapter.js");
|
||||||
|
const dv = await import("../src/services/dockerVulnSync.js");
|
||||||
|
const { evaluateHealth, collectSnapshot } = await import("../src/services/healthMonitor.js");
|
||||||
|
const { subjectOfConditionKey } = await import("../src/services/maintenance.js");
|
||||||
|
const { getAlerts, invalidateAlertsCache } = await import("../src/services/alerts.js");
|
||||||
|
const { integrationsRouter } = await import("../src/routes/integrations.js");
|
||||||
|
const { buildMetrics } = await import("../src/services/metrics.js");
|
||||||
|
const { getSettings, updateSettings, setInternalFlag } = await import("../src/services/settingsStore.js");
|
||||||
|
const { eq } = await import("drizzle-orm");
|
||||||
|
|
||||||
|
await runMigrations();
|
||||||
|
|
||||||
|
const MIN = 60_000;
|
||||||
|
|
||||||
|
const COUNT = (critical: number, scanned = 5, total = 8) => ({ total: critical + 12, summary: { total: critical + 12, critical, high: 7, medium: 4, low: 1, imagesScanned: scanned, totalImages: total }, options: { images: [], containers: [], stacks: [] } });
|
||||||
|
const finding = (cve: string, imageName: string, fixedVersion: string | null, containers: { id: string; name: string }[] = []) => ({ key: `${cve}:${imageName}`, cve, package: "openssl", severity: "critical", installedVersion: "3.0.1", fixedVersion, imageId: "sha256:x", imageName, containers });
|
||||||
|
|
||||||
|
describe("what Dockhand's API says", () => {
|
||||||
|
test("the summary is under `summary`; no critical count at all is not zero", () => {
|
||||||
|
assert.deepEqual(adapter.parseVulnSummary(COUNT(3)), { critical: 3, high: 7, medium: 4, low: 1, imagesScanned: 5, totalImages: 8 });
|
||||||
|
assert.equal(adapter.parseVulnSummary({ total: 0 }), null);
|
||||||
|
assert.equal(adapter.parseVulnSummary({ summary: { high: 2 } }), null);
|
||||||
|
assert.equal(adapter.parseVulnSummary("html"), null);
|
||||||
|
assert.equal(adapter.parseVulnSummary({ summary: { critical: 0 } })?.critical, 0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("findings: the CVE, where it is, whether there is a fix, and the containers that use the image", () => {
|
||||||
|
const list = adapter.parseVulnFindings({ findings: [finding("CVE-1", "nginx:1.25", "3.0.2", [{ id: "a", name: "web" }, { id: "b", name: "web2" }]), finding("CVE-2", "redis:7", null), { cve: "CVE-3" }, "junk"] });
|
||||||
|
assert.deepEqual(list.map((f) => [f.cve, f.imageName, f.fixedVersion, f.containers]), [["CVE-1", "nginx:1.25", "3.0.2", ["web", "web2"]], ["CVE-2", "redis:7", null, []]]);
|
||||||
|
assert.deepEqual(adapter.parseVulnFindings({ total: 0 }), []);
|
||||||
|
assert.deepEqual(adapter.parseVulnFindings(null), []);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("the fake Dockhand", () => {
|
||||||
|
let dockhand: Server;
|
||||||
|
let url = "";
|
||||||
|
const seen: string[] = [];
|
||||||
|
let countStatus: Record<string, number> = {};
|
||||||
|
let jackieCritical = 3;
|
||||||
|
|
||||||
|
before(async () => {
|
||||||
|
dockhand = createHttpServer((req, res) => {
|
||||||
|
const u = new URL(req.url ?? "", "http://x");
|
||||||
|
seen.push(`${req.method} ${u.pathname}${u.search}`);
|
||||||
|
res.setHeader("content-type", "application/json");
|
||||||
|
const env = u.searchParams.get("env") ?? "";
|
||||||
|
if (u.pathname === "/api/environments") return void res.end(JSON.stringify([{ id: 1, name: "Jackie" }, { id: 2, name: "Anna" }, { id: 3, name: "Old" }]));
|
||||||
|
if (u.pathname === "/api/vulnerabilities/count") {
|
||||||
|
if (countStatus[env]) {
|
||||||
|
res.statusCode = countStatus[env];
|
||||||
|
return void res.end("{}");
|
||||||
|
}
|
||||||
|
return void res.end(JSON.stringify(env === "1" ? COUNT(jackieCritical) : COUNT(0, 2, 2)));
|
||||||
|
}
|
||||||
|
if (u.pathname === "/api/vulnerabilities" && env === "1") {
|
||||||
|
const all = [finding("CVE-1", "nginx:1.25", "3.0.2", [{ id: "a", name: "web" }]), finding("CVE-2", "nginx:1.25", null, [{ id: "a", name: "web" }]), finding("CVE-3", "redis:7", "7.0.1")].slice(0, jackieCritical);
|
||||||
|
return void res.end(JSON.stringify({ findings: all, total: all.length, summary: { critical: all.length } }));
|
||||||
|
}
|
||||||
|
res.statusCode = 404;
|
||||||
|
res.end("{}");
|
||||||
|
});
|
||||||
|
await new Promise<void>((r) => dockhand.listen(0, "127.0.0.1", r));
|
||||||
|
url = `http://127.0.0.1:${(dockhand.address() as AddressInfo).port}`;
|
||||||
|
});
|
||||||
|
after(() => dockhand.close());
|
||||||
|
|
||||||
|
test("each host's count is read, the critical ones are listed only where there are some, a host that cannot say is unknown, and nothing is scanned", async () => {
|
||||||
|
seen.length = 0;
|
||||||
|
countStatus = { "3": 404 };
|
||||||
|
const read = await adapter.createDockhandAdapter({ url, token: "t" } as never).readVulnerabilities();
|
||||||
|
assert.deepEqual(read.environments.map((e) => [e.environmentName, e.summary?.critical ?? null, e.critical.length]), [["Jackie", 3, 3], ["Anna", 0, 0], ["Old", null, 0]]);
|
||||||
|
assert.deepEqual(read.environments[0].critical.map((f) => [f.cve, f.containers]), [["CVE-1", ["web"]], ["CVE-2", ["web"]], ["CVE-3", []]]);
|
||||||
|
assert.ok(seen.every((s) => s.startsWith("GET ")), "only reads");
|
||||||
|
assert.ok(!seen.some((s) => s.includes("scan")), "no scan endpoint");
|
||||||
|
assert.equal(seen.filter((s) => s.startsWith("GET /api/vulnerabilities?")).length, 1, "the list is asked for only where the count is above zero");
|
||||||
|
assert.match(seen.find((s) => s.startsWith("GET /api/vulnerabilities?"))!, /severity=critical/);
|
||||||
|
countStatus = {};
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("what is stored and what is wrong", () => {
|
||||||
|
let id = 0;
|
||||||
|
before(async () => {
|
||||||
|
id = (await db.insert(integrations).values({ type: "dockhand", name: "Dockhand", baseUrl: url, config: JSON.stringify({ url, token: "t" }) }).returning({ id: integrations.id }))[0].id;
|
||||||
|
});
|
||||||
|
const rows = () => db.select().from(dockhandVulnerabilities).where(eq(dockhandVulnerabilities.integrationId, id));
|
||||||
|
|
||||||
|
test("a sync stores each host, replacing the last reading; a host Dockhand no longer lists is dropped", async () => {
|
||||||
|
const status = await dv.syncDockerVulns(id);
|
||||||
|
assert.equal(status.ok, true);
|
||||||
|
const stored = (await rows()).sort((a, b) => a.environmentId - b.environmentId);
|
||||||
|
assert.deepEqual(stored.map((r) => [r.environmentName, r.critical, r.high, r.imagesScanned, r.totalImages]), [["Jackie", 3, 7, 5, 8], ["Anna", 0, 7, 2, 2], ["Old", 0, 7, 2, 2]]);
|
||||||
|
assert.equal(dv.findingsOf(stored[0]).length, 3);
|
||||||
|
await dv.storeRead(id, [{ environmentId: 1, environmentName: "Jackie", summary: { critical: 1, high: 0, medium: 0, low: 0, imagesScanned: 1, totalImages: 1 }, critical: [] }]);
|
||||||
|
assert.deepEqual((await rows()).map((r) => [r.environmentId, r.critical]), [[1, 1]]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a read that failed is said, and what was stored is left alone", async () => {
|
||||||
|
await dv.syncDockerVulns(id);
|
||||||
|
const before = await rows();
|
||||||
|
const failed = await dv.syncDockerVulns(id, new Date(), async () => {
|
||||||
|
throw new Error("connection refused");
|
||||||
|
});
|
||||||
|
assert.deepEqual([failed.ok, failed.error], [false, "connection refused"]);
|
||||||
|
assert.deepEqual((await rows()).map((r) => r.updatedAt), before.map((r) => r.updatedAt));
|
||||||
|
assert.equal(dv.isDockerVulnFresh((await dv.dockerVulnSyncStatuses())[id], Date.now()), false);
|
||||||
|
await dv.syncDockerVulns(id);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a host whose count could not be read is stored as unknown, not as none", async () => {
|
||||||
|
countStatus = { "2": 500 };
|
||||||
|
await dv.syncDockerVulns(id);
|
||||||
|
const anna = (await rows()).find((r) => r.environmentName === "Anna")!;
|
||||||
|
assert.deepEqual([anna.critical, anna.high, anna.imagesScanned], [null, null, null]);
|
||||||
|
assert.deepEqual(dv.dockerVulnProblems(id, [anna]), []);
|
||||||
|
countStatus = {};
|
||||||
|
await dv.syncDockerVulns(id);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("critical findings are a warning for the host, naming the images and how many have a fix; high and lower are never raised; zero is nothing", async () => {
|
||||||
|
const all = await rows();
|
||||||
|
const jackie = all.find((r) => r.environmentName === "Jackie")!;
|
||||||
|
const anna = all.find((r) => r.environmentName === "Anna")!;
|
||||||
|
const p = dv.dockerVulnProblems(id, [jackie, anna]);
|
||||||
|
assert.deepEqual(p.map((x) => [x.key, x.severity]), [[`docker-vuln:${id}:1`, "warning"]]);
|
||||||
|
assert.equal(p[0].message, "Jackie: 3 critical vulnerabilities in 2 images: nginx:1.25, redis:7 (2 of the first 3 have a fix)");
|
||||||
|
assert.deepEqual(dv.dockerVulnProblems(id, [{ ...jackie, critical: 1, findings: JSON.stringify([]) }]).map((x) => x.message), ["Jackie: 1 critical vulnerability"]);
|
||||||
|
assert.deepEqual(dv.dockerVulnProblems(id, [{ ...jackie, critical: 0, high: 50 }]), []);
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("the health pass, the Alerts page, the Docker page and /metrics", () => {
|
||||||
|
let app: Server;
|
||||||
|
let base = "";
|
||||||
|
const fresh = () => setInternalFlag("dockerVulnSync", JSON.stringify({ [id]: { at: new Date().toISOString(), ok: true, error: null } }));
|
||||||
|
const stale = () => setInternalFlag("dockerVulnSync", JSON.stringify({ [id]: { at: new Date(Date.now() - 60 * MIN).toISOString(), ok: true, error: null } }));
|
||||||
|
const vulnIds = async () => {
|
||||||
|
invalidateAlertsCache();
|
||||||
|
return (await getAlerts(true)).alerts.map((x) => x.id).filter((i) => i.includes("docker-vuln:"));
|
||||||
|
};
|
||||||
|
|
||||||
|
before(async () => {
|
||||||
|
await db.insert(users).values([{ oidcSub: "viewer", name: "Vic", role: "viewer" }]);
|
||||||
|
await dv.syncDockerVulns(id);
|
||||||
|
await fresh();
|
||||||
|
const a = express();
|
||||||
|
a.use(express.json());
|
||||||
|
a.use((req, _res, next) => {
|
||||||
|
(req as unknown as { session: unknown }).session = { user: { sub: "viewer" } };
|
||||||
|
next();
|
||||||
|
});
|
||||||
|
a.use("/api/integrations", integrationsRouter);
|
||||||
|
app = await new Promise<Server>((resolve) => {
|
||||||
|
const s = a.listen(0, "127.0.0.1", () => resolve(s));
|
||||||
|
});
|
||||||
|
base = `http://127.0.0.1:${(app.address() as AddressInfo).port}/api/integrations/${id}/dockhand`;
|
||||||
|
});
|
||||||
|
after(() => {
|
||||||
|
app.close();
|
||||||
|
try {
|
||||||
|
rmSync(dir, { recursive: true, force: true });
|
||||||
|
} catch {
|
||||||
|
// Windows keeps the database file locked until the process exits.
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the health pass judges it from what is stored, and holds it when the last read is old", async () => {
|
||||||
|
let { snapshot, held } = await collectSnapshot();
|
||||||
|
assert.deepEqual(snapshot.dockerVulns?.map((v) => v.integrationId), [id]);
|
||||||
|
const list = evaluateHealth(snapshot, { serverOfflineMinutes: 60, diskUsagePercent: 90 }, Date.now(), { skipServers: true });
|
||||||
|
const c = list.find((x) => x.key === `docker-vuln:${id}:1`)!;
|
||||||
|
assert.deepEqual([c.severity, c.source], ["warning", `dockhand:${id}:vuln`]);
|
||||||
|
assert.equal(subjectOfConditionKey(c.key), `integration:${id}`, "maintenance on it silences the alert");
|
||||||
|
await stale();
|
||||||
|
({ snapshot, held } = await collectSnapshot());
|
||||||
|
assert.deepEqual([snapshot.dockerVulns ?? [], held.has(`dockhand:${id}:vuln`)], [[], true]);
|
||||||
|
await fresh();
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the Alerts page shows it under Services with a link to the Docker page", async () => {
|
||||||
|
invalidateAlertsCache();
|
||||||
|
const a = (await getAlerts(true)).alerts.find((x) => x.id === `services:docker-vuln:${id}:1`)!;
|
||||||
|
assert.ok(a);
|
||||||
|
assert.deepEqual([a.severity, a.category, a.source, a.link, a.target?.type, a.target?.id], ["warning", "services", "Docker", "/docker", "integration", id]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("it can be switched off: nothing is raised, nothing is held, and the alert is gone; on again, it is back", async () => {
|
||||||
|
const checks = (await getSettings()).healthChecks;
|
||||||
|
assert.equal(checks.vulnerabilityAlerts, true, "on by default");
|
||||||
|
await updateSettings({ healthChecks: { ...checks, vulnerabilityAlerts: false } });
|
||||||
|
const { snapshot, held } = await collectSnapshot();
|
||||||
|
assert.deepEqual([snapshot.dockerVulns, [...held].filter((h) => h.endsWith(":vuln"))], [undefined, []]);
|
||||||
|
assert.deepEqual(await vulnIds(), []);
|
||||||
|
await updateSettings({ healthChecks: { ...checks, vulnerabilityAlerts: true } });
|
||||||
|
assert.deepEqual(await vulnIds(), [`services:docker-vuln:${id}:1`]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("an old setting without the switch is read as on", async () => {
|
||||||
|
const { vulnerabilityAlerts: _dropped, ...older } = (await getSettings()).healthChecks;
|
||||||
|
await updateSettings({ healthChecks: older as never });
|
||||||
|
assert.equal((await getSettings()).healthChecks.vulnerabilityAlerts, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("anyone signed in sees each host's counts, the critical findings and how many images are scanned", async () => {
|
||||||
|
const body = (await (await fetch(`${base}/vulnerabilities`, { headers: { connection: "close" } })).json()) as { alerts: boolean; fresh: boolean; environments: { environmentName: string; critical: number; high: number; imagesScanned: number; totalImages: number; findings: { cve: string }[] }[] };
|
||||||
|
assert.deepEqual([body.alerts, body.fresh], [true, true]);
|
||||||
|
const jackie = body.environments.find((e) => e.environmentName === "Jackie")!;
|
||||||
|
assert.deepEqual([jackie.critical, jackie.high, jackie.imagesScanned, jackie.totalImages, jackie.findings.map((f) => f.cve)], [3, 7, 5, 8, ["CVE-1", "CVE-2", "CVE-3"]]);
|
||||||
|
assert.equal((await fetch(`${base.replace(`/${id}/`, "/99999/")}/vulnerabilities`, { headers: { connection: "close" } })).status, 404);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("/metrics has the counts by severity and the images scanned — only while the reading is recent", async () => {
|
||||||
|
let text = await buildMetrics(new Date());
|
||||||
|
assert.match(text, /homelab_docker_vulnerabilities\{[^}]*host="Jackie"[^}]*severity="critical"[^}]*\} 3/);
|
||||||
|
assert.match(text, /homelab_docker_vulnerabilities\{[^}]*host="Jackie"[^}]*severity="high"[^}]*\} 7/);
|
||||||
|
assert.match(text, /homelab_docker_images_scanned\{[^}]*host="Jackie"[^}]*\} 5/);
|
||||||
|
await stale();
|
||||||
|
text = await buildMetrics(new Date());
|
||||||
|
assert.doesNotMatch(text, /homelab_docker_vulnerabilities/);
|
||||||
|
await fresh();
|
||||||
|
});
|
||||||
|
|
||||||
|
test("taking the integration away takes what was read with it", async () => {
|
||||||
|
await dv.forgetDockerVulns(id);
|
||||||
|
assert.equal((await dv.dockerVulnSyncStatuses())[id], undefined);
|
||||||
|
await db.delete(integrations).where(eq(integrations.id, id));
|
||||||
|
assert.deepEqual(await rows(), []);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -539,6 +539,7 @@ export interface HealthCheckSettings {
|
|||||||
baselineAlerts: boolean;
|
baselineAlerts: boolean;
|
||||||
/** A container that keeps restarting, is unhealthy, stopped unexpectedly or never started, and a Docker host that cannot be reached, is an alert. */
|
/** A container that keeps restarting, is unhealthy, stopped unexpectedly or never started, and a Docker host that cannot be reached, is an alert. */
|
||||||
containerAlerts: boolean;
|
containerAlerts: boolean;
|
||||||
|
vulnerabilityAlerts: boolean;
|
||||||
/** Containers to leave alone: a name, or "host/name"; * stands for anything. */
|
/** Containers to leave alone: a name, or "host/name"; * stands for anything. */
|
||||||
containerIgnore: string[];
|
containerIgnore: string[];
|
||||||
}
|
}
|
||||||
@@ -2517,6 +2518,38 @@ export interface DockhandContainer {
|
|||||||
leftAlone?: boolean;
|
leftAlone?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export interface DockhandVulnFinding {
|
||||||
|
cve: string;
|
||||||
|
severity: string;
|
||||||
|
package: string | null;
|
||||||
|
installedVersion: string | null;
|
||||||
|
fixedVersion: string | null;
|
||||||
|
imageName: string;
|
||||||
|
containers: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface DockhandVulnerabilityEnvironment {
|
||||||
|
environmentId: number;
|
||||||
|
environmentName: string;
|
||||||
|
critical: number | null;
|
||||||
|
high: number | null;
|
||||||
|
medium: number | null;
|
||||||
|
low: number | null;
|
||||||
|
imagesScanned: number | null;
|
||||||
|
totalImages: number | null;
|
||||||
|
/** The critical findings (at most 100). */
|
||||||
|
findings: DockhandVulnFinding[];
|
||||||
|
updatedAt: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface DockhandVulnerabilitiesResponse {
|
||||||
|
/** Whether critical findings are alerted (Settings → Notifications). */
|
||||||
|
alerts: boolean;
|
||||||
|
status: { at: string; ok: boolean; error: string | null } | null;
|
||||||
|
fresh: boolean;
|
||||||
|
environments: DockhandVulnerabilityEnvironment[];
|
||||||
|
}
|
||||||
|
|
||||||
export interface DockhandContainersResponse {
|
export interface DockhandContainersResponse {
|
||||||
/** Docker hosts Dockhand could not read this time. */
|
/** Docker hosts Dockhand could not read this time. */
|
||||||
unreachable?: { environmentId: number; environmentName: string; error: string }[];
|
unreachable?: { environmentId: number; environmentName: string; error: string }[];
|
||||||
@@ -3448,6 +3481,7 @@ export const api = {
|
|||||||
dockhand: {
|
dockhand: {
|
||||||
containers: (integrationId: number) =>
|
containers: (integrationId: number) =>
|
||||||
request<DockhandContainersResponse>(`/api/integrations/${integrationId}/dockhand/containers`),
|
request<DockhandContainersResponse>(`/api/integrations/${integrationId}/dockhand/containers`),
|
||||||
|
vulnerabilities: (integrationId: number) => request<DockhandVulnerabilitiesResponse>(`/api/integrations/${integrationId}/dockhand/vulnerabilities`),
|
||||||
leaveAlone: (integrationId: number, environmentName: string, name: string, leaveAlone: boolean) =>
|
leaveAlone: (integrationId: number, environmentName: string, name: string, leaveAlone: boolean) =>
|
||||||
request<{ containerIgnore: string[] }>(`/api/integrations/${integrationId}/dockhand/leave-alone`, { method: "PUT", body: JSON.stringify({ environmentName, name, leaveAlone }) }),
|
request<{ containerIgnore: string[] }>(`/api/integrations/${integrationId}/dockhand/leave-alone`, { method: "PUT", body: JSON.stringify({ environmentName, name, leaveAlone }) }),
|
||||||
start: (integrationId: number, envId: number, containerId: string) =>
|
start: (integrationId: number, envId: number, containerId: string) =>
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ import { useTableSearch } from "../hooks/useTableSearch";
|
|||||||
import DetailDrawer, { ObjectDetails } from "../components/DetailDrawer";
|
import DetailDrawer, { ObjectDetails } from "../components/DetailDrawer";
|
||||||
import { useRowDetail } from "../hooks/useRowDetail";
|
import { useRowDetail } from "../hooks/useRowDetail";
|
||||||
import { EmptyRow } from "../components/EmptyState";
|
import { EmptyRow } from "../components/EmptyState";
|
||||||
|
import DockerVulnerabilities from "./DockerVulnerabilities";
|
||||||
|
|
||||||
function updateBadge(c: DockhandContainer) {
|
function updateBadge(c: DockhandContainer) {
|
||||||
if (c.updateAvailable === null) return <span className="text-secondary">—</span>;
|
if (c.updateAvailable === null) return <span className="text-secondary">—</span>;
|
||||||
@@ -354,6 +355,7 @@ export default function Docker({ user }: { user: CurrentUser }) {
|
|||||||
<Pagination page={page} pageCount={pageCount} totalCount={totalCount} onPageChange={setPage} />
|
<Pagination page={page} pageCount={pageCount} totalCount={totalCount} onPageChange={setPage} />
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
{selected?.enabled && <DockerVulnerabilities integrationId={selected.id} />}
|
||||||
</>
|
</>
|
||||||
)}
|
)}
|
||||||
{detail.selected && (
|
{detail.selected && (
|
||||||
|
|||||||
@@ -0,0 +1,131 @@
|
|||||||
|
import { useEffect, useState } from "react";
|
||||||
|
import { Link } from "react-router-dom";
|
||||||
|
import { api, type DockhandVulnerabilitiesResponse, type DockhandVulnerabilityEnvironment } from "../api/client";
|
||||||
|
import TimeAgo from "../components/TimeAgo";
|
||||||
|
import { readableError } from "../utils/errors";
|
||||||
|
|
||||||
|
const count = (v: number | null) => (v === null ? "—" : v.toLocaleString());
|
||||||
|
|
||||||
|
function Host({ env }: { env: DockhandVulnerabilityEnvironment }) {
|
||||||
|
const [open, setOpen] = useState(false);
|
||||||
|
const unscanned = env.totalImages !== null && env.imagesScanned !== null ? env.totalImages - env.imagesScanned : null;
|
||||||
|
return (
|
||||||
|
<div className="mb-3">
|
||||||
|
<div className="d-flex flex-wrap align-items-center gap-2">
|
||||||
|
<strong>{env.environmentName}</strong>
|
||||||
|
{env.critical === null ? (
|
||||||
|
<span className="badge bg-secondary-lt text-secondary">Not read</span>
|
||||||
|
) : env.critical > 0 ? (
|
||||||
|
<span className="badge bg-red-lt text-red">{env.critical} critical</span>
|
||||||
|
) : (
|
||||||
|
<span className="badge bg-green-lt text-green">No critical</span>
|
||||||
|
)}
|
||||||
|
<span className="text-secondary small">
|
||||||
|
{count(env.high)} high · {count(env.medium)} medium · {count(env.low)} low
|
||||||
|
</span>
|
||||||
|
<span className="text-secondary small ms-auto">
|
||||||
|
{env.imagesScanned === null ? "" : `${env.imagesScanned} of ${env.totalImages ?? "?"} images scanned`}
|
||||||
|
{unscanned !== null && unscanned > 0 ? ` — ${unscanned} not scanned yet` : ""}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
{env.findings.length > 0 && (
|
||||||
|
<>
|
||||||
|
<button className="btn btn-link btn-sm p-0 mt-1" onClick={() => setOpen((o) => !o)}>
|
||||||
|
{open ? "Hide the critical findings" : `Show the critical findings${env.critical !== null && env.critical > env.findings.length ? ` (first ${env.findings.length} of ${env.critical})` : ""}`}
|
||||||
|
</button>
|
||||||
|
{open && (
|
||||||
|
<div className="table-responsive mt-2">
|
||||||
|
<table className="table table-sm table-vcenter">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>CVE</th>
|
||||||
|
<th>Package</th>
|
||||||
|
<th>Installed</th>
|
||||||
|
<th>Fixed in</th>
|
||||||
|
<th>Image</th>
|
||||||
|
<th>Used by</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{env.findings.map((f, i) => (
|
||||||
|
<tr key={`${f.cve}:${f.package}:${f.imageName}:${i}`}>
|
||||||
|
<td className="text-nowrap">{f.cve}</td>
|
||||||
|
<td className="text-break">{f.package}</td>
|
||||||
|
<td className="text-secondary text-break">{f.installedVersion || "—"}</td>
|
||||||
|
<td className="text-break">{f.fixedVersion ? <span className="text-green">{f.fixedVersion}</span> : <span className="text-secondary">no fix yet</span>}</td>
|
||||||
|
<td className="text-secondary text-break">{f.imageName}</td>
|
||||||
|
<td className="text-secondary text-break">{f.containers.length > 0 ? f.containers.join(", ") : "—"}</td>
|
||||||
|
</tr>
|
||||||
|
))}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** What Dockhand's own vulnerability scans found, per Docker host. Read from what was stored; nothing is asked of Dockhand here, and nothing is scanned. */
|
||||||
|
export default function DockerVulnerabilities({ integrationId }: { integrationId: number }) {
|
||||||
|
const [data, setData] = useState<DockhandVulnerabilitiesResponse | null>(null);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
useEffect(() => {
|
||||||
|
setData(null);
|
||||||
|
const load = () =>
|
||||||
|
api.integrations.dockhand
|
||||||
|
.vulnerabilities(integrationId)
|
||||||
|
.then((r) => {
|
||||||
|
setData(r);
|
||||||
|
setError(null);
|
||||||
|
})
|
||||||
|
.catch((err) => setError(readableError(err)));
|
||||||
|
void load();
|
||||||
|
const t = setInterval(load, 120_000);
|
||||||
|
return () => clearInterval(t);
|
||||||
|
}, [integrationId]);
|
||||||
|
|
||||||
|
const critical = (data?.environments ?? []).reduce((sum, e) => sum + (e.critical ?? 0), 0);
|
||||||
|
return (
|
||||||
|
<div className="card mt-3">
|
||||||
|
<div className="card-header">
|
||||||
|
<h3 className="card-title">
|
||||||
|
Vulnerabilities
|
||||||
|
{data && data.environments.length > 0 && (critical > 0 ? <span className="text-red fw-normal ms-2">{critical} critical</span> : <span className="text-secondary fw-normal ms-2">none critical</span>)}
|
||||||
|
</h3>
|
||||||
|
{data?.status && (
|
||||||
|
<div className="card-actions text-secondary small">
|
||||||
|
read <TimeAgo value={data.status.at} />
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
<div className="card-body">
|
||||||
|
{error && <div className="text-yellow">{error}</div>}
|
||||||
|
{!data && !error && <div className="text-secondary">Reading…</div>}
|
||||||
|
{data && (
|
||||||
|
<>
|
||||||
|
<p className="text-secondary small">
|
||||||
|
From the scans Dockhand has already run on its images (it scans when it is told to — nothing is scanned from here). An image that was never scanned is not known to be clean.{" "}
|
||||||
|
{data.alerts ? (
|
||||||
|
<>
|
||||||
|
<strong>Critical</strong> findings are an alert — see <Link to="/alerts">Alerts</Link>; high, medium and low are only shown here.
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
The alert for critical findings is switched off (<Link to="/settings/notifications">Settings → Notifications</Link>).
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</p>
|
||||||
|
{data.status && !data.status.ok && <div className="text-yellow mb-2">The last attempt to read them failed: {data.status.error ?? "no answer"}.</div>}
|
||||||
|
{data.environments.length === 0 && !data.status && <div className="text-secondary">Not read yet — it is read a few minutes after Dockhand is added, then every quarter of an hour.</div>}
|
||||||
|
{data.environments.length === 0 && data.status?.ok && <div className="text-secondary">Dockhand lists no Docker hosts.</div>}
|
||||||
|
{data.environments.map((e) => (
|
||||||
|
<Host key={e.environmentId} env={e} />
|
||||||
|
))}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -84,7 +84,7 @@ const DIGEST_DAYS: { value: number; label: string }[] = [
|
|||||||
{ value: 6, label: "Saturday" },
|
{ value: 6, label: "Saturday" },
|
||||||
{ value: 0, label: "Sunday" },
|
{ value: 0, label: "Sunday" },
|
||||||
];
|
];
|
||||||
const DEFAULT_HEALTH_CHECKS: HealthCheckSettings = { serverOfflineMinutes: 60, diskUsagePercent: 90, domainWarnDays: 30, diskForecastDays: 14, backupStaleHours: 48, restoreTestDays: 0, newPortAlerts: true, flapRaises: 3, deviceOfflineMinutes: 10, osSupportWarnDays: 90, baselineAlerts: true, containerAlerts: true, containerIgnore: [] };
|
const DEFAULT_HEALTH_CHECKS: HealthCheckSettings = { serverOfflineMinutes: 60, diskUsagePercent: 90, domainWarnDays: 30, diskForecastDays: 14, backupStaleHours: 48, restoreTestDays: 0, newPortAlerts: true, flapRaises: 3, deviceOfflineMinutes: 10, osSupportWarnDays: 90, baselineAlerts: true, containerAlerts: true, vulnerabilityAlerts: true, containerIgnore: [] };
|
||||||
const DEFAULT_MONTHLY: MonthlyReportSettings = { enabled: false, quarterly: false, annual: false, day: 1, time: "08:00" };
|
const DEFAULT_MONTHLY: MonthlyReportSettings = { enabled: false, quarterly: false, annual: false, day: 1, time: "08:00" };
|
||||||
|
|
||||||
type TestResult = { ok: boolean; message: string } | null;
|
type TestResult = { ok: boolean; message: string } | null;
|
||||||
@@ -865,6 +865,16 @@ export default function NotificationSettings() {
|
|||||||
onChange={(e) => setHealthChecks((h) => ({ ...h, containerIgnore: e.target.value.split("\n").map((l) => l.trim()).filter(Boolean) }))}
|
onChange={(e) => setHealthChecks((h) => ({ ...h, containerIgnore: e.target.value.split("\n").map((l) => l.trim()).filter(Boolean) }))}
|
||||||
/>
|
/>
|
||||||
<div className="form-hint">The scanner containers Dockhand starts (dockhand-grype-*, dockhand-trivy-*) are always left alone. There is also a <em>Leave alone</em> button on the Docker page.</div>
|
<div className="form-hint">The scanner containers Dockhand starts (dockhand-grype-*, dockhand-trivy-*) are always left alone. There is also a <em>Leave alone</em> button on the Docker page.</div>
|
||||||
|
<label className="form-check mt-3 mb-0">
|
||||||
|
<input type="checkbox" className="form-check-input" checked={healthChecks.vulnerabilityAlerts} onChange={(e) => setHealthChecks((h) => ({ ...h, vulnerabilityAlerts: e.target.checked }))} />
|
||||||
|
<span className="form-check-label">
|
||||||
|
Warn about critical vulnerabilities in Docker images
|
||||||
|
<span className="form-hint d-block">
|
||||||
|
From the scans Dockhand has already run (nothing is scanned from here): a warning for each Docker host whose scanned images have one or more critical findings. High, medium and low findings are shown on the Docker
|
||||||
|
page but never alerted. Dockhand can send its own vulnerability notifications too; with both switched on you hear twice.
|
||||||
|
</span>
|
||||||
|
</span>
|
||||||
|
</label>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
Reference in new issue
Block a user