diff --git a/DATABASE.md b/DATABASE.md new file mode 100644 index 0000000..4785309 --- /dev/null +++ b/DATABASE.md @@ -0,0 +1,553 @@ +# Database schema + +Homelab Manager keeps its data in one SQLite file, accessed through +[drizzle-orm](https://orm.drizzle.team) and the libSQL client. The schema is +defined in one place — [`server/src/db/schema.ts`](server/src/db/schema.ts) — +and this document describes it. It was checked against a fresh database built +from the migrations, so the tables, columns, foreign keys and indexes below are +what the app actually creates. + +- [The basics](#the-basics) +- [How the tables relate](#how-the-tables-relate) +- [Tables](#tables) +- [What's stored inside the JSON columns](#whats-stored-inside-the-json-columns) +- [Settings keys](#settings-keys) +- [What happens on delete](#what-happens-on-delete) +- [The Proxmox link's foreign key](#the-proxmox-links-foreign-key) +- [Retention and backups](#retention-and-backups) +- [Changing the schema](#changing-the-schema) + +## The basics + +| | | +|---|---| +| **File** | `DATABASE_PATH`, default `../data/homelab-manager.sqlite` (relative to `server/`; `/app/data/...` in Docker). The folder is created if missing. | +| **Foreign keys** | Switched on for every connection (`PRAGMA foreign_keys = ON`), so the `ON DELETE` rules below are enforced. | +| **Migrations** | SQL files in [`server/drizzle/`](server/drizzle) (`0000` … `0014`), applied automatically when the server starts. Which ones have run is recorded in the table `__drizzle_migrations`. | +| **Tables** | 21 application tables, plus drizzle's own `__drizzle_migrations`. | + +**Not in the database:** + +- **Login sessions** are files in `SESSION_DIR` (default `../data/sessions`), not rows. +- **The encryption key** for integration credentials (`CREDENTIALS_ENCRYPTION_KEY`) and the session secret live in the environment, never in the file. +- **Agent tokens** are never stored: only a SHA-256 hash and a short prefix of each (`servers.api_token_hash`, `api_token_prefix`). The token itself is shown once, when the server is registered. + +### Conventions + +- **Primary keys** are `INTEGER PRIMARY KEY AUTOINCREMENT` named `id` — except `settings`, which uses its text `key`. +- **Booleans** are `INTEGER` 0/1 (shown as `bool` below). +- **Timestamps are text, in two formats**, so read them with care: + - Columns the database fills in itself (`created_at`, and most `updated_at`) use SQLite's `current_timestamp`: `2026-10-03 14:05:09`, **UTC, with no zone marker**. Treat it as UTC, not local time. + - Columns the app fills in (`last_seen_at`, `last_login_at`, `synced_at`, `last_checked_at`, …) use ISO 8601: `2026-10-03T14:05:09.123Z`. + - Dates without a time (`secrets.expiry_date`, `domains.expires_at`) are `YYYY-MM-DD`. +- **JSON columns** are `TEXT` holding JSON; see [what's inside](#whats-stored-inside-the-json-columns). +- **Enumerations** (roles, types) are plain text — SQLite doesn't enforce the allowed values; the app does. +- **Indexes:** besides primary keys, the only indexes are the unique ones listed per table. There are no secondary indexes; the data sets here (hundreds to a few thousand rows) don't need them. + +## How the tables relate + +```mermaid +erDiagram + users ||--o{ audit_log : "actor (set null)" + integration_credentials ||--o{ integrations : "credential (set null)" + integration_credentials ||--o{ dns_providers : "credential (set null)" + dns_providers ||--o{ dns_zones_cache : "cascade" + dns_providers ||--o{ dns_records_cache : "cascade" + servers ||--o{ scheduled_tasks : "cascade" + servers ||--o{ server_links : "cascade" + servers ||--o{ server_ports : "cascade" + servers ||--o{ port_forwards : "optional (set null)" + integrations ||--o{ servers : "proxmox link (app clears it)" +``` + +Eight tables stand alone, with no foreign keys: `settings`, `secrets`, +`ipam_entries`, `domains`, `diag_log`, `notification_queue`, +`consistency_ignores`, `tag_definitions`. `maintenance_windows` also has no +foreign key — see [soft references](#soft-references-not-foreign-keys). + +## Tables + +Grouped by what they're for. "Null" in the notes means the column allows NULL; +everything not marked **NOT NULL** may be empty. + +### People and activity + +#### `users` + +Everyone who has signed in through Authentik. The first one becomes admin. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `oidc_sub` | text NOT NULL, **unique** | The user's stable ID from Authentik (`sub` claim). This is what a session is matched against. | +| `email`, `name` | text | From the sign-in; may be empty. | +| `role` | text NOT NULL, default `viewer` | `admin` \| `operator` \| `viewer`. | +| `created_at` | text NOT NULL | SQLite UTC. | +| `last_login_at` | text | ISO. | + +#### `audit_log` + +Who changed what. Written by the app on every change (see +[ROLES.md](ROLES.md) for who can read it). + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `actor_user_id` | integer → `users.id`, **set null** on delete | Null for automatic actions, and for entries whose user was deleted. | +| `actor_label` | text | The user's name/email as it was at the time (or `system`), so an entry still reads correctly after the account is gone. | +| `category` | text NOT NULL | Free text. In use: `server`, `integration`, `dns`, `settings`, `ipam`, `secret`, `domain`, `tag`, `task`, `session`, `network`, `maintenance`, `consistency`, `user`, `privacy`, `diag_log`. | +| `action` | text NOT NULL | `create`, `update`, `delete`, `start`, `stop`, … — free text. | +| `target_type`, `target_id` | text | What it happened to. `target_id` is text so it can hold any kind of ID; there's no foreign key. | +| `detail` | text | JSON, free-form context (what changed, names, counts). Secrets are never put here. | +| `created_at` | text NOT NULL | SQLite UTC. | + +#### `diag_log` + +One row per outbound call to an integration or DNS provider — the source of +the Diagnostic Log page and of the "integration down" alert. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `source` | text NOT NULL | The integration/provider type, e.g. `proxmox`, `cloudflare`. | +| `operation` | text NOT NULL | The adapter method, e.g. `listZones`. | +| `ok` | bool NOT NULL | | +| `latency_ms` | integer NOT NULL | | +| `error` | text | Message when `ok` is false. | +| `created_at` | text NOT NULL | SQLite UTC. | + +#### `notification_queue` + +Notifications held back during quiet hours, delivered as one digest and then +cleared. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `title`, `message` | text NOT NULL | | +| `created_at` | text NOT NULL | SQLite UTC. | + +#### `maintenance_windows` + +While a window is open, alerts about its target are silenced. An end time is +required, so a forgotten window can't silence real problems forever. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `target_type` | text NOT NULL | `server` \| `integration` \| `dns_provider`. | +| `target_id` | integer NOT NULL | The ID in the table `target_type` names. **Not a foreign key**; a window whose target was deleted is simply ignored. | +| `reason` | text | | +| `started_at`, `ends_at` | text NOT NULL | ISO. | +| `created_by` | text | Name of the person who opened it. | + +### Servers + +#### `servers` + +A machine that reports in through the agent (or is registered by hand), plus +what it last reported. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `name` | text NOT NULL | Not unique. | +| `hostname` | text | | +| `os_type` | text NOT NULL, default `linux` | | +| `description` | text | | +| `api_token_hash` | text NOT NULL | SHA-256 of the agent's token. | +| `api_token_prefix` | text NOT NULL | First characters of the token, to tell tokens apart in the UI. | +| `created_at` | text NOT NULL | SQLite UTC. | +| `last_seen_at` | text | ISO; when the agent last reported. Drives "server offline". | +| `ip_addresses` | text | JSON `string[]`. Agent-reported. | +| `cpu_model` | text | Agent-reported. | +| `cpu_cores` | integer | | +| `cpu_load_percent` | real | Load average ÷ cores × 100 — an approximation, not instantaneous usage. | +| `mem_total_bytes`, `mem_used_bytes` | integer | | +| `disks` | text | JSON, see below. Agent-reported. | +| `listening_ports` | text | JSON, see below. What the agent sees bound on the host. | +| `last_port_scan` | text | JSON summary of the latest network scan *from this app*. | +| `tags` | text | JSON `string[]` of normalised tag names. | +| `proxmox_integration_id` | integer → `integrations.id` | The database has no `ON DELETE` rule here; the app clears the link itself — see [below](#the-proxmox-links-foreign-key). | +| `proxmox_node` | text | | +| `proxmox_guest_type` | text | `qemu` \| `lxc`. | +| `proxmox_vmid` | integer | The four `proxmox_*` columns are set together or cleared together (enforced by the API), by an admin — never by the agent. | +| `hide_proxmox_link` | bool NOT NULL, default 0 | Hides the "Proxmox link" card for servers that aren't Proxmox guests. Ignored while the server is actually linked. | + +#### `scheduled_tasks` + +Cron jobs, systemd timers and Windows tasks the agent found on a server, plus +tasks added by hand. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `server_id` | integer NOT NULL → `servers.id`, **cascade** | | +| `schedule_type` | text NOT NULL | `cron` \| `systemd_timer` \| `windows_task` from agents; manual tasks may use others (`docker`, `backup`, `update`, `n8n_workflow`, `manual`). | +| `origin` | text NOT NULL, default `agent` | `agent` \| `manual`. Manual rows are never touched by agent sync. | +| `name` | text NOT NULL | | +| `command`, `schedule_expression`, `source` | text | | +| `enabled` | bool NOT NULL, default 1 | | +| `next_run_at` | text | | +| `raw_metadata` | text | JSON as the agent reported it. | +| `is_stale` | bool NOT NULL, default 0 | Set when the agent stops reporting the task. | +| `first_seen_at`, `last_seen_at` | text NOT NULL | SQLite UTC at first insert; later updates are written by the app. | + +#### `server_links` + +Admin-page bookmarks for a server (Dockge, Webmin, Cockpit, …). Shown on the +server's page and summarised under Operations → Admin Links. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `server_id` | integer NOT NULL → `servers.id`, **cascade** | | +| `label`, `url` | text NOT NULL | | +| `created_at` | text NOT NULL | SQLite UTC. | + +#### `server_ports` + +A port on one server that's been seen open by a scan, or that someone wrote a +note about. Rows exist only while they carry information. What the *agent* +sees is stored on the server row instead (`servers.listening_ports`). + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `server_id` | integer NOT NULL → `servers.id`, **cascade** | | +| `port` | integer NOT NULL | | +| `protocol` | text NOT NULL, default `tcp` | `tcp` \| `udp`. | +| `label`, `comment` | text | | +| `open` | bool NOT NULL, default 0 | True when the last scan connected to it. | +| `last_seen_open_at` | text | | +| `updated_at` | text NOT NULL | | + +Unique index **`server_ports_unique`** on (`server_id`, `port`, `protocol`). + +#### `port_forwards` + +Manually recorded port openings on something this app doesn't monitor — a +router's port forward, an edge firewall rule, a cloud security group. It +records them; it can't check or change them. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `label` | text NOT NULL | | +| `external_port` | integer NOT NULL | | +| `protocol` | text NOT NULL, default `tcp` | `tcp` \| `udp`. | +| `server_id` | integer → `servers.id`, **set null** | Optional: the tracked server it points at. | +| `destination` | text | Anything else — a bare IP, an untracked device — or detail alongside `server_id`. | +| `internal_port` | integer | When NAT changes the port. | +| `source` | text | Free text: where the rule lives ("Home router", "OPNsense WAN rule"). | +| `comment` | text | | +| `created_at`, `updated_at` | text NOT NULL | | + +### Integrations and credentials + +#### `integrations` + +A connected system: Proxmox, Synology, Semaphore, Tailscale, Gitea, Dockhand, +Uptime Kuma, phpIPAM, Proxmox Backup Server, osTicket. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `type` | text NOT NULL | `proxmox` \| `synology` \| `semaphore` \| `tailscale` \| `gitea` \| `dockhand` \| `uptimekuma` \| `phpipam` \| `pbs` \| `osticket`. | +| `name` | text NOT NULL | | +| `base_url` | text NOT NULL | For osTicket (a direct database connection) this holds the database host. | +| `credential_id` | integer → `integration_credentials.id`, **set null** | | +| `config` | text | JSON of the *non-secret* settings, see below. | +| `enabled` | bool NOT NULL, default 1 | | +| `created_at` | text NOT NULL | | + +#### `integration_credentials` + +The secret half of an integration or DNS provider, encrypted at rest. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `name` | text NOT NULL | `":"`, for recognising a row when looking at the file. | +| `encrypted_secret` | text NOT NULL | `iv:authTag:ciphertext`, each in hex — AES-256-GCM with `CREDENTIALS_ENCRYPTION_KEY`. The plaintext is a JSON object of the secret fields (an API token, a password). Without the same key it can't be read. | +| `created_at` | text NOT NULL | | + +The notification channels' credentials aren't in this table; they're encrypted in place in `settings` — see [Retention and backups](#retention-and-backups). + +### DNS + +#### `dns_providers` + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `provider_type` | text NOT NULL | `cloudflare` \| `loopia` \| `pihole` \| `azure` \| `cpanel` \| `technitium`. | +| `name` | text NOT NULL | | +| `credential_id` | integer → `integration_credentials.id`, **set null** | | +| `config` | text | JSON, non-secret provider config (base URL, zone list, …). | +| `enabled` | bool NOT NULL, default 1 | | +| `created_at` | text NOT NULL | | + +#### `dns_zones_cache` and `dns_records_cache` + +Local copies of what the providers returned at the last sync, so pages load +without calling every provider. The providers remain the source of truth. + +`dns_zones_cache` + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `provider_id` | integer NOT NULL → `dns_providers.id`, **cascade** | | +| `zone_id` | text NOT NULL | The provider's own identifier for the zone. | +| `zone_name` | text NOT NULL | | +| `synced_at` | text | ISO. | + +Unique index **`dns_zones_cache_provider_zone_idx`** on (`provider_id`, `zone_id`). + +`dns_records_cache` + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `provider_id` | integer NOT NULL → `dns_providers.id`, **cascade** | | +| `zone_id` | text NOT NULL | The provider's zone identifier — matches `dns_zones_cache.zone_id` for the same provider, but is **not a foreign key**. | +| `record_id` | text NOT NULL | The provider's own record identifier. | +| `type` | text NOT NULL | `A`, `AAAA`, `CNAME`, `TXT`, `MX`, … | +| `name`, `content` | text NOT NULL | | +| `ttl`, `priority` | integer | | +| `proxied` | bool | Cloudflare only. | + +### Address and name tracking + +#### `ipam_entries` + +IP addresses with a label and notes, entered by hand or synced. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `ip_address` | text NOT NULL, **unique** | | +| `label`, `vendor`, `location`, `notes` | text | | +| `source` | text | Null = typed in by hand; otherwise the sync that created it: `tailscale`, `proxmox` or `phpipam`. A sync only updates rows it created itself and never overwrites a manual one. | +| `created_at`, `updated_at` | text NOT NULL | | + +#### `domains` + +Registered domains whose expiry is tracked. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `name` | text NOT NULL, **unique** | The registrable domain, lowercase ASCII. | +| `origin` | text NOT NULL, default `manual` | `manual` (typed in) or `zone` (created from a synced DNS zone, removed again when the zone goes away). | +| `expires_at` | text | `YYYY-MM-DD`, as the registry reports it. Null for registries that don't publish one. | +| `registrar` | text | | +| `lookup_source` | text | `rdap` \| `whois`. | +| `last_checked_at` | text | Last attempt. | +| `last_checked_ok_at` | text | Last *successful* attempt. | +| `last_check_error` | text | | +| `created_at` | text NOT NULL | | + +#### `secrets` + +The expiry tracker for API tokens, certificates, passwords and the like. It +tracks *when* something expires; it does not store the secret itself. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `name` | text NOT NULL | | +| `type` | text NOT NULL, default `generic` | `api_token` \| `ssl_certificate` \| `password` \| `generic`. | +| `description`, `notes` | text | | +| `expiry_date` | text NOT NULL | `YYYY-MM-DD`. For a certificate with a host to check, overwritten from the live certificate. | +| `warn_days` | integer NOT NULL, default 30 | How long before expiry to start reminding. | +| `check_host`, `check_port` | text / integer | Certificates only: read the expiry from the live certificate at this host:port. | +| `last_checked_at`, `last_check_error` | text | Result of the last live check. | +| `created_at`, `updated_at` | text NOT NULL | | + +### Housekeeping + +#### `settings` + +Key/value store for app settings. One row per settings section (the value is +JSON) plus a few internal bookkeeping rows. Details under +[Settings keys](#settings-keys). + +| Column | Type | Notes | +|---|---|---| +| `key` | text PK | | +| `value` | text NOT NULL | JSON (or a plain date/time for internal flags). | +| `updated_at` | text NOT NULL | | + +#### `tag_definitions` + +Tags themselves live on the servers that carry them (`servers.tags`). A row +here adds what a server can't: a tag that exists before anything uses it, and a +chosen colour. A tag with no row is simply one in use with an automatic colour. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `name` | text NOT NULL, **unique** | Normalised. | +| `color` | text | `#rrggbb`, or null for automatic. | +| `created_at` | text NOT NULL | | + +#### `consistency_ignores` + +Consistency findings someone looked at and decided are fine. + +| Column | Type | Notes | +|---|---|---| +| `id` | integer PK | | +| `key` | text NOT NULL, **unique** | The finding's stable key, so it stays ignored across runs. | +| `title` | text NOT NULL | What the finding said when it was ignored, so the list still reads sensibly after it's gone. | +| `reason`, `created_by` | text | | +| `created_at` | text NOT NULL | | + +## What's stored inside the JSON columns + +| Column | Shape | +|---|---| +| `servers.ip_addresses` | `["10.0.0.5", "fd00::5"]` | +| `servers.disks` | `[{ "mount": "/", "sizeBytes": 32000000000, "usedBytes": 9000000000 }]` | +| `servers.listening_ports` | `[{ "protocol": "tcp", "port": 22, "address": "0.0.0.0", "process": "sshd" }]` | +| `servers.last_port_scan` | `{ "at": "", "address": "10.0.0.5", "from": 1, "to": 1024, "open": 3, "refused": 1010, "filtered": 11, "responded": true }` | +| `servers.tags` | `["prod", "media"]` | +| `audit_log.detail` | Free-form per action — e.g. `{ "name": "pve1" }`, or for settings `{ "sections": [...], "changes": { "
": { "": { "from": …, "to": … } } } }`. For the notification channels (Gotify, ntfy, SMTP, webhook) only the field *names* that changed are recorded, never values. | +| `integrations.config` | The non-secret fields for that integration type (table below). | +| `dns_providers.config` | Non-secret provider settings (base URL, zone list, …). | + +**`integrations.config` by type** (the secret fields go to `integration_credentials` instead): + +| Type | In `config` | Encrypted separately | +|---|---|---| +| `proxmox` | `url`, `tokenId`, `insecure` | `tokenSecret` | +| `pbs` | `url`, `tokenId`, `insecure` | `tokenSecret` | +| `synology` | `url`, `username`, `insecure` | `password` | +| `semaphore`, `gitea`, `dockhand` | `url` | `token` | +| `tailscale` | `tailnet` | `apiKey` | +| `uptimekuma` | `url`, `username` | `password` (an API key, or the password on old installs) | +| `phpipam` | `url`, `appId`, `insecure` | `token` | +| `osticket` | `host`, `port`, `database`, `username`, `tablePrefix` | `password` | + +## Settings keys + +Each section is one row in `settings`, with a JSON object as its value. Fields +that were never changed aren't stored; the app fills in defaults when reading. + +| Key | Holds | +|---|---| +| `gotify`, `ntfy`, `smtp`, `webhook` | The four notification channels: enabled, address, and credentials (token / password / secret). The credential fields are stored encrypted (prefix `enc:v1:`) — see [Retention and backups](#retention-and-backups). | +| `notifications` | Which events notify (`dnsAdd`, `healthAlerts`, …), the daily-reminder time (`secretCheckTime`, default `08:00`) and `timezone` (default `UTC`), and the integration-failure threshold (default 3). | +| `quietHours` | `enabled`, `start`, `end`. | +| `healthChecks` | `serverOfflineMinutes` (60), `diskUsagePercent` (90), `domainWarnDays` (30). | +| `logRetention` | `enabled`, `retentionDays` (90), `intervalHours` (24). | +| `display` | `dateFormat`, `timeFormat`, `pageSize`. | +| `providerColors`, `integrationColors` | Badge colours, by provider / integration type. | +| `consistency` | `excludedRanges` — addresses the Consistency and IP Addresses pages ignore. Default `["172.16.0.0/12"]` (Docker's networks). | +| `nameGenerator` | `themes` — the Generator's server-name lists: `[{ "id", "label", "names": [...], "lastImport"? }]`. Edited under Settings → Names. | + +Rows whose key starts with **`_internal:`** are scheduler bookkeeping, not +settings, and aren't part of the app's settings object: + +| Key | Value | +|---|---| +| `_internal:secretCheckLastRunDate`, `tailscaleKeyCheckLastRunDate`, `dockerUpdateCheckLastRunDate`, `proxmoxBackupCheckLastRunDate`, `pbsVerificationCheckLastRunDate` | The date a daily check last ran, so a restart doesn't repeat it (or skip it). | +| `_internal:logRetentionLastRunAt` | When the log purge last ran. | +| `_internal:healthActiveConditions`, `_internal:automationActiveConditions` | JSON: the problems currently active, so each is announced once and again when it clears, and survives a restart. | + +## What happens on delete + +| Deleting… | Effect | +|---|---| +| a **server** | Its `scheduled_tasks`, `server_links` and `server_ports` are deleted with it. Port forwards that pointed at it stay, with `server_id` cleared. | +| a **DNS provider** | Its cached zones and records are deleted. | +| an **integration** or **DNS provider** | The credential row it used is deleted by the app (not by the database). | +| an **integration credential** | The integration / provider using it keeps existing, with `credential_id` cleared. | +| a **user** | Their audit entries stay; `actor_user_id` is cleared and `actor_label` keeps the name. | +| a **Proxmox integration** that servers are linked to | Those servers keep existing and lose their Proxmox link (all four `proxmox_*` columns). The app does this before deleting; the database alone would refuse — see the next section. | + +### Soft references (not foreign keys) + +These point at other rows by ID or name without the database enforcing it, so +a stale value is possible and the app treats it as "nothing there": + +- `maintenance_windows.target_id` (→ a server, integration or DNS provider, by `target_type`) +- `audit_log.target_id` +- `dns_records_cache.zone_id` (→ `dns_zones_cache.zone_id`, same provider) +- `servers.tags` (→ `tag_definitions.name`) +- `consistency_ignores.key` + +## The Proxmox link's foreign key + +`servers.proxmox_integration_id` is meant to clear itself when its integration +is deleted: `schema.ts` says `onDelete: "set null"`. The database doesn't do +that. Migration `0001` added the column as a plain +`REFERENCES integrations(id)`, and SQLite can't change a foreign key afterwards +without rebuilding the table, so the rule *in the database* is **no action**: +with foreign keys on, deleting an integration that a server points at is refused +with `FOREIGN KEY constraint failed`. + +The app works around it rather than rebuilding the table. The delete route in +[`server/src/routes/integrations.ts`](server/src/routes/integrations.ts) first +clears the four `proxmox_*` columns on every server linked to that integration, +then deletes it, and the audit entry records how many servers were unlinked +(`unlinkedServers`). Deleting an integration therefore works whether or not +servers are linked to it. Anything that deletes integrations some other way — +a hand-written SQL statement, say — has to do the same first. + +## Retention and backups + +**Retention.** Only the two logs are trimmed: `audit_log` and `diag_log` entries +older than `logRetention.retentionDays` are deleted by the purge job (off by +default), and `notification_queue` is emptied each time the quiet-hours digest is +sent. Everything else stays until someone deletes it. + +**Credentials at rest.** Integration and DNS provider credentials are +encrypted in `integration_credentials` (above). The notification channels' +credentials — the Gotify and ntfy tokens, the SMTP password and the webhook +secret — are in the `settings` rows, and are encrypted in place with the same key: +each is stored as `enc:v1:` followed by the same `iv:authTag:ciphertext` form, and the +rest of the row (URLs, topics, priorities) stays readable. The app decrypts them when +settings are read and encrypts them when they're written, so nothing else sees the +difference. + +- A value saved by an older version (plain text) still works, and is encrypted the + next time the server starts. +- Without `CREDENTIALS_ENCRYPTION_KEY`, new notification credentials can only be + stored as plain text, and the server warns about it at startup. They are encrypted + at the next start once the key is set. +- If the key is changed or lost, the stored credentials can't be read: they show as + empty, and the server logs which ones. Enter them again under Settings → + Notifications. Editing a *different* field of the same channel meanwhile doesn't + overwrite the old ciphertext, so putting the right key back restores them. + +Everything else in the file — IP addresses, hostnames, secret *names* and expiry +dates, the audit log — is readable by anyone who can read the file, so treat the +file and its backups as sensitive anyway. + +**Backups.** +- **Settings → Backup** exports the settings, the integrations and the DNS + providers (with their credentials decrypted, then wrapped in a file encrypted + with a passphrase you choose). Importing merges the settings over the current ones, and adds + integrations and providers that don't exist yet (matched by type and name) — it never + overwrites an existing integration. It does **not** include servers, tasks, + secrets, IP addresses, domains, ports, tags, maintenance windows or logs. +- **A full backup** is a copy of the SQLite file, together with the value of + `CREDENTIALS_ENCRYPTION_KEY` — without that key the stored integration + credentials can't be decrypted. Copy the file while the app is stopped, or use + SQLite's `.backup` command, so you don't capture it mid-write. + +## Changing the schema + +1. Edit [`server/src/db/schema.ts`](server/src/db/schema.ts). +2. From the repo root run `npm run db:generate`; it writes a new numbered SQL + file to `server/drizzle/` (and updates `server/drizzle/meta/`). +3. Read the generated SQL. SQLite can add a column but can't change or drop a + foreign key, so some changes become a table rebuild — which is also why the + [Proxmox link](#the-proxmox-links-foreign-key) described above is the way it is. +4. Start the server (or run `npm run db:migrate`); the migration is applied and + recorded in `__drizzle_migrations`. +5. Commit the schema, the SQL file and the `meta/` changes together, and update + this document. diff --git a/README.md b/README.md index 17e298b..bb87fc3 100644 --- a/README.md +++ b/README.md @@ -327,6 +327,11 @@ needs the site to be served over HTTPS (browsers only offer install on secure origins; `localhost` also counts). The bundled service worker deliberately caches nothing, so an installed copy always shows the current build. +**More documentation**: [ROLES.md](ROLES.md) — who can see and do what; +[NOTIFICATIONS.md](NOTIFICATIONS.md) — every notification the app sends and when; +[INTEGRATIONS.md](INTEGRATIONS.md) — the credentials each integration needs; +[DATABASE.md](DATABASE.md) — the database tables, columns and relationships. + ## Requirements - Node.js 20+