Acknowledge and hand over alerts, incident timelines, and built-in database backups
- Alerts can be acknowledged (note, assignee, duration); acknowledged alerts are left out of what needs attention (sidebar, dashboard strip, cards) but stay raised and counted by /metrics - Incidents: each time an alert is raised and goes away is recorded (alert_acks, alert_incidents, incident_notes, migration 0032), with a timeline of acks, maintenance, changes, update runs, related alerts and notes; Incidents page and a digest section - Backups of Homelab Manager's own database: daily checked VACUUM INTO copies, retention, download, check, upload, delete, and restore staged and applied at start-up before the database opens (pre-restore copy kept); alerts and /metrics for stale or failing backups - Docs: README, BACKUP, DATABASE, NOTIFICATIONS, ROLES, API; tests Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
7a5b623444
commit
2744a84719
41 files changed
+6573
-26
No files matched your search
@@ -4,6 +4,7 @@ Everything Homelab Manager knows is in **one SQLite file**, plus a handful of se
|
||||
`.env`. A backup is those two things — and the part people get wrong is keeping them **apart**
|
||||
(see [Keeping backups safe](#keeping-backups-safe)).
|
||||
|
||||
- [The built-in backups (Settings → Backup)](#the-built-in-backups-settings--backup)
|
||||
- [What to back up](#what-to-back-up)
|
||||
- [Taking a backup](#taking-a-backup)
|
||||
- [Checking a backup](#checking-a-backup)
|
||||
@@ -18,6 +19,34 @@ Everything Homelab Manager knows is in **one SQLite file**, plus a handful of se
|
||||
The commands assume the standard Docker setup from the README: you run them in the folder that
|
||||
holds `docker-compose.yml`, `.env` and `data/`, and the service is called `homelab-manager`.
|
||||
|
||||
## The built-in backups (Settings → Backup)
|
||||
|
||||
You don't need the snippets below for the everyday case: Homelab Manager takes a backup of its own
|
||||
database **every day** (03:30 in the notification time zone, the newest 14 kept — both changeable, and it
|
||||
can be switched off), checks each one as it makes it, and keeps them in `data/backups/` with the same
|
||||
names the snippets use. **Settings → Backup** shows them and does the rest:
|
||||
|
||||
- **Back up now** makes one on the spot. **Download** takes a copy to your computer (the quickest way
|
||||
off the machine). **Check** opens a backup and says whether SQLite finds it sound, how many users,
|
||||
servers and integrations it holds, and whether it is from an older or newer version.
|
||||
- **Upload a backup…** takes a file from somewhere else (another machine, the script below) and keeps it
|
||||
if it checks out. A file that isn't a Homelab Manager database is refused and nothing is kept.
|
||||
- **Restore…** checks the backup and *stages* it. Nothing changes until the app next starts — restart it
|
||||
(`docker compose restart homelab-manager`) and, before anything opens the database, the staged file
|
||||
takes the database's place. The database that was in use is **kept** in `data/backups/` as
|
||||
`…-pre-restore.sqlite`, so a restore that wasn't what you wanted is undone by restoring that file. If
|
||||
something stops it half way, the old database is put back. A backup from a **newer** version than the
|
||||
one running is refused; an older one is fine (the start-up brings it up to date). **Cancel the restore**
|
||||
until you restart if you change your mind.
|
||||
- If the newest backup is more than two days old, or a scheduled one fails, there is an **alert** (and a
|
||||
message through your channels), and `/metrics` has `homelab_selfbackup_last_success_timestamp_seconds`
|
||||
for your own monitoring.
|
||||
|
||||
Three things it does **not** do, and the rest of this page is about: it does not hold
|
||||
`CREDENTIALS_ENCRYPTION_KEY` (keep that separately — see [Keeping backups safe](#keeping-backups-safe)),
|
||||
it does not copy anything off the machine on its own (Download, or copy `data/backups/` with your usual
|
||||
tool), and it does not replace a restore drill. Access to all of it is for administrators only.
|
||||
|
||||
## What to back up
|
||||
|
||||
| What | Where | Why | If you lose it |
|
||||
@@ -275,7 +304,17 @@ What you can get back, from least to most work:
|
||||
|
||||
## What has been tested, and what hasn't
|
||||
|
||||
Tested on a scratch database made with the real migrations:
|
||||
**The built-in backups** were tested with the app's own tests (a backup holds what was there and not what came
|
||||
after; two in the same second don't collide; garbage, someone else's database and a newer version's
|
||||
database are refused; the newest N are kept; a staged restore is checked, can be cancelled and puts nothing
|
||||
half-copied in place; a restore that can't complete leaves the database exactly as it was) and by a real
|
||||
restore: a backup was taken, a user added, the restore staged, the actual built app (`node dist/index.js`)
|
||||
restarted on the same data folder — the user added after the backup was gone, the previous database was
|
||||
kept as `…-pre-restore.sqlite`, and the next daily backup was scheduled. **Not** tested: running it inside the
|
||||
real Docker container, and a very large database (the copy is made on this machine's disk and takes about as
|
||||
long as copying the file).
|
||||
|
||||
The snippet-based way below was tested on a scratch database made with the real migrations:
|
||||
|
||||
- The snapshot (`VACUUM INTO`) taken from a second process **while the app was writing**: it
|
||||
completed in tens of milliseconds, passed the integrity check, held all migrations, and an
|
||||
|
||||
Reference in new issue
Block a user