Files
Homelab-manager/BACKUP.md
T
bobbanandClaude Sonnet 5.5 51ad309e35 Document the backup and restore procedure in BACKUP.md
What to back up (the database file, and CREDENTIALS_ENCRYPTION_KEY kept apart from it), two ways to take a
backup (stop and copy, or a VACUUM INTO snapshot while the app runs, using only Node and the app's own database
library since the container has no sqlite3), checking a backup, taking one before every upgrade, keeping
backups safe, a nightly script with rotation, step-by-step restore (an older backup restores into a newer
version, not the reverse; what a wrong key looks like), and how the encrypted export in Settings -> Backup
differs from a real backup.

The commands were run against a scratch database made with the real migrations: a snapshot taken while the
app was writing, an older version's database upgrading itself when the current version started, the wrong-key
error, and the rotation line. They were not run through docker compose exec in the real container.

DATABASE.md pointed at SQLite's .backup command, which the container doesn't have; it now points here.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 00:28:58 +02:00

294 lines
15 KiB
Markdown

# Backup and restore
Everything Homelab Manager knows is in **one SQLite file**, plus a handful of settings in your
`.env`. A backup is those two things — and the part people get wrong is keeping them **apart**
(see [Keeping backups safe](#keeping-backups-safe)).
- [What to back up](#what-to-back-up)
- [Taking a backup](#taking-a-backup)
- [Checking a backup](#checking-a-backup)
- [Before an upgrade](#before-an-upgrade)
- [Keeping backups safe](#keeping-backups-safe)
- [Automating it](#automating-it)
- [Restoring](#restoring)
- [The encrypted export in Settings → Backup](#the-encrypted-export-in-settings--backup)
- [If there is no backup](#if-there-is-no-backup)
- [What has been tested, and what hasn't](#what-has-been-tested-and-what-hasnt)
The commands assume the standard Docker setup from the README: you run them in the folder that
holds `docker-compose.yml`, `.env` and `data/`, and the service is called `homelab-manager`.
## What to back up
| What | Where | Why | If you lose it |
|---|---|---|---|
| **The database** | `data/homelab-manager.sqlite` | Everything: users and roles, servers and their agent tokens (as hashes), scheduled tasks, secrets, IP addresses, DNS caches, domains, ports, tags, maintenance windows, both logs, settings, the integrations and DNS providers (with their credentials, encrypted), the alert-ticket memory | You start from nothing — see [If there is no backup](#if-there-is-no-backup) |
| **`CREDENTIALS_ENCRYPTION_KEY`** | `.env` | Decrypts every stored credential: integration tokens and passwords, DNS provider credentials, and the Gotify/ntfy/SMTP/webhook credentials | The database restores, but every credential in it is unreadable and has to be entered again |
| **The rest of `.env`** | `.env` | `SESSION_SECRET`, `APP_BASE_URL`, the Authentik client id and secret | `SESSION_SECRET`: everyone has to sign in again, nothing more. The Authentik values can be copied from Authentik again |
| Your `docker-compose.yml` and the image version | next to `.env` | So you can bring it back the way it was. Note which **version** each backup came from (the image tag, or the git commit) | Re-create it from the README |
You do **not** need `data/sessions/` (the sign-ins; everyone just signs in again), the Docker image
(it is rebuilt or pulled), or anything in Authentik, osTicket, Proxmox and the other systems the
app talks to — those have their own backups.
The agents on your servers keep working after a restore: each server's token is stored in the
database, as a hash, and the agent holds the token itself.
## Taking a backup
### Option A — stop, copy, start (simplest)
A few seconds of downtime, nothing to install, and nothing that can go wrong halfway:
```bash
docker compose stop homelab-manager
sudo tar czf ~/homelab-manager-data-$(date +%F-%H%M).tar.gz --exclude=data/sessions --exclude=data/backups data
docker compose start homelab-manager
```
(`sudo` because the container runs as root, so the files in `data/` are root's.)
### Option B — while it runs
Safe to run while the app is working: SQLite makes a consistent snapshot of the file with
`VACUUM INTO`, even if something is writing at that moment. The container has no `sqlite3` tool,
so this uses Node and the library the app already has. The snapshot lands in `data/backups/` on
your host, named with the date and time:
```bash
docker compose exec -T homelab-manager node - <<'EOF'
const fs = require("fs");
const { createClient } = require("@libsql/client");
const dir = "/app/data/backups";
fs.mkdirSync(dir, { recursive: true });
const stamp = new Date().toISOString().slice(0, 19).replace(/[:T]/g, "-");
const out = `${dir}/homelab-manager-${stamp}.sqlite`;
const c = createClient({ url: "file:/app/data/homelab-manager.sqlite" });
c.execute(`VACUUM INTO '${out}'`).then(
() => { console.log("wrote", out); c.close(); },
(e) => { console.error("backup failed:", e.message); process.exit(1); },
);
EOF
```
It takes well under a second for a database of this size. SQLite will not overwrite an existing
file, which is why the name includes the time; two runs in the same second would collide.
**Which one?** Use B for the regular, automatic backup and A before a risky change if you want
the extra certainty. Either way, the file stays on the same disk as the app until you copy it
somewhere else — see below.
Also copy `.env` — separately from the database (next section):
```bash
cp .env ~/homelab-manager-env-$(date +%F)
chmod 600 ~/homelab-manager-env-$(date +%F)
```
## Checking a backup
A backup you have never opened is a hope, not a backup. This opens the **newest** file in
`data/backups/`, runs SQLite's own integrity check and shows what is in it:
```bash
docker compose exec -T homelab-manager node - <<'EOF'
const fs = require("fs");
const { createClient } = require("@libsql/client");
const dir = "/app/data/backups";
const newest = fs.readdirSync(dir).filter((f) => f.endsWith(".sqlite")).sort().pop();
if (!newest) { console.error("no backups found in", dir); process.exit(1); }
const c = createClient({ url: `file:${dir}/${newest}` });
(async () => {
const one = async (sql) => Object.values((await c.execute(sql)).rows[0])[0];
console.log(newest);
console.log(" integrity check:", await one("PRAGMA integrity_check"));
console.log(" migrations applied:", await one("select count(*) from __drizzle_migrations"));
console.log(" users:", await one("select count(*) from users"), "| servers:", await one("select count(*) from servers"), "| integrations:", await one("select count(*) from integrations"));
c.close();
})();
EOF
```
You want `integrity check: ok` and numbers that look like your system. The migration count is the
version of the database: it goes up by one whenever a release changes the schema.
A tar from Option A is checked by listing it: `tar tzf homelab-manager-data-….tar.gz | head`.
Now and then — say after you first set it up, and after big changes — do a real **restore
drill**: copy a backup to a spare machine or a second folder, start a second instance on another
port with the same `.env`, and sign in. That is the only test that proves the whole chain,
including the key.
## Before an upgrade
Take a backup first, every time. A release can change the database, and the change **only goes
forward**: the new version upgrades the file the first time it starts.
- Restoring an **older** backup into a **newer** version is fine; the newer version brings it
up to date when it starts, and keeps the data.
- Restoring a **newer** backup into an **older** version is **not supported**. The older version
doesn't know the newer tables or columns and may fail in odd ways. If you roll back the
software, roll back the data with it: use the backup you took before the upgrade.
## Keeping backups safe
A database backup holds a lot: the audit log, every server's name and address, the hashed agent
tokens, and all your credentials — encrypted, which is exactly why the key matters.
1. **Keep the key apart from the database.** If the database file and `CREDENTIALS_ENCRYPTION_KEY`
are stolen together, the credentials are readable. Put the database backups where they are
convenient, and the key in a password manager or somewhere else entirely. You need both to
restore a working system, and neither alone is useful to a thief.
2. **Copy it off the machine.** `data/backups/` sits on the same disk as the app, so a dead disk
takes both. Copy the files to another machine, a NAS, or encrypted cloud storage.
3. **Restrict access.** `sudo chmod -R go-rwx data/backups` and the same for wherever you copy to.
4. **Keep a few.** The last 7 daily and 4 weekly is plenty for a system that changes this slowly.
A single backup is not enough: a problem you only notice a week later is already in last night's.
## Automating it
A script that takes the snapshot, keeps the newest 14, and (if you fill it in) copies the newest
off the machine. Save it as `/usr/local/bin/homelab-manager-backup.sh`, make it executable, and
set `APP_DIR`:
```bash
#!/usr/bin/env bash
set -euo pipefail
APP_DIR=/opt/homelab-manager # the folder with docker-compose.yml and data/
KEEP=14
cd "$APP_DIR"
docker compose exec -T homelab-manager node - <<'EOF'
const fs = require("fs");
const { createClient } = require("@libsql/client");
const dir = "/app/data/backups";
fs.mkdirSync(dir, { recursive: true });
const stamp = new Date().toISOString().slice(0, 19).replace(/[:T]/g, "-");
const out = `${dir}/homelab-manager-${stamp}.sqlite`;
const c = createClient({ url: "file:/app/data/homelab-manager.sqlite" });
c.execute(`VACUUM INTO '${out}'`).then(
() => { console.log("wrote", out); c.close(); },
(e) => { console.error("backup failed:", e.message); process.exit(1); },
);
EOF
# keep the newest $KEEP, delete the older ones (other files in the folder are left alone)
ls -1t data/backups/homelab-manager-*.sqlite | tail -n +$((KEEP + 1)) | xargs -r rm --
# copy the newest one off this machine — edit, or remove if you do that another way
# rsync -a "$(ls -1t data/backups/homelab-manager-*.sqlite | head -1)" backuphost:/backups/homelab-manager/
```
Run it nightly from root's crontab (`sudo crontab -e`):
```
17 3 * * * /usr/local/bin/homelab-manager-backup.sh >> /var/log/homelab-manager-backup.log 2>&1
```
A script that fails quietly is the usual way backups stop. Look at the log now and then, or add
a line that pings something you watch (Uptime Kuma's push monitors are good at this: no ping,
you get told).
## Restoring
You need: a backup of the database, and the `CREDENTIALS_ENCRYPTION_KEY` that was in use when it
was taken (from your `.env` backup).
1. **Stop the app.**
```bash
docker compose stop homelab-manager
```
2. **Keep what is there**, in case the restore isn't what you wanted:
```bash
sudo mv data "data.before-restore-$(date +%F-%H%M)"
mkdir data
```
(If your backups live in `data/backups/`, they are now in the folder you just moved. Take the
one you want from there.)
3. **Put the database in place**, under the name the app expects:
```bash
sudo cp /path/to/homelab-manager-2026-10-07-03-17-00.sqlite data/homelab-manager.sqlite
```
From an Option A tar instead: `sudo tar xzf homelab-manager-data-….tar.gz` in this folder.
4. **Check `.env`.** On the same machine it is probably still right. On a new machine, restore it
from your backup. The one value that must match is `CREDENTIALS_ENCRYPTION_KEY`.
5. **Start it.** The first start brings the database up to the current version by itself.
```bash
docker compose up -d
docker compose logs --tail 30 homelab-manager
curl -fsS http://localhost:3000/health
```
6. **Sign in** and look around: the Servers page should show your servers (as "last seen" at the
time of the backup, until their agents report again, within 15 minutes), and Integrations
should list everything.
### If the key doesn't match
Symptoms: an integration or a DNS provider fails with *"Unsupported state or unable to
authenticate data"*, and notification channels that worked before don't. The database is fine;
only the stored credentials can't be read. If you can't find the old key, enter the credentials
again (Integrations → edit, DNS providers, Settings → Notifications). Nothing else is lost.
**Don't** change `CREDENTIALS_ENCRYPTION_KEY` on a running system to "fix" this: it makes
everything stored so far unreadable.
### After a restore
- Servers whose agents were installed **after** the backup was taken have no token in the
restored database. Press **Rotate token** on that server's row on the Servers page: it shows the new token together with a ready-to-copy install command, to run on that server.
- Anything changed since the backup is gone: new users, edits, audit entries. The audit log will
not show the gap, so it is worth noting it somewhere.
- Everyone has to sign in again if you restored without `data/sessions/`. That is expected.
## The encrypted export in Settings → Backup
A different tool, for a different job. **Settings → Backup** downloads one file, encrypted with a
passphrase you choose (AES-256-GCM, key derived with scrypt), holding:
- the settings (notification channels, display, log retention, alert tickets, and so on),
- the integrations and DNS providers, with their credentials.
Because it is encrypted with your passphrase and not with `CREDENTIALS_ENCRYPTION_KEY`, it is
**portable**: it restores onto a fresh install that has a different key. Importing merges the
settings over the current ones and adds integrations and providers that don't exist yet, matched
by type and name; it never overwrites an existing one.
It does **not** contain servers (and their agent tokens), tasks, secrets, IP addresses, domains,
ports, tags, maintenance windows, the users, or either log. So it is the right thing for moving
your *configuration* to a new instance, or as a second copy of the part that is tedious to
re-enter. It is **not** a backup of the system: use the database backup for that.
Keep the passphrase somewhere safe; it can't be recovered.
## If there is no backup
What you can get back, from least to most work:
- **You have a Settings → Backup export:** a fresh install plus an import restores the settings,
integrations and DNS providers. Everything else is re-created.
- **You only have the `.env`:** a fresh install with the same file works, and everything in the
database starts empty. Sign in (the first account becomes admin), re-add the integrations,
register each server again on the Servers page — it shows a ready-to-copy install command with each
new token — and run that on the server. IP addresses, secrets, domains, ports and tags
are typed in again; DNS records, IPAM syncs and the Tailscale/Proxmox/Docker views come back by
syncing.
- **You have neither:** the same, plus new integration credentials and a new key.
## What has been tested, and what hasn't
Tested on a scratch database made with the real migrations:
- The snapshot (`VACUUM INTO`) taken from a second process **while the app was writing**: it
completed in tens of milliseconds, passed the integrity check, held all migrations, and an
encrypted credential in it decrypted with the same key.
- The snippets above, run as written from standard input (`node -`), including a repeat run (which
is why the name has the time in it), the check snippet, and the "keep the newest N" line.
- An **older version's** database restored into the current version: it upgraded itself on start and
kept its data.
- A stored credential with the **wrong** key fails with the message quoted above.
**Not** tested here: running the commands through `docker compose exec` in the real container
(there was no Docker in the test environment — the snippets use only Node and the app's own
database library, which the container has, and the app's working folder `/app` is where `require`
looks), the cron line, the off-machine copy, and a full restore drill on your hardware. Do that
drill once; it is the one check that covers all of them.