With Settings -> Alert tickets switched on (off by default), every 15 minutes the problems on the
Alerts page are looked at and one that is new and has lasted gets one ticket in osTicket, through
its API (POST api/tickets.json, key in X-API-Key). One problem opens one ticket however many checks
see it; the app remembers what it has seen in a new alert_tickets table (migration 0015).
- Which: critical and warning by default, every kind; severities and kinds are settings.
- A problem must be seen in 2 checks in a row first (1-4, a setting), so a blip that fixes itself,
like one failed call to an integration, never opens a ticket.
- One ticket per occurrence; a problem that clears and returns within 60 minutes is the same one.
- More than 5 new problems in one check become a single summary ticket.
- Never for alerts under a maintenance window, nor about osTicket itself or its overdue tickets.
- Problems already there when it is switched on are left alone by default (the first check records
them as a baseline); can be set to ticket them too.
- If osTicket can't be reached nothing is recorded, so the next check retries; if a source couldn't
be read, a problem isn't treated as gone. Every ticket is audited.
The ticket API address, key (stored encrypted) and self-signed option are optional fields on the
osTicket integration, which otherwise still only reads the database. The settings page can send a
test ticket and run a check now; the Alerts page shows each alert's ticket number, linked to osTicket.
osTicket's API can't close or update a ticket, so a ticket opened here is the end of this app's part
in it. Tested against a stand-in osTicket, not a real one.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
One list of the current problems across servers and integrations, instead
of waiting for a notification or visiting each page: servers that stopped
reporting, full or nearly full disks and volumes (critical from 95%),
Synology volume/disk problems, failed or uncovered Proxmox backups,
failed Proxmox Backup Server verifications, container image updates,
expired or expiring secrets/domains/Tailscale keys, failed Semaphore and
Gitea runs, Uptime Kuma monitors that are down, overdue osTicket tickets,
and integrations whose calls keep failing. Visible to every role, with
severity and kind filters, search, sorting, CSV export and "Check now".
It runs the same checks that send the notifications rather than a second
copy of them: the detection in the health, automation, Proxmox backup,
PBS, Docker update and Tailscale key checks is pulled out into shared
collectors that both the schedulers and the page call, so the two can't
disagree about what counts as a problem. Notification behaviour is
unchanged, including the scheduled backup checks skipping integrations
under a maintenance window. Unlike the notifications the page ignores the
on/off toggles, and keeps problems under a maintenance window, marked
silenced and counted apart.
It reads live, so a result is reused for a minute (and Refresh can't
re-run everything more than once every ten seconds), and every source has
a 20 s limit so one hung integration can't hang the page. Anything it
couldn't read is called out at the top instead of looking like all clear,
and server checks pause for the same 20 minutes after a restart as the
notifications do, with a note saying so.
Also gives the newer integrations (PBS, osTicket, Uptime Kuma, phpIPAM)
proper names in "integration down" notifications instead of their ids.
Verified through the real routes against a scratch database with fake
backends (offline and full-disk servers, secrets and domains, a silenced
server, a fake PBS with failed verification, a hanging integration, a
refused one, a failing-calls streak, caching, the restart grace period,
auth), and by rendering the real page against that data in a browser:
filters, search, silenced toggle, sorting, Check now, dark mode.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>