IT Break-Glass Sheet

CONFIDENTIAL. This is the online copy of the break-glass sheet. The printed copy is in the office safe, and only the printed copy has the passwords, handwritten after printing. Never type them into this page.

When to use this sheet

Open the envelope in the safe only if Robert can't be reached for more than 4 hours and one of these is true:

Authorised to open it: Richard (MD). Anyone else needs Richard's permission.

When it's opened, write down who, when and why on the printed sheet. Afterwards Robert changes every password used and prints a new sheet.

Getting in

Every other password is in Bitwarden. Get into Bitwarden first; everything else follows.

Item Where to find it
Bitwarden address vault.maticmedia.co.uk
Bitwarden account email Printed sheet in the safe
Bitwarden master password Printed sheet in the safe
Bitwarden two-factor recovery code Printed sheet in the safe
Bitwarden folder with admin logins Printed sheet in the safe
Proxmox root password (if Bitwarden is down) Printed sheet in the safe
OPNsense root password (if Bitwarden is down) Printed sheet in the safe
Office Wi-Fi / LAN access, if needed on site Printed sheet in the safe

If Bitwarden itself is down: it runs in container 110 on pv1. Log in to Proxmox with the root password from the printed sheet, select pv1 → 110 (vault) → Start. Richard can also use Bitwarden emergency access from his own account; that request unlocks after the waiting period Robert set.

Working remotely: remote access is WireGuard on the OPNsense firewall (62.31.105.226, UDP 51820). Each device has its own peer, listed under VPN → WireGuard → Peers. A lost device is cut off by deleting its peer there.

Who to call

Who For Phone Account / reference
Robert McCombe Everything

Outside IT support Hands-on help using this sheet

Virgin Media Business Internet line, static IPs 62.31.105.224/29

SIP provider Phone lines (.230)

DNSimple DNS for most domains

Other domain registrar(s) Domains not on DNSimple

Backblaze Off-site backup of everything

Server hardware supplier pv0, pv1, pv2 hardware faults

Key addresses

As of 20 Nov 2026. Addresses change as the network plan goes in: update this page and reprint after each stage.

What Address Notes
Proxmox (all three servers) https://192.168.0.210:8006 Log in as root. Ignore the certificate warning
OPNsense firewall http://192.168.0.1 Also reachable from the Proxmox console of VM 111 on pv1
Office public IP (internet line) 62.31.105.226 Gateway 62.31.105.225
Remote access VPN WireGuard, 62.31.105.226 UDP 51820 Tunnel 10.8.1.0/24. Firewall → VPN → WireGuard
Websites, GraphyFlow 62.31.105.226–.229 See the network plan
Phones 62.31.105.230 FreePBX, VM 806 on pv1
Network plan and docs BookStack: manual.maticmedia.co.uk Printed PDF copy kept with the sheet in the safe
File server / backups matictrack-storage, 192.168.0.10 (OpenMediaVault) Sends everything to Backblaze
Server Cores / RAM Runs
pv0 56 threads / 252 GB GraphyFlow (902), websites (811), search (813), dev (814), Frigate (801)
pv1 24 threads / 252 GB Firewall (111), database (200), Redis (250, 251), mail (106), phones (806), Bitwarden (110)
pv2 32 threads / 189 GB File storage (201), queue workers (903)

Recovery order if everything is down

Start things in this order in Proxmox (select the server → the guest → Start). Wait for each step to come up before the next.

  1. Servers: power on pv1, then pv2, then pv0. Wait 5 minutes
  2. Firewall: VM 111 (opnsense-v3) on pv1. Without it nothing has internet
  3. File storage: VM 201 (matictrack-storage) on pv2. The others mount its shares
  4. Database and cache: CT 200 (MySQL), CT 250 and 251 (Redis) on pv1
  5. GraphyFlow: CT 813 (search), CT 902 (GraphyFlow) on pv0, CT 903 (queues) on pv2
  6. Mail and phones: VM 106 (Zimbra), VM 806 (FreePBX) on pv1
  7. Everything else: Bitwarden (110), FreeScout (808), Rocket.Chat (107), websites (811), then the rest

Check: graphicwarehouse.co.uk and graphyflow.com load, staff can log in, email arrives, phones ring.

Common situations

Situation First steps
Internet down for everyone Check the Virgin router lights and call Virgin. In Proxmox, check VM 111 (firewall) on pv1 is running; start it if not
GraphyFlow down, internet fine Check CT 902 on pv0 and CT 200 on pv1 are running. Restart 902 first, then 200 only if still down
One server won't start Its guests are lost until it's fixed. Call the hardware supplier. Don't reinstall it: the guests' disks are still on it. Outside IT can restore guests to another server from the nightly backups
Suspected hack or ransomware Unplug the internet line at the Virgin router. Don't pay, delete or reboot anything. Call outside IT support. Backblaze holds clean copies
A file or folder was deleted Restore from OpenMediaVault on matictrack-storage (192.168.0.10), or from Backblaze
A server or container needs restoring Proxmox → Datacenter → cifs-backup → Backups → pick the guest and date → Restore

Keeping this sheet current

Bitwarden emergency access for Richard set up and sheet printed: 20 Nov 2026.


Revision #1
Created 20 November 2026 21:36:21 by Admin
Updated 20 November 2026 21:37:32 by Admin