IT Break-Glass Sheet
CONFIDENTIAL. This is the online copy of the break-glass sheet. The printed copy is in the office safe, and only the printed copy has the passwords, handwritten after printing. Never type them into this page.
When to use this sheet
Open the envelope in the safe only if Robert can't be reached for more than 4 hours and one of these is true:
- GraphyFlow, email, phones or the internet are down and production is affected
- A security incident is suspected (ransom note, unknown logins, files encrypted or vanishing)
- Robert is incapacitated or has left the business
When it's opened, write down who, when and why on the printed sheet. Afterwards Robert changes every password used and prints a new sheet.
Getting in
Every other password is in Bitwarden. Get into Bitwarden first; everything else follows.
| Item | Where to find it |
|---|---|
| Bitwarden address | vault.maticmedia.co.uk |
| Bitwarden account email | Printed sheet in the safe |
| Bitwarden master password | Printed sheet in the safe |
| Bitwarden two-factor recovery code | Printed sheet in the safe |
| Bitwarden folder with admin logins | Printed sheet in the safe |
| Proxmox root password (if Bitwarden is down) | Printed sheet in the safe |
| OPNsense root password (if Bitwarden is down) | Printed sheet in the safe |
| Office Wi-Fi / LAN access, if needed on site | Printed sheet in the safe |
If Bitwarden itself is down: it runs in container 110 on pv1. Log in to Proxmox with the root password from the printed sheet, select pv1 → 110 (vault) → Start. Richard can also use Bitwarden emergency access from his own account; that request unlocks after the waiting period Robert set.
Working remotely: remote access is WireGuard on the OPNsense firewall (62.31.105.226, UDP 51820). Each device has its own peer, listed under VPN → WireGuard → Peers. A lost device is cut off by deleting its peer there.
Who to call
| Who | For | Phone | Account / reference |
|---|---|---|---|
| Robert McCombe | Everything | ||
| Outside IT support | Hands-on help using this sheet | ||
| Virgin Media Business | Internet line, static IPs 62.31.105.224/29 | ||
| SIP provider | Phone lines (.230) | ||
| DNSimple | DNS for most domains | ||
| Other domain registrar(s) | Domains not on DNSimple | ||
| Backblaze | Off-site backup of everything | ||
| Server hardware supplier | pv0, pv1, pv2 hardware faults |
Key addresses
As of 20 Nov 2026. Addresses change as the network plan goes in: update this page and reprint after each stage.
| What | Address | Notes |
|---|---|---|
| Proxmox (all three servers) | https://192.168.0.210:8006 | Log in as root. Ignore the certificate warning |
| OPNsense firewall | http://192.168.0.1 | Also reachable from the Proxmox console of VM 111 on pv1 |
| Office public IP (internet line) | 62.31.105.226 | Gateway 62.31.105.225 |
| Remote access VPN | WireGuard, 62.31.105.226 UDP 51820 | Tunnel 10.8.1.0/24. Firewall → VPN → WireGuard |
| Websites, GraphyFlow | 62.31.105.226–.229 | See the network plan |
| Phones | 62.31.105.230 | FreePBX, VM 806 on pv1 |
| Network plan and docs | BookStack: manual.maticmedia.co.uk | Printed PDF copy kept with the sheet in the safe |
| File server / backups | matictrack-storage, 192.168.0.10 (OpenMediaVault) | Sends everything to Backblaze |
| Server | Cores / RAM | Runs |
|---|---|---|
| pv0 | 56 threads / 252 GB | GraphyFlow (902), websites (811), search (813), dev (814), Frigate (801) |
| pv1 | 24 threads / 252 GB | Firewall (111), database (200), Redis (250, 251), mail (106), phones (806), Bitwarden (110) |
| pv2 | 32 threads / 189 GB | File storage (201), queue workers (903) |
Recovery order if everything is down
Start things in this order in Proxmox (select the server → the guest → Start). Wait for each step to come up before the next.
- Servers: power on pv1, then pv2, then pv0. Wait 5 minutes
- Firewall: VM 111 (opnsense-v3) on pv1. Without it nothing has internet
- File storage: VM 201 (matictrack-storage) on pv2. The others mount its shares
- Database and cache: CT 200 (MySQL), CT 250 and 251 (Redis) on pv1
- GraphyFlow: CT 813 (search), CT 902 (GraphyFlow) on pv0, CT 903 (queues) on pv2
- Mail and phones: VM 106 (Zimbra), VM 806 (FreePBX) on pv1
- Everything else: Bitwarden (110), FreeScout (808), Rocket.Chat (107), websites (811), then the rest
Check: graphicwarehouse.co.uk and graphyflow.com load, staff can log in, email arrives, phones ring.
Common situations
| Situation | First steps |
|---|---|
| Internet down for everyone | Check the Virgin router lights and call Virgin. In Proxmox, check VM 111 (firewall) on pv1 is running; start it if not |
| GraphyFlow down, internet fine | Check CT 902 on pv0 and CT 200 on pv1 are running. Restart 902 first, then 200 only if still down |
| One server won't start | Its guests are lost until it's fixed. Call the hardware supplier. Don't reinstall it: the guests' disks are still on it. Outside IT can restore guests to another server from the nightly backups |
| Suspected hack or ransomware | Unplug the internet line at the Virgin router. Don't pay, delete or reboot anything. Call outside IT support. Backblaze holds clean copies |
| A file or folder was deleted | Restore from OpenMediaVault on matictrack-storage (192.168.0.10), or from Backblaze |
| A server or container needs restoring | Proxmox → Datacenter → cifs-backup → Backups → pick the guest and date → Restore |
Keeping this sheet current
- Update this page and reprint after any change to a password on the sheet, and after each stage of the network plan
- Check Richard's Bitwarden emergency access works once a year
- Shred the old copy when the new one goes in the safe
Bitwarden emergency access for Richard set up and sheet printed: 20 Nov 2026.