# IT Break-Glass Sheet

**CONFIDENTIAL.** This is the online copy of the break-glass sheet. The printed copy is in the office safe, and only the printed copy has the passwords, handwritten after printing. Never type them into this page.

## When to use this sheet

Open the envelope in the safe only if Robert can't be reached for more than 4 hours **and** one of these is true:

- GraphyFlow, email, phones or the internet are down and production is affected
- A security incident is suspected (ransom note, unknown logins, files encrypted or vanishing)
- Robert is incapacitated or has left the business

**Authorised to open it:** Richard (MD). Anyone else needs Richard's permission.

When it's opened, write down who, when and why on the printed sheet. Afterwards Robert changes every password used and prints a new sheet.

## Getting in

Every other password is in Bitwarden. Get into Bitwarden first; everything else follows.

<table id="bkmrk-item-where-to-find-i" style="width: 100%;"><thead><tr><th>Item</th><th>Where to find it</th></tr></thead><tbody><tr><td>Bitwarden address</td><td>vault.maticmedia.co.uk</td></tr><tr><td>Bitwarden account email</td><td>Printed sheet in the safe</td></tr><tr><td>Bitwarden master password</td><td>Printed sheet in the safe</td></tr><tr><td>Bitwarden two-factor recovery code</td><td>Printed sheet in the safe</td></tr><tr><td>Bitwarden folder with admin logins</td><td>Printed sheet in the safe</td></tr><tr><td>Proxmox root password (if Bitwarden is down)</td><td>Printed sheet in the safe</td></tr><tr><td>OPNsense root password (if Bitwarden is down)</td><td>Printed sheet in the safe</td></tr><tr><td>Office Wi-Fi / LAN access, if needed on site</td><td>Printed sheet in the safe</td></tr></tbody></table>

**If Bitwarden itself is down:** it runs in container 110 on pv1. Log in to Proxmox with the root password from the printed sheet, select pv1 → 110 (vault) → Start. Richard can also use Bitwarden emergency access from his own account; that request unlocks after the waiting period Robert set.

**Working remotely:** remote access is WireGuard on the OPNsense firewall (62.31.105.226, UDP 51820). Each device has its own peer, listed under VPN → WireGuard → Peers. A lost device is cut off by deleting its peer there.

## Who to call

<table id="bkmrk-who-for-phone-accoun" style="width: 100%;"><thead><tr><th>Who</th><th>For</th><th>Phone</th><th>Account / reference</th></tr></thead><tbody><tr><td>Robert McCombe</td><td>Everything</td><td>  
</td><td>  
</td></tr><tr><td>Outside IT support</td><td>Hands-on help using this sheet</td><td>  
</td><td>  
</td></tr><tr><td>Virgin Media Business</td><td>Internet line, static IPs 62.31.105.224/29</td><td>  
</td><td>  
</td></tr><tr><td>SIP provider</td><td>Phone lines (.230)</td><td>  
</td><td>  
</td></tr><tr><td>DNSimple</td><td>DNS for most domains</td><td>  
</td><td>  
</td></tr><tr><td>Other domain registrar(s)</td><td>Domains not on DNSimple</td><td>  
</td><td>  
</td></tr><tr><td>Backblaze</td><td>Off-site backup of everything</td><td>  
</td><td>  
</td></tr><tr><td>Server hardware supplier</td><td>pv0, pv1, pv2 hardware faults</td><td>  
</td><td>  
</td></tr></tbody></table>

## Key addresses

As of 20 Nov 2026. Addresses change as the network plan goes in: update this page and reprint after each stage.

<table id="bkmrk-what-address-notes-p" style="width: 100%;"><thead><tr><th>What</th><th>Address</th><th>Notes</th></tr></thead><tbody><tr><td>Proxmox (all three servers)</td><td>https://192.168.0.210:8006</td><td>Log in as root. Ignore the certificate warning</td></tr><tr><td>OPNsense firewall</td><td>http://192.168.0.1</td><td>Also reachable from the Proxmox console of VM 111 on pv1</td></tr><tr><td>Office public IP (internet line)</td><td>62.31.105.226</td><td>Gateway 62.31.105.225</td></tr><tr><td>Remote access VPN</td><td>WireGuard, 62.31.105.226 UDP 51820</td><td>Tunnel 10.8.1.0/24. Firewall → VPN → WireGuard</td></tr><tr><td>Websites, GraphyFlow</td><td>62.31.105.226–.229</td><td>See the network plan</td></tr><tr><td>Phones</td><td>62.31.105.230</td><td>FreePBX, VM 806 on pv1</td></tr><tr><td>Network plan and docs</td><td>BookStack: manual.maticmedia.co.uk</td><td>Printed PDF copy kept with the sheet in the safe</td></tr><tr><td>File server / backups</td><td>matictrack-storage, 192.168.0.10 (OpenMediaVault)</td><td>Sends everything to Backblaze</td></tr></tbody></table>

<table id="bkmrk-server-cores-%2F-ram-r" style="width: 100%;"><thead><tr><th>Server</th><th>Cores / RAM</th><th>Runs</th></tr></thead><tbody><tr><td>pv0</td><td>56 threads / 252 GB</td><td>GraphyFlow (902), websites (811), search (813), dev (814), Frigate (801)</td></tr><tr><td>pv1</td><td>24 threads / 252 GB</td><td>Firewall (111), database (200), Redis (250, 251), mail (106), phones (806), Bitwarden (110)</td></tr><tr><td>pv2</td><td>32 threads / 189 GB</td><td>File storage (201), queue workers (903)</td></tr></tbody></table>

## Recovery order if everything is down

Start things in this order in Proxmox (select the server → the guest → Start). Wait for each step to come up before the next.

1. **Servers:** power on pv1, then pv2, then pv0. Wait 5 minutes
2. **Firewall:** VM 111 (opnsense-v3) on pv1. Without it nothing has internet
3. **File storage:** VM 201 (matictrack-storage) on pv2. The others mount its shares
4. **Database and cache:** CT 200 (MySQL), CT 250 and 251 (Redis) on pv1
5. **GraphyFlow:** CT 813 (search), CT 902 (GraphyFlow) on pv0, CT 903 (queues) on pv2
6. **Mail and phones:** VM 106 (Zimbra), VM 806 (FreePBX) on pv1
7. **Everything else:** Bitwarden (110), FreeScout (808), Rocket.Chat (107), websites (811), then the rest

Check: graphicwarehouse.co.uk and graphyflow.com load, staff can log in, email arrives, phones ring.

## Common situations

<table id="bkmrk-situation-first-step" style="width: 100%;"><thead><tr><th>Situation</th><th>First steps</th></tr></thead><tbody><tr><td>Internet down for everyone</td><td>Check the Virgin router lights and call Virgin. In Proxmox, check VM 111 (firewall) on pv1 is running; start it if not</td></tr><tr><td>GraphyFlow down, internet fine</td><td>Check CT 902 on pv0 and CT 200 on pv1 are running. Restart 902 first, then 200 only if still down</td></tr><tr><td>One server won't start</td><td>Its guests are lost until it's fixed. Call the hardware supplier. Don't reinstall it: the guests' disks are still on it. Outside IT can restore guests to another server from the nightly backups</td></tr><tr><td>Suspected hack or ransomware</td><td>**Unplug the internet line** at the Virgin router. Don't pay, delete or reboot anything. Call outside IT support. Backblaze holds clean copies</td></tr><tr><td>A file or folder was deleted</td><td>Restore from OpenMediaVault on matictrack-storage (192.168.0.10), or from Backblaze</td></tr><tr><td>A server or container needs restoring</td><td>Proxmox → Datacenter → cifs-backup → Backups → pick the guest and date → Restore</td></tr></tbody></table>

## Keeping this sheet current

- Update this page and reprint after any change to a password on the sheet, and after each stage of the network plan
- Check Richard's Bitwarden emergency access works once a year
- Shred the old copy when the new one goes in the safe

Bitwarden emergency access for Richard set up and sheet printed: 20 Nov 2026.