VPN Standup — 2026-03-11 Window:

02:54 – 14:54 UTC

🔴 CRITICAL — Production ozone-proxy has zero traffic All production metrics read zero for the entire 12h window: active tunnels, connections, bytes relayed. Latency histograms return NaN. Either the service is down, clients aren't routing to prod, or the scrape target is unreachable. → Action: Verify ozone-vpn-prod-us-west-1 is running and accepting connections. Check Prometheus scrape target status. ⚠️ WARNING — Staging inference backend returning 500s [INFERENCE] activity reporting returning

500 Internal Server Error

and [ENFORCEMENT] returning

434

across multiple sessions (Claude AI Assistant, Notion, Slack, Zoom). 15+ occurrences. Not blocking proxy function but session tracking is broken. → Action: Investigate inference API health. ⚠️ WARNING — Staging DOMAINCOLLECTION 400 errors with corrupted data Continuous

400 Bad Request

from domain cooccurrence recording. Domain lists contain null-byte entries mixed with legitimate domains. Affects data pipeline, not proxy routing. → Action: Sanitize domain collection input; check for garbled SNI from raw IP connections. ⚠️ WARNING — Staging direct dial p99 latency spikes to ~12s Periodic spikes well above the 500ms direct threshold. Correlates with repeated i/o timeouts to

68.232.201.178:443

(~10 failures). Setup p99 otherwise healthy at ~248ms. → Action: Investigate connectivity to 68.232.201.178. Consider blackholing if unreachable. ℹ️ INFO — High never-connected WireGuard peers Prod: 24 never-connected, Staging: 54 never-connected. Consider auditing stale provisioned entries. ℹ️ INFO — iOS client telemetry silent Zero events across all VPN telemetry (both prod and staging PostHog projects). No connect attempts, no tunnel starts, no errors, no active devices. Possible telemetry regression — verify PostHog SDK initialization. ℹ️ INFO — Tempo error traces correlate with Loki findings ~20 ozone-proxy + ~20 wgcm error traces in Tempo. No slow DNS, no slow connections, no wgcm 401s. Error traces share root cause with inference backend 500s. ✅ Services OK • wgcm: Zero operation errors. Peers stable (prod=16, staging=36, dev=48). No restarts. No 401s. • Restarts: ozone-proxy=0, wgcm=0 🎫 Open Linear Tickets • CLE-158 — iOS VPN stale DNS on device wake (In Progress, Urgent)