:red_circle: HIGH SEV · Mock-IP Allocation Failure...
# alarms-prod
a
🔴 HIGH SEV · Mock-IP Allocation Failure Rate Critical (DNS broken for a real share of lookups) 🔥 Over 5% of DNS lookups in the tunnel are failing — kids' apps are intermittently not loading <|📈 graph> · <|📊 service dashboard> · 🚨 alarm definition · 🔕 silence _IMPACT: roughly 1 in 20 first-time domain lookups returns SERVFAIL, so apps fail to load. Expect support contact. It gets WORSE the longer it runs, because already-visited sites keep working from the device's 60s DNS cache and only new lookups break. NOT AT RISK: screen-time data. DNS failures are retryable; nothing is permanently lost and no enforcement is bypassed. CAUSE: ozone-proxy cannot allocate mock IPs because Redis calls are failing (2s timeout). There is no resolver fallback — all tunnel DNS is forced to our own mock server, so the device has nowhere else to go. DO: 1. Is Redis up and responsive? Check 'Redis Down', 'Redis Command Latency High', 'Redis Fork Stall'. 2. Is the pool exhausted rather than erroring? Check 'Mock-IP Pool Exhausted' — different fix. 3. If Redis looks healthy, suspect proxy→Redis connectivity or a maxclients ceiling ('Redis Connections Rejected'). NORMAL: ~1 error per week out of ~1.8M lookups. Escalated from the warning tier 'Mock-IP Allocation Errors Elevated'. Threshold is a RATIO with a 20-error floor on purpose — do not replace it with a count. A count-based version paged Urgent for a single transient error (CLE-699)._ Grafana vmain-b1e6358