https://livekit.io logo
Join Slack
Powered by
LiveKit team - Is livekit cloud down in US? Seeing...
# cloud
p
LiveKit team - Is livekit cloud down in US? Seeing multiple errors in the system
Copy code
Cannot connect to host prodigal-prod-eshhqp5l.livekit.cloud:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1006)')]", "room_name": "tcn_1fa_use2_000_inbound_prod_deployment_+14357996345_7TkEinap9j2T", "tenant_id": "1fa_use2_000", "call_id": "call_AJ_oMhZNZYK3ZNg", "exc_info": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.11/site-packages/
@dry-elephant-14928 @magnificent-art-43333 -> Apologies for tagging you but seems like livekit is down in US region?
Copy code
{
  "message": "livekit_ffi::server::room:286:livekit_ffi::server::room - error while connecting to a room: engine: signal failure: failed to retrieve region info: error sending request for url (<https://test-ow0r7onl.livekit.cloud/settings/regions>)",
  "level": "ERROR",
  "name": "livekit",
  "room": "twilio_dev_use2_000_inbound_dev_deployment_+14083573628_ZhnDB9TkVN99",
  "worker_id": "AW_BUR2kcmPQW9u",
  "pid": 206,
  "job_id": "AJ_6sDtXzJhaLES",
  "room_id": "RM_NFCGkK2Auc4f",
  "timestamp": "2025-12-28T00:03:19.576850+00:00"
}
Facing issues in both our projects - prod and dev
@alert-easter-19258 team - pls help we are facing production outage -
Copy code
nlivekit.rtc.room.ConnectError: engine: signal failure: failed to retrieve region info: error sending request for url (<https://prodigal-prod-eshhqp5l.livekit.cloud/settings/regions>
d
@proud-match-31207 this showing SSL being rejected on the client. where are you running the agent?
are you able to replicate this by visiting your project site?
p
This is a self hosted agent that we are hosting on our side
All of the calls are failing to connect with this erro rmessage
Copy code
engine: signal failure: failed to retrieve region info: error sending request for url (<https://prodigal-prod-eshhqp5l.livekit.cloud/settings/regions>): error trying to connect: invalid peer certificate
Is this related to any certificate that we should have on our end? The agent is self-hosted but we are using Livekit cloud
d
where is it hosted at? and does it have CA certs installed?
p
It is hosted in us-east-2. Yeah it was working fine 10 mins back. Apologies again if this is something related on our side. But nothing has changed in the last 10 mins
Environment is AWS EKS, running as a pod with python3.11 runtime and 1.2.17 agents SDK version Even the latest agents sdk with 1.3.10 in dev env is failing with the same error.
are you able to replicate this by visiting your project site?
I am unable to see any error on the UI dashboard of livekit cloud.
d
looking into it. the error is related to SSL, possibly on one of our clusters
🙏 1
f
We (cai) also started seeing massive connection failures right at midnight GMT (ie 34 min ago)
but seems to be recovering
d
the issue should be recovering. for some reasons the auto renewal process was stuck for this region.
🙏 2
p
Thank you can confirm - I don't see the error anymore.
🙏 1
d
sorry about that folks. we will be sharing more details in the post mortem to be posted later: https://status.livekit.io/incidents/g4p04fj2vnh0
👍 1
Open in Slack
PreviousNext
Powered by