Why is my egress setup failing?
# ask-aib
broad-teacher-94403
02/17/2026, 10:57 AMWhy is my egress setup failing?
t
tall-airline-16756
02/17/2026, 10:57 AMThanks for reaching out, Prathamesh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
tall-airline-16756
02/17/2026, 10:58 AMThanks — I’m sorry you’re running into egress failures. I can help debug this, but I need a bit more detail to pinpoint the root cause. Below are the most common causes and the quick checks you can run now, plus the exact information I need from you to continue.
Common causes and quick checks
• Upload to cloud storage failed: If the webhook
egress_endedbackupStorageUsed: trues3_assume_role_enabled=trues3_assume_role_arns3_assume_role_external_idegress_endedegress_endedbackupStorageUsedb
broad-teacher-94403
02/17/2026, 2:04 PMI am using a self hosted livekit and egress. The bucket is GCP with workload identity. Here are the logs from egress:
2026-02-16T152747.313Z INFO egress redis/redis.go:142 connecting to redis {"nodeID": "NE_iofUiWxBotVd", "clusterID": "", "simple": true, "addr": "redis-14764.redis-dev-west.corp.cvscaremark.com:14764"}
2026-02-16T152747.595Z INFO egress stats/monitor.go:137 cpu available: 768.000000 max cost: 4.000000 {"nodeID": "NE_iofUiWxBotVd", "clusterID": ""}
2026-02-16T152747.600Z DEBUG egress server/server.go:141 starting service {"nodeID": "NE_iofUiWxBotVd", "clusterID": "", "version": "1.8.6"}
2026-02-16T152747.601Z INFO egress server/server.go:147 service ready {"nodeID": "NE_iofUiWxBotVd", "clusterID": ""}
2026-02-16T152747.601Z DEBUG egress server/server.go:133 starting template server on address localhost:7980 {"nodeID": "NE_iofUiWxBotVd", "clusterID": ""}
2026-02-16T152800.798Z DEBUG egress stats/monitor.go:151 cpu check {"nodeID": "NE_iofUiWxBotVd", "clusterID": "", "total": 768, "available": 768, "pending": 0, "used": 0, "activeRequests": 0, "activeWeb": 0, "required": 1, "canAccept": true}
2026-02-16T152801.300Z INFO egress server/server_rpc.go:58 request received {"nodeID": "NE_iofUiWxBotVd", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152801.303Z INFO egress server/server_rpc.go:75 request validated {"nodeID": "NE_iofUiWxBotVd", "clusterID": "", "egressID": "EG_JvTieZd74TmW", "requestType": "room_composite", "outputType": "file", "room": "outbound-call-20260216-152756-ebf04b0e", "request": {"RoomComposite":{"room_name":"outbound-call-20260216-152756-ebf04b0e","audio_only":true,"Output":null,"Options":null,"file_outputs":[{"file_type":2,"filepath":"recordings/1251a2b4-84f5-4793-a2e9-19b8fb8c25e5/a40e3fa3-0cf0-4070-8d7e-d8694308f230.ogg","Output":{"Gcp":{"bucket":"livekit-recordings-nonprod"}}}]}}}
2026-02-16T152801.334Z DEBUG egress server/main.go:177 handler launched {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152801.334Z INFO egress redis/redis.go:142 connecting to redis {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW", "simple": true, "addr": "redis-14764.redis-dev-west.corp.cvscaremark.com:14764"}
2026-02-16T152801.465Z DEBUG egress source/web.go:159 creating pulse sink {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152801.471Z DEBUG egress source/web.go:183 creating X display {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW", "display": ":2124302578", "dims": "1280x720x24"}
2026-02-16T152801.471Z DEBUG egress source/web.go:214 launching chrome {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW", "url": "http://localhost:7980/?layout=&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3NzEzNDIwODEsImlzcyI6IkFQSUtFWTY4YjBlMGMyZTA2YWVhNTEiLCJraW5kIjoiZWdyZXNzIiwibmJmIjoxNzcxMjU1NjgxLCJzdWIiOiJFR19KdlRpZVpkNzRUbVciLCJ2aWRlbyI6eyJjYW5QdWJsaXNoIjpmYWxzZSwiY2FuUHVibGlzaERhdGEiOmZhbHNlLCJjYW5TdWJzY3JpYmUiOnRydWUsImhpZGRlbiI6dHJ1ZSwicmVjb3JkZXIiOnRydWUsInJvb20iOiJvdXRib3VuZC1jYWxsLTIwMjYwMjE2LTE1Mjc1Ni1lYmYwNGIwZSIsInJvb21Kb2luIjp0cnVlfX0.ilWXkVo-Nr2xFAMrQLxQpArPrnOSvs0AYK_KBB1rBJs&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com", "sandbox": false, "insecure": false}
00000.140815851 95 0x56453105aa10 WARN cudaloader gstcudaloader.c169gst_cuda_load_library: Could not open library libcuda.so.1, libcuda.so.1: cannot open shared object file: No such file or directory
00000.140838381 95 0x56453105aa10 WARN nvcodec plugin.c94plugin_init: Failed to load cuda library
00000.163465325 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0x120000: 'AVR (Audio Visual Research)' is not mapped
00000.163480955 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0x180000: 'CAF (Apple Core Audio File)' is not mapped
00000.163485365 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0x100000: 'HTK (HMM Tool Kit)' is not mapped
00000.163490285 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0xc0000: 'MAT4 (GNU Octave 2.0 / Matlab 4.2)' is not mapped
00000.163494125 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0xd0000: 'MAT5 (GNU Octave 2.1 / Matlab 5.0)' is not mapped
00000.163499045 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0x210000: 'MPC (Akai MPC 2k)' is not mapped
00000.163502135 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0x230000: 'MPEG-1/2 Audio' is not mapped
00000.163506495 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0xe0000: 'PVF (Portable Voice Format)' is not mapped
00000.163511895 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0x160000: 'SD2 (Sound Designer II)' is not mapped
00000.163518675 95 0x56453105aa10 WARN default gstsfelement.c97gst_sf_create_audio_template_caps: format 0x190000: 'WVE (Psion Series 3)' is not mapped
2026-02-16T152802.191Z DEBUG egress gstreamer/bin.go:66 adding src audio to pipeline {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152802.192Z DEBUG egress gstreamer/bin.go:72 adding sink file to pipeline {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152802.192Z DEBUG egress pipeline/controller.go:215 waiting for start signal {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152817.474Z INFO egress source/web.go:298 chrome: END_RECORDING {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152817.474Z DEBUG egress gstreamer/state.go:75 pipeline state building -> stopping {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152817.474Z DEBUG egress source/web.go:125 closing chrome {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152817.474Z DEBUG egress gstreamer/state.go:75 pipeline state stopping -> finished {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152817.520Z DEBUG egress source/web.go:131 closing X display {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152817.525Z DEBUG egress source/web.go:138 unloading pulse module {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW"}
2026-02-16T152817.530Z DEBUG egress sink/file.go:73 removing temporary directory {"nodeID": "NE_iofUiWxBotVd", "handlerID": "EGH_X4poTFHxwbB2", "clusterID": "", "egressID": "EG_JvTieZd74TmW", "path": "/home/egress/tmp/EG_JvTieZd74TmW/"}
2026-02-16T152817.537Z INFO egress service/io.go:88 egress updated {"nodeID": "NE_iofUiWxBotVd", "clusterID": "", "egressID": "EG_JvTieZd74TmW", "requestType": "room_composite", "outputType": "file", "status": "EGRESS_ABORTED"}
t
tall-airline-16756
02/17/2026, 2:05 PMThanks — this is very helpful. I reviewed the logs you provided and have a clear diagnosis plus targeted next steps.
Short summary (what happened)
• Egress started and the GStreamer pipeline was created for an audio-only
.oggEND_RECORDINGEGRESS_ABORTEDoutbound-call-20260216-152756-ebf04b0eroom_finishedbackupStorageUsed: true--enable-logging --v=1 --no-sandbox --disable-gpustorage.objects.createlivekit-recordings-nonprodgsutil ls <gs://livekit-recordings-nonprod>b
broad-teacher-94403
02/17/2026, 2:12 PMGive the following:
If you’d like I can help with any of these steps — for example:
• Example curl/openssl commands you can paste into the pod,
• A short snippet to enable Chrome verbose logging in your egress deployment,
•
t
tall-airline-16756
02/17/2026, 2:12 PMThanks — below are concise, copy-pasteable checks you can run inside the egress pod and a small deployment snippet to enable verbose Chrome logging for the egress compositor.
Run these inside the egress container/pod (adjust pod name or exec method as needed)
1) Verify HTTPS/TLS and certificate chain
• Test HTTPS request and show TLS details:
◦ curl (display cert info and full verbose output)
▪︎ curl -vI https://devservices-colo-west-livekit.corp.cvscaremark.com
◦ openssl (show certificate chain and subject/issuer)
▪︎ openssl s_client -connect devservices-colo-west-livekit.corp.cvscaremark.com:443 -showcerts
◦ Check certificate expiration:
▪︎ echo | openssl s_client -connect devservices-colo-west-livekit.corp.cvscaremark.com:443 2>/dev/null | openssl x509 -noout -dates
2) Confirm TLS trust from the pod (detect untrusted CA)
• Use curl to check if the pod trusts the cert:
◦ curl --cacert /etc/ssl/certs/ca-certificates.crt -vI https://devservices-colo-west-livekit.corp.cvscaremark.com
◦ If you don’t have a single CA bundle path, just run:
▪︎ curl -vI https://devservices-colo-west-livekit.corp.cvscaremark.com
◦ If curl fails with "certificate signed by unknown authority", Chrome will also refuse the WSS connection in secure mode.
3) Check connectivity to WSS (basic TCP test)
• Validate you can open a TCP connection to port 443:
◦ nc -vz devservices-colo-west-livekit.corp.cvscaremark.com 443
◦ Or using bash + /dev/tcp:
▪︎ (echo > /dev/tcp/devservices-colo-west-livekit.corp.cvscaremark.com/443) && echo "ok" || echo "connect failed"
4) Quick WebRTC/network diagnostics (optional)
• Check DNS resolution:
◦ nslookup devservices-colo-west-livekit.corp.cvscaremark.com
• Show routing to host:
◦ traceroute devservices-colo-west-livekit.corp.cvscaremark.com
What to look for in results
• openssl shows the certificate chain and issuer. If the cert is issued by an internal CA, note the issuer name.
• curl error "certificate signed by unknown authority" or similar → container does not trust the cert.
• nc/connect failure → firewall/network issue prevents connection to the LiveKit server.
Snippet to enable verbose Chrome/WebRTC logging in the egress deployment
• Add or update the egress container args to include Chrome flags that increase logging. Below is a minimal Kubernetes container snippet showing how to pass Chrome args. Adapt to your egress helm/kustomize values or manifest.
Copy code
# example container args for livekit-egress pod (edit per your deployment)
containers:
- name: egress
image: livekit/egress:1.8.6
args:
- "--web"
- "--templates-host=localhost:7980"
# Increase Chrome logging verbosity and enable logging to stdout
- "--chrome-args=--enable-logging --v=1 --no-sandbox --disable-gpu --enable-webrtc-logging=1 --enable-logging=stderr"
env:
# keep your existing env vars here
- name: LIVEKIT_API_KEY
valueFrom:
secretKeyRef:
name: livekit-secrets
key: api_key--v=2--v=3END_RECORDINGb
broad-teacher-94403
02/17/2026, 2:17 PMAlso, call ends as soon as the we get the trigger such as bye, thank you etc from the user.
Code:
Copy code
async def start_recording(ctx: agents.JobContext):
"""
Start audio recording for the room using LiveKit Egress.
Records audio-only to GCS bucket with Workload Identity.
Controlled by recordAudio flag in room metadata (passed from controller via RECORD_AUDIO env var).
"""
# Parse room metadata (could be JSON string or dict)
room_metadata = {}
try:
if ctx.room.metadata:
if isinstance(ctx.room.metadata, str):
room_metadata = json.loads(ctx.room.metadata)
elif isinstance(ctx.room.metadata, dict):
room_metadata = ctx.room.metadata
except Exception as e:
error(f"Failed to parse room metadata for recording: {e}")
# Default to recording enabled if parsing fails
room_metadata = {"recordAudio": True}
# signal(f"Room metadata: {room_metadata}")
# Check if recording is enabled via room metadata
record_audio = room_metadata.get("recordAudio", True)
if not record_audio:
signal("Audio recording disabled via room metadata (RECORD_AUDIO env var)")
return
try:
# Get GCS bucket from config
gcs_bucket = CONFIG.get("cvs.gcp-bucket")
if not gcs_bucket:
error("GCS bucket not configured in application.yaml (cvs.gcp-bucket)")
return
signal(f"Starting audio recording to GCS bucket: {gcs_bucket}")
campaign_id = room_metadata.get("campaignId")
session_id = room_metadata.get("sessionId")
# Validate required fields
if not campaign_id:
error("Cannot start recording: campaignId missing from room metadata")
return
if not session_id:
error("Cannot start recording: sessionId missing from room metadata")
return
filepath = f"recordings/{campaign_id}/{session_id}.ogg"
# Create egress request for audio-only recording
# Using Workload Identity - no credentials needed
req = api.RoomCompositeEgressRequest(
room_name=ctx.room.name,
audio_only=True, # Audio only for voice calls
file_outputs=[
api.EncodedFileOutput(
file_type=api.EncodedFileType.OGG, # OGG format for audio
filepath=filepath,
gcp=api.GCPUpload(
bucket=gcs_bucket,
# No credentials - Workload Identity handles authentication
),
)
],
)
# Start recording
lkapi = api.LiveKitAPI()
egress_info = await lkapi.egress.start_room_composite_egress(req)
await lkapi.aclose()
signal(f"Recording started successfully: egress_id={egress_info.egress_id}, file={filepath}")
except Exception as e:
error(f"Failed to start audio recording: {e}")
# Don't fail the agent if recording fails
import traceback
error(traceback.format_exc()) Copy code
async def hangup_call(self, delay_seconds: float = 1.0):
"""
Hang up the call by deleting the room.
This ends the call for all participants.
Args:
delay_seconds: Time to wait before hanging up to allow TTS playout to complete.
Default is 3 seconds which should be sufficient for most goodbye messages.
"""
ctx = get_job_context()
if ctx is None:
error("Cannot hang up: Not running in a job context")
return
try:
# Wait for TTS playout to complete
# We add a delay to ensure the final message is heard by the user
signal(f"Waiting {delay_seconds}s for final message playout before hanging up...")
await asyncio.sleep(delay_seconds)
signal(f"Hanging up call by deleting room: {ctx.room.name}")
await ctx.api.room.delete_room(
api.DeleteRoomRequest(
room=ctx.room.name,
)
)
signal("Call hung up successfully")
except Exception as e:
error(f"Error hanging up call: {e}")t
tall-airline-16756
02/17/2026, 2:17 PMThanks — this is very clear and helpful. I reviewed your code and logs and can confirm the likely cause and a few clean ways to fix it.
Short answer
• Deleting the room immediately after the user says “bye/thank you” causes the hidden Chrome compositor to disconnect, which triggers
END_RECORDINGEGRESS_ABORTEDEND_RECORDINGegress_startedegress_endedegress_idCOMPLETEDFAILEDegress_endedTrackCompositeTrackEgressegress_endedegress_endedegress_endedegress_endedb
broad-teacher-94403
02/17/2026, 6:10 PMmy current egress logs, what can be fixed:
Copy code
2026-02-17T18:04:23.954Z INFO egress server/server_rpc.go:75 request validated {"nodeID": "NE_KcJ455zyJKrD", "clusterID": "", "egressID": "EG_fss3GntECB9K", "requestType": "room_composite", "outputType": "file", "room": "outbound-call-20260217-180419-59679a22", "request": {"RoomComposite":{"room_name":"outbound-call-20260217-180419-59679a22","audio_only":true,"Output":null,"Options":null,"file_outputs":[{"file_type":2,"filepath":"recordings/1251a2b4-84f5-4793-a2e9-19b8fb8c25e5/70963170-7d71-4471-9331-790725734da5.ogg","Output":{"Gcp":{"bucket":"livekit-recordings-nonprod"}}}]}}}
2026-02-17T18:04:23.979Z DEBUG egress server/main.go:177 handler launched {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:23.980Z INFO egress redis/redis.go:142 connecting to redis {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K", "simple": true, "addr": "<http://redis-14764.redis-dev-west.corp.cvscaremark.com:14764|redis-14764.redis-dev-west.corp.cvscaremark.com:14764>"}
2026-02-17T18:04:24.126Z DEBUG egress source/web.go:159 creating pulse sink {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:24.132Z DEBUG egress source/web.go:183 creating X display {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K", "display": ":162038793", "dims": "1280x720x24"}
2026-02-17T18:04:24.133Z DEBUG egress source/web.go:214 launching chrome {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K", "url": "<http://localhost:7980/?layout=&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3NzE0Mzc4NjMsImlzcyI6IkFQSUtFWTY4YjBlMGMyZTA2YWVhNTEiLCJraW5kIjoiZWdyZXNzIiwibmJmIjoxNzcxMzUxNDYzLCJzdWIiOiJFR19mc3MzR250RUNCOUsiLCJ2aWRlbyI6eyJjYW5QdWJsaXNoIjpmYWxzZSwiY2FuUHVibGlzaERhdGEiOmZhbHNlLCJjYW5TdWJzY3JpYmUiOnRydWUsImhpZGRlbiI6dHJ1ZSwicmVjb3JkZXIiOnRydWUsInJvb20iOiJvdXRib3VuZC1jYWxsLTIwMjYwMjE3LTE4MDQxOS01OTY3OWEyMiIsInJvb21Kb2luIjp0cnVlfX0.0wJDGNO1troOvdZssOzk47ryFVFiO6-bvu8qqzIgVOc&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com>", "sandbox": false, "insecure": true}
0:00:00.147244263 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00m cudaloader gstcudaloader.c:169:gst_cuda_load_library:[00m Could not open library libcuda.so.1, libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.147266143 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00m nvcodec plugin.c:94:plugin_init:[00m Failed to load cuda library
0:00:00.171134656 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0x120000: 'AVR (Audio Visual Research)' is not mapped
0:00:00.171152126 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0x180000: 'CAF (Apple Core Audio File)' is not mapped
0:00:00.171156686 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0x100000: 'HTK (HMM Tool Kit)' is not mapped
0:00:00.171161406 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0xc0000: 'MAT4 (GNU Octave 2.0 / Matlab 4.2)' is not mapped
0:00:00.171165206 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0xd0000: 'MAT5 (GNU Octave 2.1 / Matlab 5.0)' is not mapped
0:00:00.171168476 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0x210000: 'MPC (Akai MPC 2k)' is not mapped
0:00:00.171171606 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0x230000: 'MPEG-1/2 Audio' is not mapped
0:00:00.171175466 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0xe0000: 'PVF (Portable Voice Format)' is not mapped
0:00:00.171180006 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0x160000: 'SD2 (Sound Designer II)' is not mapped
0:00:00.171185426 [36m 89[00m 0x555cb2c81a10 [33;01mWARN [00m [00;04m default gstsfelement.c:97:gst_sf_create_audio_template_caps:[00m format 0x190000: 'WVE (Psion Series 3)' is not mapped
2026-02-17T18:04:24.985Z DEBUG egress gstreamer/bin.go:66 adding src audio to pipeline {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:24.986Z DEBUG egress gstreamer/bin.go:72 adding sink file to pipeline {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:24.986Z DEBUG egress pipeline/controller.go:215 waiting for start signal {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:40.276Z INFO egress source/web.go:298 chrome: END_RECORDING {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:40.276Z DEBUG egress gstreamer/state.go:75 pipeline state building -> stopping {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:40.276Z DEBUG egress gstreamer/state.go:75 pipeline state stopping -> finished {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:40.276Z DEBUG egress source/web.go:125 closing chrome {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:40.318Z DEBUG egress source/web.go:131 closing X display {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:40.324Z DEBUG egress source/web.go:138 unloading pulse module {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K"}
2026-02-17T18:04:40.328Z DEBUG egress sink/file.go:73 removing temporary directory {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_fMKZ2nDSR5zj", "clusterID": "", "egressID": "EG_fss3GntECB9K", "path": "/home/egress/tmp/EG_fss3GntECB9K/"}
2026-02-17T18:04:40.337Z INFO egress service/io.go:88 egress updated {"nodeID": "NE_KcJ455zyJKrD", "clusterID": "", "egressID": "EG_fss3GntECB9K", "requestType": "room_composite", "outputType": "file", "status": "EGRESS_ABORTED"}
2026-02-17T18:07:56.905Z DEBUG egress stats/monitor.go:151 cpu check {"nodeID": "NE_KcJ455zyJKrD", "clusterID": "", "total": 768, "available": 768, "pending": 0, "used": 0, "activeRequests": 0, "activeWeb": 0, "required": 1, "canAccept": true}
2026-02-17T18:07:57.411Z INFO egress server/server_rpc.go:58 request received {"nodeID": "NE_KcJ455zyJKrD", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:07:57.415Z INFO egress server/server_rpc.go:75 request validated {"nodeID": "NE_KcJ455zyJKrD", "clusterID": "", "egressID": "EG_r7PojiGNxBP6", "requestType": "room_composite", "outputType": "file", "room": "outbound-call-20260217-233750-49d6ba5e", "request": {"RoomComposite":{"room_name":"outbound-call-20260217-233750-49d6ba5e","audio_only":true,"Output":null,"Options":null,"file_outputs":[{"file_type":2,"filepath":"recordings/1251a2b4-84f5-4793-a2e9-19b8fb8c25e5/58948066-774c-4d26-b573-93953ce0092f.ogg","Output":{"Gcp":{"bucket":"livekit-recordings-nonprod"}}}]}}}
2026-02-17T18:07:57.441Z DEBUG egress server/main.go:177 handler launched {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:07:57.441Z INFO egress redis/redis.go:142 connecting to redis {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6", "simple": true, "addr": "<http://redis-14764.redis-dev-west.corp.cvscaremark.com:14764|redis-14764.redis-dev-west.corp.cvscaremark.com:14764>"}
2026-02-17T18:07:57.517Z DEBUG egress source/web.go:159 creating pulse sink {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:07:57.522Z DEBUG egress source/web.go:183 creating X display {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6", "display": ":1921563997", "dims": "1280x720x24"}
2026-02-17T18:07:57.523Z DEBUG egress source/web.go:214 launching chrome {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6", "url": "<http://localhost:7980/?layout=&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3NzE0MzgwNzcsImlzcyI6IkFQSUtFWTY4YjBlMGMyZTA2YWVhNTEiLCJraW5kIjoiZWdyZXNzIiwibmJmIjoxNzcxMzUxNjc3LCJzdWIiOiJFR19yN1BvamlHTnhCUDYiLCJ2aWRlbyI6eyJjYW5QdWJsaXNoIjpmYWxzZSwiY2FuUHVibGlzaERhdGEiOmZhbHNlLCJjYW5TdWJzY3JpYmUiOnRydWUsImhpZGRlbiI6dHJ1ZSwicmVjb3JkZXIiOnRydWUsInJvb20iOiJvdXRib3VuZC1jYWxsLTIwMjYwMjE3LTIzMzc1MC00OWQ2YmE1ZSIsInJvb21Kb2luIjp0cnVlfX0.Q5cStWhQas8tB9wjoiMBkurL8fvSxRteWeuUNELQBfo&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com>", "sandbox": false, "insecure": true}
2026-02-17T18:07:57.924Z DEBUG egress gstreamer/bin.go:66 adding src audio to pipeline {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:07:57.925Z DEBUG egress gstreamer/bin.go:72 adding sink file to pipeline {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:07:57.925Z DEBUG egress pipeline/controller.go:215 waiting for start signal {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:08:13.239Z INFO egress source/web.go:298 chrome: END_RECORDING {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:08:13.239Z DEBUG egress gstreamer/state.go:75 pipeline state building -> stopping {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:08:13.240Z DEBUG egress source/web.go:125 closing chrome {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:08:13.240Z DEBUG egress gstreamer/state.go:75 pipeline state stopping -> finished {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:08:13.283Z DEBUG egress source/web.go:131 closing X display {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:08:13.286Z DEBUG egress source/web.go:138 unloading pulse module {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6"}
2026-02-17T18:08:13.291Z DEBUG egress sink/file.go:73 removing temporary directory {"nodeID": "NE_KcJ455zyJKrD", "handlerID": "EGH_z4anGPzpAij7", "clusterID": "", "egressID": "EG_r7PojiGNxBP6", "path": "/home/egress/tmp/EG_r7PojiGNxBP6/"}
2026-02-17T18:08:13.297Z INFO egress service/io.go:88 egress updated {"nodeID": "NE_KcJ455zyJKrD", "clusterID": "", "egressID": "EG_r7PojiGNxBP6", "requestType": "room_composite", "outputType": "file", "status": "EGRESS_ABORTED"}t
tall-airline-16756
02/17/2026, 6:10 PMThanks — this is clear and you’ve provided everything needed to pinpoint the problem. Below is a focused analysis of your latest logs, what can be fixed, and concrete, copy‑pasteable changes you can make in your code and deployment to eliminate the EGRESS_ABORTED behavior.
What the logs show (short)
• Egress is created and starts normally (room_composite, audio_only -> .ogg in your GCS bucket).
• Chrome (the headless compositor) posts
chrome: END_RECORDINGEGRESS_ABORTEDegress_ended Copy code
egress_info = await lkapi.egress.start_room_composite_egress(req)
egress_id = egress_info.egress_id
# persist egress_id somewhere accessible at hangup time:
# - ctx.room.metadata (if safe), or database keyed by room name/session id
await save_egress_id_for_room(ctx.room.name, egress_id) Copy code
egress_id = await load_egress_id_for_room(ctx.room.name)
if egress_id:
await lkapi.egress.stop_room_composite_egress(api.StopEgressRequest(egress_id=egress_id))
# Poll status until completed/failed/aborted
while True:
status = await lkapi.egress.get_egress_status(api.GetEgressRequest(egress_id=egress_id))
if status.state in ["COMPLETED","FAILED","ABORTED"]:
break
await asyncio.sleep(1)
# Now safe to delete the room
await ctx.api.room.delete_room(api.DeleteRoomRequest(room=ctx.room.name))egress_endedegress_ended Copy code
POST /webhook/egress
payload = request.json()
if payload["event"] == "egress_ended":
egress_id = payload["egress_id"]
# correlate to room/session and then delete the room if required
delete_room_for_egress(egress_id)insecure=trueegress_endedb
broad-teacher-94403
02/17/2026, 6:16 PM• Provide a ready-to-drop-in patch for your start_recording and hangup_call functions (with exact calls to the LiveKit Python client you are using).
• Or provide a webhook handler example (flask/fastapi) to trigger room deletion on
egress_endedt
tall-airline-16756
02/17/2026, 6:17 PMThanks — below are two ready-to-drop-in options you can use immediately. Both prevent premature egress abortion by stopping/waiting for egress to finish before deleting the room.
Option A: Patch your start_recording and hangup_call (stop-and-poll)
• What this does: start_recording saves the returned egress_id; hangup_call stops the egress, polls until it ends, then deletes the room.
• Use this if you prefer keeping logic inside the agent/job.
Notes:
• Adjust imports and LiveKit client initialization to match your environment.
• I use the LiveKit Python client calls you referenced (async style). Replace helper persistence functions (save/load) with your DB or job-context storage.
Code (drop-in):
Copy code
# imports (adjust based on your package)
import asyncio
import json
from livekit import api # adapt to your SDK import path
from livekit.api import LiveKitAPI
# assume get_job_context, signal, error exist in your environment
# create a singleton LiveKit API client (configure env or pass params)
lkapi = LiveKitAPI()
# simple persistence helpers - replace with your DB or job context storage
_egress_store = {} # in-memory for example; replace in prod
async def save_egress_id_for_room(room_name: str, egress_id: str):
_egress_store[room_name] = egress_id
async def load_egress_id_for_room(room_name: str):
return _egress_store.get(room_name)
async def delete_egress_record(room_name: str):
_egress_store.pop(room_name, None)
# Patched start_recording (based on your function)
async def start_recording(ctx):
try:
# parse metadata same as you already do...
room_metadata = {}
try:
if ctx.room.metadata:
if isinstance(ctx.room.metadata, str):
room_metadata = json.loads(ctx.room.metadata)
elif isinstance(ctx.room.metadata, dict):
room_metadata = ctx.room.metadata
except Exception as e:
error(f"Failed to parse room metadata for recording: {e}")
room_metadata = {"recordAudio": True}
record_audio = room_metadata.get("recordAudio", True)
if not record_audio:
signal("Audio recording disabled via room metadata")
return
gcs_bucket = CONFIG.get("cvs.gcp-bucket")
if not gcs_bucket:
error("GCS bucket not configured in application.yaml (cvs.gcp-bucket)")
return
campaign_id = room_metadata.get("campaignId")
session_id = room_metadata.get("sessionId")
if not campaign_id or not session_id:
error("campaignId or sessionId missing from room metadata")
return
filepath = f"recordings/{campaign_id}/{session_id}.ogg"
req = api.RoomCompositeEgressRequest(
room_name=ctx.room.name,
audio_only=True,
file_outputs=[
api.EncodedFileOutput(
file_type=api.EncodedFileType.OGG,
filepath=filepath,
gcp=api.GCPUpload(bucket=gcs_bucket),
)
],
)
egress_info = await lkapi.egress.start_room_composite_egress(req)
egress_id = egress_info.egress_id
await save_egress_id_for_room(ctx.room.name, egress_id)
signal(f"Recording started: egress_id={egress_id}, file={filepath}")
return egress_id
except Exception as e:
error(f"Failed to start audio recording: {e}")
import traceback
error(traceback.format_exc())
# Patched hangup_call: stop egress, poll until it ends, then delete room
async def hangup_call(self, delay_seconds: float = 1.0):
ctx = get_job_context()
if ctx is None:
error("Cannot hang up: Not running in a job context")
return
try:
signal(f"Waiting {delay_seconds}s for final message playout before stopping recording...")
await asyncio.sleep(delay_seconds)
egress_id = await load_egress_id_for_room(ctx.room.name)
if egress_id:
signal(f"Stopping egress: {egress_id}")
try:
await lkapi.egress.stop_egress(api.StopEgressRequest(egress_id=egress_id))
except Exception as e:
# stop may fail if already stopping; log and continue
error(f"Stop egress call failed (may already be stopping): {e}")
# Poll for egress end (timeout in seconds)
timeout = 60
poll_interval = 1
waited = 0
while waited < timeout:
try:
status_resp = await lkapi.egress.get_egress_status(api.GetEgressRequest(egress_id=egress_id))
state = getattr(status_resp, "state", None) or getattr(status_resp, "status", None)
# state may be enum/int depending on SDK; adapt checks if needed
if state and str(state).upper() in ("COMPLETED", "FAILED", "ABORTED"):
signal(f"Egress ended with state={state}")
break
except Exception as e:
error(f"Error polling egress status: {e}")
await asyncio.sleep(poll_interval)
waited += poll_interval
else:
error("Timeout waiting for egress to finish; proceeding to delete room anyway")
await delete_egress_record(ctx.room.name)
else:
signal("No egress_id found for room; proceeding to delete room")
signal(f"Hanging up call by deleting room: {ctx.room.name}")
await ctx.api.room.delete_room(api.DeleteRoomRequest(room=ctx.room.name))
signal("Call hung up successfully")
except Exception as e:
error(f"Error hanging up call: {e}") Copy code
from fastapi import FastAPI, Request, HTTPException
import os
from livekit.api import WebhookReceiver # adapt to your SDK path
from livekit import api, LiveKitAPI
app = FastAPI()
WEBHOOK_SECRET = os.getenv("LIVEKIT_API_SECRET") # same secret set in LiveKit
webhook_receiver = WebhookReceiver(api_secret=WEBHOOK_SECRET)
# LiveKit API client for room deletion
lkapi = LiveKitAPI()
@app.post("/webhook")
async def handle_webhook(request: Request):
body = await request.body()
auth_header = request.headers.get("Authorization")
try:
event = webhook_receiver.receive(body, auth_header)
except Exception as e:
raise HTTPException(status_code=401, detail="Invalid webhook signature")
# event structure depends on SDK; adapt field names if necessary
if getattr(event, "event", "") == "egress_ended":
egress_info = getattr(event, "egress_info", None) or {}
room_name = egress_info.get("room_name") if isinstance(egress_info, dict) else getattr(egress_info, "room_name", None)
status = egress_info.get("status") if isinstance(egress_info, dict) else getattr(egress_info, "status", None)
print(f"Egress ended for room={room_name}, status={status}")
# Only delete the room after egress completed/failed/aborted
if status and str(status).upper() in ("EGRESS_COMPLETE", "COMPLETED", "FAILED", "ABORTED"):
try:
await lkapi.room.delete_room(api.DeleteRoomRequest(room=room_name))
print(f"Deleted room {room_name} after egress ended")
except Exception as e:
print(f"Failed to delete room {room_name}: {e}")
return {"ok": True} Copy code
from flask import Flask, request
import os, asyncio
from livekit.api import WebhookReceiver
from livekit import LiveKitAPI, api
app = Flask(__name__)
webhook_receiver = WebhookReceiver(api_secret=os.getenv("LIVEKIT_API_SECRET"))
lkapi = LiveKitAPI()
@app.route("/webhook", methods=["POST"])
def webhook():
body = request.get_data()
auth_header = request.headers.get("Authorization")
try:
event = webhook_receiver.receive(body, auth_header)
except Exception:
return "Unauthorized", 401
if getattr(event, "event", "") == "egress_ended":
egress_info = getattr(event, "egress_info", None) or {}
room_name = egress_info.get("room_name") if isinstance(egress_info, dict) else getattr(egress_info, "room_name", None)
status = egress_info.get("status") if isinstance(egress_info, dict) else getattr(egress_info, "status", None)
if status and str(status).upper() in ("EGRESS_COMPLETE", "COMPLETED", "FAILED", "ABORTED"):
# use asyncio.run to call async delete in sync Flask handler
asyncio.run(lkapi.room.delete_room(api.DeleteRoomRequest(room=room_name)))
return "", 200b
broad-teacher-94403
02/17/2026, 6:33 PM000022 INFO cvs_event_logger CvsEventEnum.SIGNAL: Stopping egress before room deletion:
EG_hSF77CXoXUxF
INFO cvs_event_logger CvsEventEnum.SIGNAL: Stopping egress EG_hSF77CXoXUxF ...
000026 INFO cvs_event_logger CvsEventEnum.ERROR: Stop egress call failed (may already be stopped):
TwirpError(code=failed_precondition, message=egress with status
EGRESS_ABORTED cannot be stopped, status=412)
INFO cvs_event_logger CvsEventEnum.SIGNAL: Egress EG_hSF77CXoXUxF status: 5 (elapsed 0s)
INFO cvs_event_logger CvsEventEnum.ERROR: Egress EG_hSF77CXoXUxF ended with status: 5
t
tall-airline-16756
02/17/2026, 6:33 PMThanks — this is clear and the logs pinpoint the behavior. I’ll explain what’s happening, why the Stop call fails with a 412, and give a small, safe patch you can drop into your hangup flow so it handles this case cleanly and avoids race conditions.
Summary of what your logs show
• You call Stop on the egress: "Stopping egress EG_hSF77CXoXUxF ..."
• LiveKit responds: TwirpError(code=failed_precondition, message=egress with status EGRESS_ABORTED cannot be stopped, status=412)
• The egress is already in state EGRESS_ABORTED when you attempted to stop it.
What that means
• A Stop request only works when egress is still running or in a stoppable state. If the egress already transitioned to EGRESS_ABORTED (or COMPLETED/FAILED), Stop is rejected with a failed_precondition/412 because there’s nothing to stop.
• In practice this happens when Chrome has already sent END_RECORDING (or the worker aborted for another reason) before your Stop call arrives — a race between the compositor and your control flow.
What to do (short)
• Check egress state before calling Stop and handle the 412 gracefully.
• Prefer webhook-based cleanup (delete room after receiving egress_ended).
• If polling or Stop is used, add robust handling for already-ended states so hangup never fails catastrophically.
Small, ready-to-drop-in patch for hangup_call
• This keeps your existing flow but:
◦ Loads saved egress_id (if present),
◦ Fetches current egress state,
◦ Calls Stop only if egress is in a stoppable state,
◦ If Stop returns a 412 or the egress is already ended, it logs and continues to delete the room.
• Adapt the LiveKit client calls to your exact SDK if names differ.
Patch (conceptual, matches your async style):
Copy code
async def hangup_call(self, delay_seconds: float = 1.0):
ctx = get_job_context()
if ctx is None:
error("Cannot hang up: Not running in a job context")
return
try:
signal(f"Waiting {delay_seconds}s for final message playout before hanging up...")
await asyncio.sleep(delay_seconds)
# load previously saved egress_id for this room (persisted when starting)
egress_id = await load_egress_id_for_room(ctx.room.name)
if egress_id:
signal(f"Stopping egress before room deletion: {egress_id}")
# First, fetch current egress status to avoid Stop races
try:
status_resp = await lkapi.egress.get_egress_status(
api.GetEgressRequest(egress_id=egress_id)
)
# adapt to how your SDK returns state; here we handle numeric/str states
state = getattr(status_resp, "state", None) or getattr(status_resp, "status", None)
signal(f"Egress {egress_id} status: {state}")
except Exception as e:
error(f"Failed to fetch egress status for {egress_id}: {e}")
state = None
# Only attempt Stop if egress appears to be running/stoppable
stoppable_states = ("STARTED", "RUNNING", "IN_PROGRESS", 1, 2) # adapt enums as needed
if state and str(state).upper() not in ("COMPLETED", "FAILED", "ABORTED"):
try:
await lkapi.egress.stop_egress(api.StopEgressRequest(egress_id=egress_id))
signal(f"Stop egress call sent for {egress_id}")
except Exception as e:
# If it's a TwirpError 412 (failed_precondition), egress already ended — log and continue
err_msg = str(e)
if "failed_precondition" in err_msg or "EGRESS_ABORTED" in err_msg or "status=412" in err_msg:
error(f"Stop egress call failed (may already be stopped): {e}")
else:
# other errors should be surfaced
error(f"Stop egress call error: {e}")
else:
signal(f"Egress {egress_id} already ended or non-stoppable (state={state}), skipping Stop")
# Optionally poll until egress really ends (short timeout), otherwise proceed
timeout = 30
poll_interval = 1
waited = 0
while waited < timeout:
try:
st = await lkapi.egress.get_egress_status(api.GetEgressRequest(egress_id=egress_id))
cur_state = getattr(st, "state", None) or getattr(st, "status", None)
if cur_state and str(cur_state).upper() in ("COMPLETED", "FAILED", "ABORTED"):
signal(f"Egress {egress_id} ended with state: {cur_state} (elapsed {waited}s)")
break
except Exception as e:
error(f"Error polling egress status: {e}")
await asyncio.sleep(poll_interval)
waited += poll_interval
else:
error("Timeout waiting for egress to finish; proceeding to delete room")
# cleanup local record
await delete_egress_record(ctx.room.name)
else:
signal("No egress_id found for room; proceeding to delete room")
signal(f"Hanging up call by deleting room: {ctx.room.name}")
await ctx.api.room.delete_room(api.DeleteRoomRequest(room=ctx.room.name))
signal("Call hung up successfully")
except Exception as e:
error(f"Error hanging up call: {e}")egress_ended Copy code
@app.post("/webhook")
async def handle_webhook(req: Request):
event = verify_and_parse(req) # verify signature
if event["type"] == "egress_ended":
egress_id = event["egress"]["id"]
room_name = event["egress"]["room_name"]
status = event["egress"]["status"]
# only delete once egress is ended
if status.upper() in ("COMPLETED", "FAILED", "ABORTED"):
await delete_room(room_name)get_egress_statusstop_egressb
broad-teacher-94403
02/18/2026, 5:42 AMWhat is:
Copy code- "--templates-host=localhost:7980"
t
tall-airline-16756
02/18/2026, 5:42 AMGood question — happy to clarify.
What
--templates-host=localhost:7980localhost:7980--templates-hosttemplates.my-domain:7980--templates-host=127.0.0.1:7980--templates-host=templates-service.namespace:7980b
broad-teacher-94403
02/18/2026, 5:46 AMWhich would you like to check first — curling the template URL from the egress pod or inspecting the template page Chrome is launching (from your logs)?
t
tall-airline-16756
02/18/2026, 5:46 AMGreat — I can help with both checks. I recommend starting with a quick curl from the egress pod to confirm the template server is reachable, then inspecting the exact template URL Chrome launches (from your logs) to validate the token/WSS query params and any errors returned by the template page.
Below are concise, copy‑pasteable steps for each check and what to look for.
1) Curl the template server from the egress pod (quick reachability + HTML)
• Exec into your egress pod/container, then run:
◦ curl -v "http://localhost:7980/"
◦ curl -v "http://localhost:7980/?token=SHORTENED_TOKEN" # use the same query path but redact long token
• If the template server is running on another host/sidecar, replace localhost with that host.
What to look for:
• HTTP 200 and an HTML page body → template server is up.
• HTTP 4xx/5xx or connection refused → template server not reachable or error.
• Verbose output (curl -v) will show redirects or connection issues.
2) Curl the exact template URL Chrome launches (validate token & WSS param)
• From your logs you have a URL similar to:
http://localhost:7980/?layout=&token=...&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com
• Copy the URL (you can remove the long JWT token or replace it with a short placeholder) and run:
◦ curl -v --max-time 10 "http://localhost:7980/?layout=&token=REDACTED&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com"
• Or fetch the full page and save for inspection:
◦ curl -s "http://localhost:7980/?token=REDACTED&url=..." -o template.html
◦ Then open template.html or cat it to inspect.
What to look for:
• The HTML should reference your client JS and include a script that reads the
tokenurlinsecure=true/falseurl=b
broad-teacher-94403
02/18/2026, 5:53 AM Copy code
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="/favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="Recording interface for LiveKit"/><link rel="apple-touch-icon" href="/logo.png"/><link rel="manifest" href="/manifest.json"/><title>LiveKit Egress</title><script defer="defer" src="/static/js/main.30791970.js"></script><link href="/static/css/main.b0672d20.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>t
tall-airline-16756
02/18/2026, 5:53 AMThanks — that HTML means the template server is responding correctly. Good news: the template page is served and the browser compositor can load the app. The remaining failures are likely happening when Chrome tries to use the query params (token + WSS url) to join the LiveKit room.
Next checks I recommend (in order)
1) Inspect the exact launch URL from your egress logs
• Find the log line where egress launches Chrome (you previously shared examples). It looks like:
http://localhost:7980/?layout=&token=<JWT>&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com
• Copy that full URL (redact the JWT if you paste here) and confirm the
url=b
broad-teacher-94403
02/18/2026, 5:56 AMMy rooms are only for audio recording. So the setup is fine right?
Copy code
2026-02-18T05:50:14.905Z DEBUG egress source/web.go:214 launching chrome {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4", "url": "<http://localhost:7980/?layout=&token={REDACTED}&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com>", "sandbox": false, "insecure": true}t
tall-airline-16756
02/18/2026, 5:56 AMThanks — and good question. Based on the log line you pasted, your template server and the Chrome launch call look correct for an audio-only Room Composite egress. A few points and a short checklist to confirm everything is truly OK and to stop the quick aborts you’ve been seeing.
What the log shows (good)
• Chrome is launched with the template URL:
http://localhost:7980/?layout=&token={REDACTED}&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com
•
insecure=trueb
broad-teacher-94403
02/18/2026, 5:58 AM Copy code
egress@livekit-egress-66d5fc5f7d-9bsfm:/$ curl -vI <https://devservices-colo-west-livekit.corp.cvscaremark.com>
* processing: <https://devservices-colo-west-livekit.corp.cvscaremark.com>
* Trying 10.20.46.62:443...
* Connected to <http://devservices-colo-west-livekit.corp.cvscaremark.com|devservices-colo-west-livekit.corp.cvscaremark.com> (10.20.46.62) port 443
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted h2
* Server certificate:
* subject: C=US; ST=Rhode Island; L=Woonsocket; O=CVS Pharmacy Inc; CN=<http://devservices-colo-west.corp.cvscaremark.com|devservices-colo-west.corp.cvscaremark.com>
* start date: Feb 12 00:00:00 2026 GMT
* expire date: Feb 12 23:59:59 2027 GMT
* subjectAltName: host "<http://devservices-colo-west-livekit.corp.cvscaremark.com|devservices-colo-west-livekit.corp.cvscaremark.com>" matched cert's "<http://devservices-colo-west-livekit.corp.cvscaremark.com|devservices-colo-west-livekit.corp.cvscaremark.com>"
* issuer: C=US; O=DigiCert Inc; CN=DigiCert Global G2 TLS RSA SHA256 2020 CA1
* SSL certificate verify ok.
* using HTTP/2
* h2 [:method: HEAD]
* h2 [:scheme: https]
* h2 [:authority: <http://devservices-colo-west-livekit.corp.cvscaremark.com|devservices-colo-west-livekit.corp.cvscaremark.com>]
* h2 [:path: /]
* h2 [user-agent: curl/8.2.1]
* h2 [accept: */*]
* Using Stream ID: 1
> HEAD / HTTP/2
> Host: <http://devservices-colo-west-livekit.corp.cvscaremark.com|devservices-colo-west-livekit.corp.cvscaremark.com>
> User-Agent: curl/8.2.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/2 200
HTTP/2 200
< vary: Origin
vary: Origin
< date: Wed, 18 Feb 2026 05:57:45 GMT
date: Wed, 18 Feb 2026 05:57:45 GMT
< content-length: 2
content-length: 2
< content-type: text/plain; charset=utf-8
content-type: text/plain; charset=utf-8
< x-envoy-upstream-service-time: 0
x-envoy-upstream-service-time: 0
< server: istio-envoy
server: istio-envoy
<
* Connection #0 to host <http://devservices-colo-west-livekit.corp.cvscaremark.com|devservices-colo-west-livekit.corp.cvscaremark.com> left intactt
tall-airline-16756
02/18/2026, 5:58 AMThanks — this is great evidence. The curl output shows a successful TLS/WSS connection and a trusted certificate, so network/TLS is not the cause here. That narrows the problem to lifecycle/room timing (the compositor ending the session) or WebRTC negotiation failures that happen after the WSS connection (less likely given the short runtime you’ve seen).
Key takeaways from your curl output
• TLS handshake succeeded (TLSv1.3) and certificate verification passed: "SSL certificate verify ok."
• The server’s cert matches the hostname and is issued by DigiCert — Chrome should trust it.
• The WSS host (devservices-colo-west-livekit.corp.cvscaremark.com) is reachable from the egress pod.
Given that, focus on two things in priority order
1) Room lifecycle race (most likely)
• Your logs show Chrome launching and then immediately sending END_RECORDING ~10–20s later, and you are deleting/closing the room during hangup.
• Deleting the room while Chrome is the compositor will cause Chrome to end recording and the egress to become EGRESS_ABORTED.
• Fix: stop egress and wait for it to finish, or rely on the egress webhook
egress_endedegress_endedegress_endedb
broad-teacher-94403
02/18/2026, 6:00 AMCertificate:
Copy code
egress@livekit-egress-66d5fc5f7d-9bsfm:/$ openssl s_client -connect <http://devservices-colo-west-livekit.corp.cvscaremark.com:443|devservices-colo-west-livekit.corp.cvscaremark.com:443> -showcerts
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = <http://www.digicert.com|www.digicert.com>, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert Global G2 TLS RSA SHA256 2020 CA1
verify return:1
depth=0 C = US, ST = Rhode Island, L = Woonsocket, O = CVS Pharmacy Inc, CN = <http://devservices-colo-west.corp.cvscaremark.com|devservices-colo-west.corp.cvscaremark.com>
verify return:1
---
Certificate chain
0 s:C = US, ST = Rhode Island, L = Woonsocket, O = CVS Pharmacy Inc, CN = <http://devservices-colo-west.corp.cvscaremark.com|devservices-colo-west.corp.cvscaremark.com>
i:C = US, O = DigiCert Inc, CN = DigiCert Global G2 TLS RSA SHA256 2020 CA1
a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
v:NotBefore: Feb 12 00:00:00 2026 GMT; NotAfter: Feb 12 23:59:59 2027 GMT
-----BEGIN CERTIFICATE-----
MIIPYzCCDkugAwIBAgIQDqyshri31P3R9KiYO0GqrjANBgkqhkiG9w0BAQsFADBZ
MQswCQYDVQQGENd8vSMhd4reK2NCc2XH8YT5xIJu78m
x732s9bIWQ==
-----END CERTIFICATE-----
1 s:C = US, O = DigiCert Inc, CN = DigiCert Global G2 TLS RSA SHA256 2020 CA1
i:C = US, O = DigiCert Inc, OU = <http://www.digicert.com|www.digicert.com>, CN = DigiCert Global Root G2
a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
v:NotBefore: Sep 24 00:00:00 2020 GMT; NotAfter: Sep 23 23:59:59 2030 GMT
-----BEGIN CERTIFICATE-----
MIIE9DCCA9ygAwIBAgIQCF+eWSm4w5VsTuVP61gM2eKrLanXPDtWlIb
1GHhJRLmB7WqlLLwKPZhJl5VHPgB63dx
-----END CERTIFICATE-----
---
Server certificate
subject=C = US, ST = Rhode Island, L = Woonsocket, O = CVS Pharmacy Inc, CN = <http://devservices-colo-west.corp.cvscaremark.com|devservices-colo-west.corp.cvscaremark.com>
issuer=C = US, O = DigiCert Inc, CN = DigiCert Global G2 TLS RSA SHA256 2020 CA1
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 5710 bytes and written 432 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---t
tall-airline-16756
02/18/2026, 6:01 AMThanks — this is great confirmation. The openssl output shows a valid, trusted certificate chain (Verify return code: 0), so TLS/WSS is not the cause. Because network/TLS checks passed, the egress aborts you’ve seen are almost certainly due to the room lifecycle (Chrome compositor ending when the room is deleted or participants leave) rather than a certificate or connectivity problem.
Quick summary of findings
• Template server responded with the HTML page — the compositor can load the template.
• curl and openssl show a successful TLS handshake and a cert issued by DigiCert that matches your WSS host.
• Chrome is launched with a valid template URL and WSS param.
• Egress still ends quickly with chrome: END_RECORDING → EGRESS_ABORTED. That matches a room-deletion or participant-leave race, not TLS.
Recommended next steps (prioritized)
1. Stop deleting the room immediately — ensure egress finishes first (highest impact)
◦ Persist the egress_id when you start recording.
◦ On hangup, call stop_egress (if still running) and wait for a terminal state, or rely on the webhook below.
◦ I can provide the exact stop-and-poll patch adapted to your SDK if you want (I already provided a template earlier).
1. Deploy an egress webhook and delete the room from the webhook handler (best practice)
◦ Configure LiveKit to call your /webhook endpoint for egress_ended.
◦ When you receive egress_ended for that egress_id, delete the room. This avoids race conditions entirely.
1. (Optional) Turn on verbose Chrome/WebRTC logging while you test
◦ Add Chrome flags to the egress container to capture ICE/SDP/TURN errors if you still see unexplained aborts:
--chrome-args="--enable-logging --v=1 --enable-webrtc-logging=1 --no-sandbox --disable-gpu --enable-logging=stderr"
• Collect logs around the END_RECORDING time to confirm there are no ICE/TURN failures.
1. Small safety in hangup flow (handle Stop 412)
◦ If you still use stop_egress, check state before calling Stop and treat a 412 / EGRESS_ABORTED as benign — proceed to cleanup instead of failing.
If you want, I can:
• Provide the exact stop-and-poll patch (complete functions) ready to drop into your code (I can adapt it to your LiveKit Python client method names).
• Or produce the full FastAPI webhook handler file (ready to deploy) plus instructions to configure the webhook in LiveKit.
Which would you like me to produce now — the stop-and-poll hangup patch or the webhook handler file?
b
broad-teacher-94403
02/18/2026, 6:02 AMThis is my current code:
Copy code
"""
Egress recording management.
Handles starting audio recording for rooms using LiveKit Egress
with GCS bucket storage via Workload Identity, and graceful
egress stop + polling to ensure uploads complete before room deletion.
"""
import asyncio
import json
from typing import Optional
from livekit import agents, api
from src.common.config.app_config import get_application_config
from src.common.service.logging.cvs_logger import error, signal
# Load application config
CONFIG = get_application_config()
async def start_recording(ctx: agents.JobContext) -> Optional[str]:
"""
Start audio recording for the room using LiveKit Egress.
Records audio-only to GCS bucket with Workload Identity.
Controlled by recordAudio flag in room metadata (passed from controller via RECORD_AUDIO env var).
Returns:
The egress_id string if recording was started successfully, None otherwise.
"""
# Parse room metadata (could be JSON string or dict)
room_metadata = {}
try:
if ctx.room.metadata:
if isinstance(ctx.room.metadata, str):
room_metadata = json.loads(ctx.room.metadata)
elif isinstance(ctx.room.metadata, dict):
room_metadata = ctx.room.metadata
except Exception as e:
error(f"Failed to parse room metadata for recording: {e}")
# Default to recording enabled if parsing fails
room_metadata = {"recordAudio": True}
# Check if recording is enabled via room metadata
record_audio = room_metadata.get("recordAudio", True)
if not record_audio:
signal("Audio recording disabled via room metadata (RECORD_AUDIO env var)")
return None
try:
# Get GCS bucket from config
gcs_bucket = CONFIG.get("cvs.gcp-bucket")
if not gcs_bucket:
error("GCS bucket not configured in application.yaml (cvs.gcp-bucket)")
return None
signal(f"Starting audio recording to GCS bucket: {gcs_bucket}")
campaign_id = room_metadata.get("campaignId")
session_id = room_metadata.get("sessionId")
# Validate required fields
if not campaign_id:
error("Cannot start recording: campaignId missing from room metadata")
return None
if not session_id:
error("Cannot start recording: sessionId missing from room metadata")
return None
filepath = f"recordings/{campaign_id}/{session_id}.ogg"
# Create egress request for audio-only recording
# Using Workload Identity - no credentials needed
# DUAL_CHANNEL_AGENT separates audio into:
# Left channel → agent (TTS) audio
# Right channel → caller (SIP participant) audio
req = api.RoomCompositeEgressRequest(
room_name=ctx.room.name,
audio_only=True, # Audio only for voice calls
audio_mixing=api.AudioMixing.DUAL_CHANNEL_AGENT,
file_outputs=[
api.EncodedFileOutput(
file_type=api.EncodedFileType.OGG, # OGG format for audio
filepath=filepath,
gcp=api.GCPUpload(
bucket=gcs_bucket,
# No credentials - Workload Identity handles authentication
),
)
],
)
# Start recording
lkapi = api.LiveKitAPI()
egress_info = await lkapi.egress.start_room_composite_egress(req)
await lkapi.aclose()
signal(f"Recording started successfully: egress_id={egress_info.egress_id}, file={filepath}")
return egress_info.egress_id
except Exception as e:
error(f"Failed to start audio recording: {e}")
# Don't fail the agent if recording fails
import traceback
error(traceback.format_exc())
return None
async def stop_and_wait_for_egress(
egress_id: str,
timeout: float = 10.0,
poll_interval: float = 2.0,
) -> bool:
"""
Gracefully stop an egress job and poll until it reaches a terminal state.
This ensures the recording is fully uploaded to GCS before the room is deleted.
Flow:
1. Fetches current egress status to avoid calling Stop on an already-ended
egress (prevents noisy 412 / ``failed_precondition`` errors).
2. If the egress is still stoppable, calls ``stop_egress()`` to signal the
egress worker to flush & upload.
3. Polls ``list_egress()`` every ``poll_interval`` seconds until the status
reaches a terminal state (EGRESS_COMPLETE, EGRESS_FAILED, EGRESS_ABORTED,
or EGRESS_LIMIT_REACHED).
4. Times out after ``timeout`` seconds to prevent infinite hangs.
Args:
egress_id: The egress ID returned by start_recording().
timeout: Maximum seconds to wait for egress completion.
poll_interval: Seconds between status polls.
Returns:
True if egress completed successfully, False otherwise.
"""
if not egress_id:
return False
terminal_statuses = {
api.EgressStatus.EGRESS_COMPLETE,
api.EgressStatus.EGRESS_FAILED,
api.EgressStatus.EGRESS_ABORTED,
api.EgressStatus.EGRESS_LIMIT_REACHED,
}
lkapi = api.LiveKitAPI()
try:
# Step 1: Fetch current egress status before attempting Stop
current_status = None
try:
egress_list = await lkapi.egress.list_egress(
api.ListEgressRequest(egress_id=egress_id)
)
if egress_list.items:
current_status = egress_list.items[0].status
signal(f"Egress {egress_id} current status: {current_status}")
except Exception as e:
error(f"Failed to fetch egress status for {egress_id}: {e}")
# If already in a terminal state, return immediately — no Stop needed
if current_status in terminal_statuses:
success = current_status == api.EgressStatus.EGRESS_COMPLETE
if success:
signal(f"Egress {egress_id} already completed successfully")
else:
error(f"Egress {egress_id} already ended with status: {current_status}")
return success
# Step 2: Egress is still active/starting — send Stop signal
signal(f"Stopping egress {egress_id} ...")
try:
await lkapi.egress.stop_egress(
api.StopEgressRequest(egress_id=egress_id)
)
signal(f"Stop signal sent to egress {egress_id}")
except Exception as e:
err_msg = str(e)
if "failed_precondition" in err_msg or "status=412" in err_msg:
# Egress ended between our status check and the Stop call — race but safe
error(f"Stop egress raced with completion (already ended): {e}")
else:
error(f"Stop egress call error: {e}")
# Step 3: Poll until terminal state
elapsed = 0.0
while elapsed < timeout:
try:
egress_list = await lkapi.egress.list_egress(
api.ListEgressRequest(egress_id=egress_id)
)
if egress_list.items:
status = egress_list.items[0].status
signal(f"Egress {egress_id} status: {status} (elapsed {elapsed:.0f}s)")
if status in terminal_statuses:
success = status == api.EgressStatus.EGRESS_COMPLETE
if success:
signal(f"Egress {egress_id} completed successfully")
else:
error(f"Egress {egress_id} ended with status: {status}")
return success
except Exception as e:
error(f"Error polling egress status: {e}")
await asyncio.sleep(poll_interval)
elapsed += poll_interval
error(f"Egress {egress_id} timed out after {timeout}s — proceeding with room deletion")
return False
finally:
await lkapi.aclose()t
tall-airline-16756
02/18/2026, 6:03 AMThanks — your code for starting egress and the stop-and-wait helper look solid. You’ve covered the important pieces: creating the RoomComposite egress, returning the egress_id, fetching status before Stop, handling the 412 race, and polling for a terminal state. A few small suggestions and a ready-to-drop-in hangup integration to ensure you never delete the room prematurely.
What’s good
• You correctly create a RoomComposite audio-only request and return the egress_id.
• You check current status before calling Stop to avoid noisy 412s.
• You poll until a terminal state and time out to avoid blocking indefinitely.
• You close the LiveKit client in finally — good resource handling.
Small improvements and safety checks
• Persist egress_id in a durable place tied to the room (DB, Redis, or ctx.room.metadata) so hangup can always find it even if the job restarts.
• Slightly increase default timeout for uploads (10s may be short for larger files or slow networks). 30–60s is a safer default in production.
• Be explicit about the status enum checks: comparing enum members directly (if SDK uses enums) is clearer than relying on string/number casts.
• Log the egress_id, room_name, and timestamps for easier correlation between webhooks and logs.
• After a Stop race or timeout, prefer relying on the webhook (if configured) as an additional safety net before deleting the room.
Drop‑in hangup_call integration Below is a ready-to-drop-in async hangup_call function that:
• reads the saved egress_id (replace load_egress_id_for_room with your persistence),
• calls stop_and_wait_for_egress with a safer timeout,
• falls back to webhook-safety if egress already ended,
• then deletes the room.
Paste this into the same module and adapt the persistence functions or ctx.api calls as needed.
Copy code
async def hangup_call(self, delay_seconds: float = 1.0):
"""
Hang up the call without aborting egress:
1. Wait a short delay for final playout.
2. Stop egress and wait for it to finish (uploads).
3. Delete the room only after egress reaches a terminal state or timeout.
"""
ctx = get_job_context()
if ctx is None:
error("Cannot hang up: Not running in a job context")
return
try:
signal(f"Waiting {delay_seconds}s for final message playout before stopping egress...")
await asyncio.sleep(delay_seconds)
# Retrieve egress_id (replace with your DB/ctx lookup)
egress_id = await load_egress_id_for_room(ctx.room.name)
if egress_id:
signal(f"Stopping egress before room deletion: {egress_id}")
# Use a larger timeout in production (e.g., 30-60s)
finished = await stop_and_wait_for_egress(
egress_id=egress_id,
timeout=45.0,
poll_interval=2.0,
)
if not finished:
# If we timed out or egress failed/aborted, log and proceed.
error(f"Egress {egress_id} did not complete successfully before timeout; proceeding to cleanup")
else:
signal("No egress_id found for this room; proceeding to delete room")
# At this point, egress is completed/failed/aborted or timed out.
# If you also use webhooks, the webhook will confirm uploads; you can optionally wait for that instead.
signal(f"Hanging up call by deleting room: {ctx.room.name}")
await ctx.api.room.delete_room(api.DeleteRoomRequest(room=ctx.room.name))
signal("Call hung up successfully")
except Exception as e:
error(f"Error hanging up call: {e}")egress_endedegress_idbackupStorageUsedegress_endedb
broad-teacher-94403
02/18/2026, 6:05 AMThis is my current hangup_call method:
Copy code
sync def hangup_call(self, delay_seconds: float = 1.0):
"""
Hang up the call by stopping egress (if active) and deleting the room.
This ends the call for all participants.
Args:
delay_seconds: Time to wait before hanging up to allow TTS playout to complete.
"""
ctx = get_job_context()
if ctx is None:
error("Cannot hang up: Not running in a job context")
return
try:
signal(f"Waiting {delay_seconds}s for final message playout before hanging up...")
await asyncio.sleep(delay_seconds)
# Mark agent-initiated hangup so the participant_disconnected handler
# (in entrypoint.py) doesn't publish a false user_hangup event or
# attempt duplicate egress cleanup.
ctx._call_ended = True
ctx._egress_stopped = True
await end_call(ctx.room.name, egress_id=self.egress_id)
except Exception as e:
error(f"Error hanging up call: {e}")t
tall-airline-16756
02/18/2026, 6:05 AMThanks — this hangup_call looks mostly correct: it waits briefly, marks the context to avoid duplicate cleanup, and calls end_call with the egress_id. To confirm there’s no race that causes premature egress abortion I need one more piece:
Please paste the implementation of end_call (or show the code path it calls). I want to verify that end_call:
• looks up/accepts the egress_id you passed (i.e., uses the same egress_id returned by start_recording),
• calls stop_egress (or your stop_and_wait_for_egress) and waits for a terminal state before deleting the room,
• handles the 412 / failed_precondition case gracefully (egress already aborted/completed),
• does not immediately delete the room if stop/poll times out (unless you intentionally want that fallback).
If you prefer, you can also confirm these three behaviors from end_call instead of pasting it:
1. It calls stop_and_wait_for_egress (or equivalent) with a non-trivial timeout (30–60s recommended).
2. It only calls ctx.api.room.delete_room after stop_and_wait_for_egress returns (or after receiving egress_ended webhook).
3. It persists and uses the same egress_id you saved when starting egress.
Share end_call (or confirm the three items above) and I’ll point out any edits needed or provide a drop-in patched end_call that uses your existing stop_and_wait_for_egress implementation.
b
broad-teacher-94403
02/18/2026, 6:06 AM Copy code
async def end_call(room_name: str, egress_id: Optional[str] = None):
"""
End a call by (optionally) stopping egress and then deleting the room.
If an ``egress_id`` is provided the function first calls
``stop_and_wait_for_egress`` so the recording is fully uploaded to GCS
before the room is destroyed — this prevents EGRESS_ABORTED.
Uses its own API client so it is safe to call even when the job context
HTTP session has already been closed.
"""
# Step 1: Gracefully stop egress and wait for upload (if recording)
if egress_id:
signal(f"Stopping egress before room deletion: {egress_id}")
await stop_and_wait_for_egress(egress_id)
# Step 2: Delete the room
signal(f"Ending call by deleting room: {room_name}")
lkapi = api.LiveKitAPI()
try:
await lkapi.room.delete_room(api.DeleteRoomRequest(room=room_name))
signal("Call ended successfully")
finally:
await lkapi.aclose()t
tall-airline-16756
02/18/2026, 6:06 AMThanks — everything you’ve shown (start_recording, stop_and_wait_for_egress, end_call, and your hangup_call that calls end_call) looks correct and follows the recommended sequence to avoid EGRESS_ABORTED. Good work.
Quick confirmation of correctness
• start_recording returns and logs the egress_id (you persist it by returning it to the caller).
• stop_and_wait_for_egress:
◦ checks current status before Stop,
◦ handles the 412 / failed_precondition race,
◦ polls until a terminal state,
◦ closes the LiveKit client in finally.
• end_call calls stop_and_wait_for_egress (if egress_id provided) and only deletes the room afterwards.
• hangup_call awaits end_call and marks ctx to avoid duplicate cleanup.
Why this should prevent EGRESS_ABORTED
• You no longer delete the room while the Chrome compositor is still active. Stopping egress and waiting for an EGRESS_COMPLETE (or other terminal state) gives the worker time to flush and upload the recording before the room is destroyed — this removes the main race that caused Chrome to emit END_RECORDING early.
A few final, practical recommendations and small tweaks to make the flow more robust
1) Increase the default timeout
• Change the default timeout from 10s to 30–60s in stop_and_wait_for_egress. Uploads and finalization sometimes take longer on busy nodes or slower networks.
2) Persist egress_id durably
• Ensure egress_id survives process restarts. Store it in DB/Redis or persist it in ctx.room metadata (if safe) so end_call can always find it.
3) Clear the persisted egress_id after end_call completes
• You already call delete_egress_record in earlier code examples; ensure you remove the persisted record after egress ends to avoid stale lookups.
4) Treat 412/EGRESS_ABORTED as benign in hangup flow
• You already handle this in stop_and_wait_for_egress; keep logging it as informational, then proceed to cleanup so hangup never blocks.
5) Add egress_ended webhook as a safety net (recommended)
• Webhooks remove races entirely. Configure a webhook endpoint to receive egress_ended and perform any final cleanup or verification (delete room only after egress_ended if you prefer that model).
6) Optional: enable Chrome verbose logging for intermittent issues
• If you still see unexplained early aborts after the above, add:
--chrome-args="--enable-logging --v=1 --enable-webrtc-logging=1 --no-sandbox --disable-gpu --enable-logging=stderr" Capture logs around END_RECORDING to spot ICE/SDP/TURN issues.
7) Add observability
• Log timestamps when you:
◦ start_recording (record start time + egress_id),
◦ call stop_egress,
◦ detect terminal egress state,
◦ delete the room.
This makes it trivial to correlate agent activity, egress lifecycle, and webhooks.
Example small tweak (recommended defaults)
• Increase timeout and make poll_interval configurable:
stop_and_wait_for_egress(egress_id, timeout=45.0, poll_interval=2.0)
What to do next (pick one)
• If you want a final drop-in change, I can:
◦ update your stop_and_wait_for_egress default timeout to 45s and add durable persistence calls around egress_id, OR
◦ provide a ready-to-deploy FastAPI webhook handler that verifies signatures and deletes rooms on egress_ended.
Which would you like me to produce now — the small timeout/persistence patch, or the webhook handler file ready to deploy?
b
broad-teacher-94403
02/18/2026, 6:07 AMThis is what I see in worker logs during call end:
Copy code
11:20:45 INFO cvs_event_logger CvsEventEnum.SIGNAL: Stopping egress before room deletion: EG_EBHUA2pjEov4
INFO cvs_event_logger CvsEventEnum.SIGNAL: Egress EG_EBHUA2pjEov4 current status: 5
INFO cvs_event_logger CvsEventEnum.ERROR: Egress EG_EBHUA2pjEov4 already ended with status: 5
INFO cvs_event_logger CvsEventEnum.SIGNAL: Ending call by deleting room:
outbound-call-20260218-112008-ad6fa4aa
11:20:46 INFO cvs_event_logger CvsEventEnum.SIGNAL: Call ended successfullyt
tall-airline-16756
02/18/2026, 6:07 AMThanks — this is very useful detail. Your worker logs show the exact flow and confirm what's happening; here’s a short, actionable summary and next steps you can apply immediately.
What the logs mean
• Your worker attempted to stop egress EG_EBHUA2pjEov4, but the egress was already in terminal state 5 when checked:
◦ Status 5 corresponds to EGRESS_ABORTED (egress ended prematurely).
• Because the egress was already ended, your stop call is a no-op and you then proceed to delete the room — that’s safe, but it indicates the egress aborted before your stop arrived.
Why this happened (most likely)
• The Chrome compositor or the egress worker ended before you could stop it. Common causes:
◦ Room lifecycle: the room ended or participants left (or you deleted the room) before the egress completed.
◦ Egress started too early — no audio tracks were present when the composite started, so Chrome had nothing to record and the job aborted.
◦ Less likely now: network/TLS problems — you already proved TLS/WSS is fine.
Immediate, practical recommendations
1. Start egress only after tracks exist
◦ Delay starting Room Composite until at least one audio track is published (use a track_published webhook or poll ListParticipants). This prevents empty recordings that abort immediately.
1. Stop-and-wait remains correct, but add webhook safety
◦ Keep your stop_and_wait_for_egress logic (good).
◦ Also register an egress_ended webhook and use it as the authoritative signal for final cleanup. If stop races and you see EGRESS_ABORTED, the webhook will confirm final status and let you handle backupStorageUsed if needed.
1. Treat EGRESS_ABORTED as informational at hangup
◦ Your code already logs and proceeds to delete the room when egress is already ended. That’s correct: handle the 412 / aborted case gracefully and continue cleanup.
1. Increase timeouts and persist egress_id
◦ Use a longer timeout for polling (30–60s) to allow uploads to finish on slow networks.
◦ Persist egress_id durably (DB/Redis) so end_call can always find it even across restarts.
Quick checklist / actions to apply now
• Add a guard before start_recording:
◦ Wait for track_published (or at least one participant with an audio track).
• Keep your current end_call which calls stop_and_wait_for_egress, but:
◦ Increase default timeout to ~45s.
◦ After a Stop race (EGRESS_ABORTED), rely on the egress_ended webhook if you need to inspect backupStorageUsed.
• Configure and deploy an egress webhook endpoint that:
◦ Verifies signature,
◦ On egress_ended, checks status & backupStorageUsed,
◦ Deletes the room (or performs final cleanup) only after receiving egress_ended for that egress_id.
If you want I can:
• Provide a small patch to start_recording that waits for track_published before starting egress.
• Provide a complete FastAPI webhook handler that verifies LiveKit signatures and deletes the room on egress_ended (including handling backupStorageUsed).
Which would be most helpful next — the track-presence guard patch or the webhook handler code?
Sources: Why is there extra audio at the end of my recordings of an agent session? |... | Backup Egress Recordings | Universal Egress | Universal Egress | Best practices for managing webhook event streams | LiveKit
b
broad-teacher-94403
02/18/2026, 6:10 AMThis is my egress container command:
Copy code
containers:
- args:
- '--web'
- '--templates-host=localhost:7980'
- >-
--chrome-args=--enable-logging --v=3 --no-sandbox --disable-gpu
--enable-webrtc-logging=1 --enable-logging=debug
env:
- name: EGRESS_CONFIG_BODY
valueFrom:
configMapKeyRef:
key: config.yaml
name: livekit-egress
- name: GOOGLE_APPLICATION_CREDENTIALS
value: /etc/workload-identity/credential-configuration.json
- name: LIVEKIT_EGRESS_ENABLE_NVENC
value: 'false'
- name: NVIDIA_VISIBLE_DEVICES
value: none
- name: NVIDIA_DRIVER_CAPABILITIESentrypoint Copy code
@server.rtc_session(agent_name=AGENT_NAME)
async def outbound_agent_entrypoint(ctx: agents.JobContext):
"""
Agent entrypoint - called when agent is dispatched to room.
This function dynamically configures the agent based on campaign data
stored in room metadata. The agent calls an external API instead of
using the built-in LLM.
"""
# Connect to room with external TURN if configured
rtc_config = await get_external_turn_config()
if rtc_config:
await ctx.connect(rtc_config=rtc_config)
else:
await ctx.connect()
# Fetch room metadata explicitly from LiveKit API to ensure we have the latest data
# ctx.room.metadata might not be populated when the agent first starts
room_metadata = {}
try:
# Try to get metadata from ctx.room first (might be available)
if ctx.room.metadata:
if isinstance(ctx.room.metadata, str):
room_metadata = json.loads(ctx.room.metadata)
elif isinstance(ctx.room.metadata, dict):
room_metadata = ctx.room.metadata
except Exception as e:
error(f"WARNING: Could not parse ctx.room.metadata: {e}")
# If metadata is empty, fetch room explicitly from LiveKit API
if not room_metadata:
lkapi = None
try:
lkapi = api.LiveKitAPI()
room_info = await lkapi.room.list_rooms(
api.ListRoomsRequest(names=[ctx.room.name])
)
if room_info.rooms and len(room_info.rooms) > 0:
fetched_room = room_info.rooms[0]
if fetched_room.metadata:
room_metadata = json.loads(fetched_room.metadata)
else:
error("WARNING: Room found but metadata is empty")
else:
error("WARNING: Room not found in LiveKit API")
except Exception as e:
error(f"ERROR: Failed to fetch room from LiveKit API: {e}")
import traceback
traceback.print_exc()
finally:
# Always close the API client to prevent resource leaks
if lkapi:
await lkapi.aclose()
# -----------------------------------------------------------------------
# egress_id is initialised as None and set after the SIP participant
# joins (see below). Closures (_on_participant_disconnected,
# _shutdown_callback) read it by reference, so they always see the
# current value at execution time.
# -----------------------------------------------------------------------
egress_id = None
# -----------------------------------------------------------------------
# Shared call-state flags (stored on ctx so hangup_call / conversation
# handlers in other modules can set them too):
#
# ctx._call_ended – True once the *agent* initiates end-of-call
# (hangup_call / one-way end_call). Prevents the
# participant_disconnected handler from publishing a
# false ``user_hangup`` event when the room is
# deleted by us.
# ctx._egress_stopped – True once any code path has started stopping the
# egress. Prevents duplicate stop_and_wait calls.
# -----------------------------------------------------------------------
ctx._call_ended = False
ctx._egress_stopped = False
@ctx.room.on("participant_disconnected")
def _on_participant_disconnected(participant: rtc.RemoteParticipant):
"""
Handle user (SIP) participant disconnect.
Note: The LiveKit ``participant_disconnected`` event only passes the
participant object — there is no ``reason`` parameter.
"""
# Only act on non-agent participants (i.e. the SIP/user participant)
if participant.kind == rtc.ParticipantKind.PARTICIPANT_KIND_AGENT:
return
# If the agent initiated the disconnect (room deletion), this fires
# with reason ROOM_DELETED — skip to avoid a false user_hangup event
# and redundant egress cleanup.
if ctx._call_ended:
signal(
f"Participant {participant.identity} disconnected after agent-initiated "
f"hangup — skipping user_hangup event."
)
return
signal(
f"User participant disconnected: {participant.identity}. "
f"Publishing user_hangup event..."
)
# Publish user_hangup audit event
asyncio.ensure_future(
publish_conversation_event(
session_id=session_id,
campaign_id=campaign_id,
phone_number=phone_number,
user_query="",
bot_response="",
status=USER_HANGUP,
extra_data={"participantIdentity": participant.identity},
)
)
if ctx._egress_stopped or not egress_id:
return
signal(f"Stopping egress {egress_id} proactively...")
ctx._egress_stopped = True
asyncio.ensure_future(_graceful_egress_and_cleanup(ctx.room.name, egress_id))
# Shutdown callback — last-resort safety net if the event was missed
async def _shutdown_callback():
if ctx._egress_stopped or not egress_id:
return
signal(f"Shutdown callback: stopping egress {egress_id} as safety net")
ctx._egress_stopped = True
await stop_and_wait_for_egress(egress_id)
ctx.add_shutdown_callback(_shutdown_callback)
# Parse campaign config dict back into Pydantic model
campaign_config_dict = room_metadata.get("campaignConfig", {})
campaign_config = (
CampaignConfig(**campaign_config_dict) if campaign_config_dict else None
)
campaign_id = room_metadata.get("campaignId", "")
phone_number = room_metadata.get("phoneNumber", "")
session_id = room_metadata.get("sessionId")
# Validate required session_id
if not session_id:
raise ValueError("sessionId is required but missing from room metadata")
# Get all metadata (includes opportunityId, sessionId, patientId, data, etc.)
# The 'data' field is passed through as-is to other services
data = room_metadata.get("data", {})
# Extract ASR/TTS/Agent config from campaign using Pydantic model
asr_config = campaign_config.asr if campaign_config else {}
tts_config = campaign_config.tts if campaign_config else {}
agent_config = campaign_config.agent if campaign_config else {}
# Determine campaign type early so we can skip irrelevant warnings
campaign_type = campaign_config.campaign.get("type", "two_way_conversation") if campaign_config else "two_way_conversation"
# Agent API URL from campaign config (only required for two-way conversations)
agent_api_url = agent_config.get("path", "")
if not agent_api_url and campaign_type != "one_way_conversation":
error(
"WARNING: agent.path not configured in campaign, agent will not work correctly"
)
# Determine if we should use CVS models based on USE_LIVEKIT_DEFAULT_MODELS env variable
# Default is false (use CVS models)
use_livekit_defaults = (
CONFIG.get("USE_LIVEKIT_DEFAULT_MODELS", "false").lower() == "true"
)
use_cvs_models = not use_livekit_defaults
if use_cvs_models:
# Get defaults from application.yaml
default_asr_model = CONFIG.get("cvs.asr.model")
default_tts_model = CONFIG.get("cvs.tts.model")
cvs_tts_base_url = CONFIG.get("cvs.tts.base_url")
cvs_asr_base_url = CONFIG.get("cvs.asr.base_url")
# Validate that CVS models are configured in application.yaml
if not default_asr_model:
raise ValueError(
"CVS ASR model not configured in application.yaml (cvs.asr.model)"
)
if not default_tts_model:
raise ValueError(
"CVS TTS model not configured in application.yaml (cvs.tts.model)"
)
if not cvs_tts_base_url:
raise ValueError(
"CVS TTS base URL not configured in application.yaml (cvs.tts.base_url)"
)
if not cvs_asr_base_url:
raise ValueError(
"CVS ASR base URL not configured in application.yaml (cvs.asr.base_url)"
)
# For CVS models, create plugin instances directly to avoid LiveKit Cloud inference connections
# This prevents AgentSession from trying to connect to agent-gateway.livekit.cloud
stt_model = deepgram.STT(
model=default_asr_model,
base_url=cvs_asr_base_url,
api_key=DUMMY_KEY,
sample_rate=16000,
)
tts_model = deepgram.TTS(
model=default_tts_model,
base_url=cvs_tts_base_url,
api_key=DUMMY_KEY,
sample_rate=16000,
encoding="linear16",
)
else:
# Get models from campaign config
stt_model = asr_config.get("model")
tts_model = tts_config.get("model")
# Validate that models are provided in campaign config
if not stt_model:
raise ValueError(
f"ASR model not configured in campaign config for campaign_id: {campaign_id}"
)
if not tts_model:
raise ValueError(
f"TTS model not configured in campaign config for campaign_id: {campaign_id}"
)
signal(f"Campaign type: {campaign_type}")
signal("Waiting for SIP participant to join before starting egress...")
try:
sip_participant = await asyncio.wait_for(
ctx.wait_for_participant(), timeout=60
)
signal(f"SIP participant joined: {sip_participant.identity}")
except asyncio.TimeoutError:
error(
"No participant joined within 60 s — starting recording anyway "
"(egress may abort if no audio tracks are present)"
)
# Start audio recording now that audio tracks are in the room
egress_id = await start_recording(ctx)
if campaign_type == "one_way_conversation":
await handle_one_way_conversation(
ctx, campaign_config, data, tts_model,
campaign_id, phone_number, session_id,
egress_id=egress_id,
)
else:
await handle_two_way_conversation(
ctx, campaign_config, data, stt_model, tts_model,
use_cvs_models, campaign_id, phone_number, session_id,
egress_id=egress_id,
)
async def _graceful_egress_and_cleanup(room_name: str, egress_id: str):
"""
Stop egress gracefully and then delete the room.
Used by the participant_disconnected handler when the user hangs up,
ensuring the recording uploads to GCS before the room is destroyed.
"""
try:
await end_call(room_name, egress_id=egress_id)
except Exception as e:
error(f"Error in graceful egress cleanup: {e}")
if __name__ == "__main__":
agents.cli.run_app(server)t
tall-airline-16756
02/18/2026, 6:11 AMThanks — you’ve shared the right pieces and your setup looks mostly correct. A few targeted observations and next steps to stop the premature EGRESS_ABORTED behavior.
What’s good
• Template server is reachable and returns HTML.
• TLS/WSS is valid (you confirmed with curl/openssl).
• You launch Chrome with verbose flags in the container (good for debugging).
• Your entrypoint waits for a SIP participant (up to 60s) before starting egress, and your hangup/stop flow calls stop_and_wait_for_egress — all best-practice patterns.
Why you still see early aborts (most likely)
• Waiting for a participant to join does not guarantee an audio track is published yet. Room Composite needs audio tracks present (and the browser compositor to successfully subscribe to them) — if egress starts before tracks are available the compositor may quickly signal END_RECORDING → EGRESS_ABORTED.
• A race can still occur between participant disconnect / room deletion and egress startup or upload completion, even with your stop/poll logic. Your logs showing status 5 (EGRESS_ABORTED) indicate the compositor or worker ended before your stop call took effect.
Small, concrete changes to eliminate the race 1) Wait for an audio track (recommended)
• After participant join, wait until at least one audio track is published/available before calling start_recording.
• Minimal approach: after sip_participant join, poll ctx.room participants’ tracks or use ctx.wait_for_event / webhook for a
track_published Copy code
# wait up to 10s for an audio track to appear
for _ in range(10):
for p in ctx.room.participants:
if any(t.kind == "audio" and t.is_subscribed for t in p.tracks):
start_recording(...)
break
await asyncio.sleep(1)
# fallback: log and start anyway after timeoutegress_endedegress_endedb
broad-teacher-94403
02/18/2026, 6:12 AM+ rm -rf '/home/egress/tmp/*'
+ rm -rf /var/run/pulse /var/lib/pulse /home/egress/.config/pulse /home/egress/.cache/xdgr/pulse
+ pulseaudio -D --verbose --exit-idle-time=-1 --disallow-exit
I: [pulseaudio] main.c: Daemon startup successful.
+ exec /tini -- egress
2026-02-18T054123.659Z INFO egress redis/redis.go:142 connecting to redis {"nodeID": "NE_swc9SECkfYt7", "clusterID": "", "simple": true, "addr": "redis-14764.redis-dev-west.corp.cvscaremark.com:14764"}
2026-02-18T054123.885Z INFO egress stats/monitor.go:137 cpu available: 768.000000 max cost: 4.000000 {"nodeID": "NE_swc9SECkfYt7", "clusterID": ""}
2026-02-18T054123.890Z DEBUG egress server/server.go:141 starting service {"nodeID": "NE_swc9SECkfYt7", "clusterID": "", "version": "1.8.6"}
2026-02-18T054123.890Z DEBUG egress server/server.go:133 starting template server on address localhost:7980 {"nodeID": "NE_swc9SECkfYt7", "clusterID": ""}
2026-02-18T054123.890Z INFO egress server/server.go:147 service ready {"nodeID": "NE_swc9SECkfYt7", "clusterID": ""}
2026-02-18T055014.160Z DEBUG egress stats/monitor.go:151 cpu check {"nodeID": "NE_swc9SECkfYt7", "clusterID": "", "total": 768, "available": 768, "pending": 0, "used": 0, "activeRequests": 0, "activeWeb": 0, "required": 1, "canAccept": true}
2026-02-18T055014.662Z INFO egress server/server_rpc.go:58 request received {"nodeID": "NE_swc9SECkfYt7", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055014.665Z INFO egress server/server_rpc.go:75 request validated {"nodeID": "NE_swc9SECkfYt7", "clusterID": "", "egressID": "EG_EBHUA2pjEov4", "requestType": "room_composite", "outputType": "file", "room": "outbound-call-20260218-112008-ad6fa4aa", "request": {"RoomComposite":{"room_name":"outbound-call-20260218-112008-ad6fa4aa","audio_only":true,"Output":null,"Options":null,"file_outputs":[{"file_type":2,"filepath":"recordings/1251a2b4-84f5-4793-a2e9-19b8fb8c25e5/cfa1a3e5-9d7e-4210-b089-6c93a0a49645.ogg","Output":{"Gcp":{"bucket":"livekit-recordings-nonprod"}}}]}}}
2026-02-18T055014.696Z DEBUG egress server/main.go:177 handler launched {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055014.697Z INFO egress redis/redis.go:142 connecting to redis {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4", "simple": true, "addr": "redis-14764.redis-dev-west.corp.cvscaremark.com:14764"}
2026-02-18T055014.898Z DEBUG egress source/web.go:159 creating pulse sink {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055014.904Z DEBUG egress source/web.go:183 creating X display {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4", "display": ":1085744831", "dims": "1280x720x24"}
2026-02-18T055014.905Z DEBUG egress source/web.go:214 launching chrome {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4", "url": "http://localhost:7980/?layout=&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3NzE0ODAyMTQsImlzcyI6IkFQSUtFWTY4YjBlMGMyZTA2YWVhNTEiLCJraW5kIjoiZWdyZXNzIiwibmJmIjoxNzcxMzkzODE0LCJzdWIiOiJFR19FQkhVQTJwakVvdjQiLCJ2aWRlbyI6eyJjYW5QdWJsaXNoIjpmYWxzZSwiY2FuUHVibGlzaERhdGEiOmZhbHNlLCJjYW5TdWJzY3JpYmUiOnRydWUsImhpZGRlbiI6dHJ1ZSwicmVjb3JkZXIiOnRydWUsInJvb20iOiJvdXRib3VuZC1jYWxsLTIwMjYwMjE4LTExMjAwOC1hZDZmYTRhYSIsInJvb21Kb2luIjp0cnVlfX0.jD7BRDjnbHqntOX_JUd_5yGH-hbcueXtYJ_k933sPXg&url=wss%3A%2F%2Fdevservices-colo-west-livekit.corp.cvscaremark.com", "sandbox": false, "insecure": true}
00000.140400262 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00m cudaloader gstcudaloader.c169gst_cuda_load_library:[00m Could not open library libcuda.so.1, libcuda.so.1: cannot open shared object file: No such file or directory
00000.140421972 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00m nvcodec plugin.c94plugin_init:[00m Failed to load cuda library
00000.163263445 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0x120000: 'AVR (Audio Visual Research)' is not mapped
00000.163280785 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0x180000: 'CAF (Apple Core Audio File)' is not mapped
00000.163285035 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0x100000: 'HTK (HMM Tool Kit)' is not mapped
00000.163289555 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0xc0000: 'MAT4 (GNU Octave 2.0 / Matlab 4.2)' is not mapped
00000.163293555 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0xd0000: 'MAT5 (GNU Octave 2.1 / Matlab 5.0)' is not mapped
00000.163297725 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0x210000: 'MPC (Akai MPC 2k)' is not mapped
00000.163301075 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0x230000: 'MPEG-1/2 Audio' is not mapped
00000.163306445 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0xe0000: 'PVF (Portable Voice Format)' is not mapped
00000.163311235 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0x160000: 'SD2 (Sound Designer II)' is not mapped
00000.163317305 [36m 161[00m 0x5584d3fd1280 [33;01mWARN [00m [00;04m default gstsfelement.c97gst_sf_create_audio_template_caps:[00m format 0x190000: 'WVE (Psion Series 3)' is not mapped
2026-02-18T055015.608Z DEBUG egress gstreamer/bin.go:66 adding src audio to pipeline {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055015.609Z DEBUG egress gstreamer/bin.go:72 adding sink file to pipeline {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055015.609Z DEBUG egress pipeline/controller.go:215 waiting for start signal {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055030.975Z INFO egress source/web.go:298 chrome: END_RECORDING {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055030.975Z DEBUG egress gstreamer/state.go:75 pipeline state building -> stopping {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055030.975Z DEBUG egress gstreamer/state.go:75 pipeline state stopping -> finished {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055030.975Z DEBUG egress source/web.go:125 closing chrome {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055031.017Z DEBUG egress source/web.go:131 closing X display {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055031.022Z DEBUG egress source/web.go:138 unloading pulse module {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4"}
2026-02-18T055031.027Z DEBUG egress sink/file.go:73 removing temporary directory {"nodeID": "NE_swc9SECkfYt7", "handlerID": "EGH_oCRq2kdAkhA3", "clusterID": "", "egressID": "EG_EBHUA2pjEov4", "path": "/home/egress/tmp/EG_EBHUA2pjEov4/"}
2026-02-18T055031.034Z INFO egress service/io.go:88 egress updated {"nodeID": "NE_swc9SECkfYt7", "clusterID": "", "egressID": "EG_EBHUA2pjEov4", "requestType": "room_composite", "outputType": "file", "status": "EGRESS_ABORTED"}
t
tall-airline-16756
02/18/2026, 6:13 AMThanks — this is very helpful. Your latest egress logs make the cause clear and show exactly where to focus. Below is a concise diagnosis and a small, ready-to-drop-in patch you can add to your entrypoint to eliminate the most likely cause (starting egress before an audio track is present). I also include a couple of short, practical recommendations you can apply immediately.
Quick diagnosis (based on the logs you pasted)
• Egress launches Chrome and the gstreamer pipeline, then Chrome immediately emits
END_RECORDINGEGRESS_ABORTED Copy code
async def wait_for_audio_track(ctx, max_wait: float = 10.0, poll_interval: float = 0.5) -> bool:
"""
Wait for an audio track to be published in the room.
Returns True if an audio track was observed within max_wait seconds.
"""
elapsed = 0.0
while elapsed < max_wait:
try:
# ctx.room.participants is current view of participants; check their tracks
for p in ctx.room.participants.values():
# depending on SDK, track list structure may vary; adapt if needed
for pub in getattr(p, "published_tracks", []) or getattr(p, "tracks", []) or []:
kind = getattr(pub, "kind", None) or getattr(pub, "track", None) and getattr(pub.track, "kind", None)
# also check for subscription / enabled flags if available
if kind and str(kind).lower() == "audio":
signal(f"Found audio track for participant {p.identity}")
return True
except Exception as e:
# non-fatal; continue polling
error(f"Error while checking tracks: {e}")
await asyncio.sleep(poll_interval)
elapsed += poll_interval
# fallback: no audio track found
error(f"No audio track detected within {max_wait}s; starting egress anyway (may abort)")
return False Copy code
# after sip_participant joined...
await wait_for_audio_track(ctx, max_wait=10.0, poll_interval=0.5)
egress_id = await start_recording(ctx)egress_endedb
broad-teacher-94403
02/18/2026, 6:16 AMis there any way to do audio recording without composite tracks?
t
tall-airline-16756
02/18/2026, 6:16 AMYes — you can record audio without using Room Composite / the headless Chrome compositor. Use Track-based egress instead: Track Composite or Track Egress. Both record server-side (GStreamer) and avoid Chrome, so they remove the compositor lifecycle and TLS/ICE race issues you’ve been seeing.
Quick comparison and when to use each
• Track Composite
◦ What: Server-side recording that composes one or more server tracks into a file (MP4/MKV/WebM/WAV etc.) without Chrome.
◦ Use when: You want an encoded file (ogg/wav/mp4) of a participant’s audio, possibly synchronized with a specific track or multiple tracks.
◦ Benefits: No browser, lower resource usage, deterministic lifecycle, built-in encoding.
◦ Example use case: Record the SIP participant’s audio track into an OGG/WAV file for archival.
• Track Egress (raw track / websocket)
◦ What: Stream or save a single track’s raw frames (Opus in OGG or raw PCM) or forward them in real time via WebSocket.
◦ Use when: You need raw audio for real-time transcription, low-latency processing, or you want to avoid server-side transcoding.
◦ Benefits: Minimal overhead, immediate access to raw media, good for integrations like ASR pipelines.
Why these solve your current problem
• No headless Chrome means no chrome: END_RECORDING caused by a browser disconnect when the room is deleted or if Chrome can’t subscribe to tracks.
• Lifecycle becomes simpler: you stop egress via the API or let it finish; room deletion won’t forcibly abort a browser compositor because none is used.
• Faster startup and lower memory/CPU usage on the egress node.
Short examples (conceptual)
• Track Composite (record a specific audio track to a file):
◦ Start a TrackComposite egress specifying room name and audio track id, and a file output (e.g., WAV or OGG). LiveKit will encode server-side and upload to your storage (GCS/S3).
• Track Egress (stream raw audio via WebSocket):
◦ Start TrackEgress with a websocket output. Your server accepts the websocket and receives binary frames containing raw audio (or Opus in OGG) for real-time processing.
If you’d like, I can:
• Provide an exact example request in your LiveKit Python client for Track Composite to record a SIP participant’s audio to GCS (OGG/WAV), adapted to your workload-identity setup.
• Or provide a Track Egress websocket example (server-side) if you want raw audio streamed for ASR.
Which would you prefer — a Track Composite example to produce an encoded file in GCS, or a Track Egress websocket example for raw audio?
Sources: Universal Egress
b
broad-teacher-94403
02/18/2026, 6:17 AMI just need audio recording for auditiing, which one is better
t
tall-airline-16756
02/18/2026, 6:17 AMFor audio-only auditing, use Track Composite.
Why Track Composite is the better fit
• Produces a ready-to-store encoded file (OGG/WAV/MP3) directly — ideal for archival and compliance.
• Server-side encoding with GStreamer — no headless Chrome, so you avoid compositor lifecycle races and EGRESS_ABORTED issues.
• Simpler lifecycle: start the track egress for the participant’s audio track, wait for completion (or use webhooks), and the file is uploaded to your GCS bucket.
• Easier to manage retention, playback, and downstream processing (e.g., review, QA, or legal requests) because the output is a single container file per call/session.
When Track Egress is better (not recommended for pure auditing)
• If you need real-time streaming to an ASR/transcription pipeline or want raw frames for low-latency processing.
• Track Egress is more for live processing, not archival files.
Recommended next step
• I can provide a ready-to-drop-in Track Composite example using your existing LiveKit Python client and Workload Identity + GCS settings (OGG or WAV output to your bucket). It will:
◦ Find the participant’s audio track ID (or accept it if you already store it),
◦ Start Track Composite egress to GCS,
◦ Persist egress_id and show how to handle egress_ended webhook or poll status.
Which would you like — a Track Composite code snippet that:
• a) Records a specific track ID to GCS (you supply track ID), or
• b) Looks up the SIP participant and records their primary audio track automatically?
I’ll tailor the snippet to your choice and include the minimal webhook/poll guidance to ensure reliable uploads.
b
broad-teacher-94403
02/18/2026, 6:20 AMcool, so I should be able to record both audios agent and SIP, this does that right?
t
tall-airline-16756
02/18/2026, 6:21 AMYes — you can record both agent and SIP audio. There are two practical approaches; pick the one that best matches how you want stored files and how you want to lifecycle egress:
1) Recommended — Track Composite (two egresses) — server-side, no Chrome
• What it does: Record each participant’s audio track server-side (GStreamer) into an encoded file (OGG/WAV/MP3) and upload to GCS.
• How to use it for both sides:
◦ Start one Track Composite egress for the SIP participant’s audio track (captures what the caller speaks).
◦ Start another Track Composite egress for the agent’s audio track (captures TTS/agent output).
• Pros:
◦ No headless Chrome, no compositor lifecycle races (avoids EGRESS_ABORTED caused by Chrome).
◦ Produces ready-to-store files for auditing (one file per side).
◦ Lower resource usage and faster start time.
• Cons:
◦ Produces two files (one per participant). If you need a single mixed file you must merge them later or use Room Composite with mixing.
2) Alternative — Room Composite (audio_only) — single mixed file
• What it does: Browser compositor (or your template) mixes both sides into one file (you can configure dual-channel or mix rules).
• Pros:
◦ Single file with both sides (can be stereo-separated).
◦ Easier for playback if you want one combined recording.
• Cons:
◦ Depends on headless Chrome; earlier issues you saw (END_RECORDING / EGRESS_ABORTED) come from this dependency and the lifecycle race when the room is deleted.
◦ Slightly more fragile: needs careful track-presence guards, stop-and-wait, or webhook-driven cleanup.
Which to choose for auditing
• If you want simple, reliable archival files per call and to avoid Chrome-related failures: use Track Composite (two server-side egresses).
• If you must have a single mixed/multi-channel file right away (no post-processing): use Room Composite audio_only but add the protections you already built — wait for tracks, stop-and-wait or webhook-driven deletion, and robust retries.
Practical recommendation for your case
• Use Track Composite for both agent and SIP participant:
◦ It matches your auditing requirement (durable encoded files).
◦ It avoids the Chrome lifecycle problems you were debugging.
◦ You already have Workload Identity + GCS; configure the Track Composite outputs to your bucket.
If you want, I’ll provide a ready-to-drop-in Python example using your LiveKit client that:
• Finds the SIP participant and agent track IDs, then
• Starts two Track Composite egresses (OGG/WAV) to your GCS bucket, and
• Persists/handles egress_id and egress_ended webhook handling.
Which example would help most — a single code snippet that:
• a) automatically finds the two track IDs and starts two Track Composite egresses, or
• b) shows how to start one Track Composite for a given track ID (you call it twice)? Sources: Why is there extra audio at the end of my recordings of an agent session? |... | Universal Egress | Backup Egress Recordings
b
broad-teacher-94403
02/18/2026, 6:22 AMBut I need one continuos mixed audio, how do I mix them then?
t
tall-airline-16756
02/18/2026, 6:23 AMYou can get a single continuous mixed audio file — yes. There are two ways to do it; I’ll keep this short and show the recommended approach and the practical steps to make it reliable.
Recommended approach — Room Composite (audio_only)
• What it gives you: one continuous mixed audio file containing both agent and SIP audio (mixed together or stereo-separated if you prefer).
• Why use it: it produces a single file for auditing without post-processing; Room Composite handles mixing for you.
• Caveat: Room Composite uses a headless Chrome compositor. To avoid the EGRESS_ABORTED issues you saw, you must ensure tracks are present before starting egress and avoid deleting the room while egress is still running.
Practical checklist to make it reliable
1. Start Room Composite only after audio tracks are present
◦ Wait for the SIP participant to publish an audio track (not just join).
◦ I gave a small wait_for_audio_track snippet earlier — insert that before start_recording.
1. Keep your stop-and-wait or webhook flow (you already have this)
◦ Call StopEgress and poll, or better: delete the room only after receiving
egress_ended--templates-hostb
broad-teacher-94403
02/18/2026, 6:25 AMAny way to merge?
t
tall-airline-16756
02/18/2026, 6:26 AMYes — you can merge both sides into a single continuous mixed audio file. There are two practical ways to do this; pick one based on how you want the file produced and how much control you need.
1) Recommended: Room Composite (audio_only) — single mixed file, no post-processing
• What it does: headless Chrome (the compositor) joins the room and records the room’s mixed audio into one file (OGG/WAV/MP4). That file contains both agent + SIP audio continuously.
• Pros: single file, no extra processing, simplest for auditing.
• Cons: depends on the Chrome compositor lifecycle (so you must ensure tracks exist before starting egress and avoid deleting the room while egress is active).
• How to make it reliable:
◦ Wait for audio track presence (not just participant join) before starting Room Composite egress.
◦ Stop egress and wait (or use egress_ended webhook) before deleting the room to avoid EGRESS_ABORTED.
◦ Optionally use a custom template to stereo-map agent/caller to separate channels (left/right) if you want the two channels preserved in one file.
• When to use: you want one ready-to-store file and prefer no post-processing.
2) Alternative: Record separate tracks then merge (Track Composite + post-processing)
• What it does: record agent and SIP audio separately (Track Composite or Track Egress), then merge them into one file (FFmpeg) offline.
• Pros: full control over mixing, levels, channel mapping; no Chrome dependency if you use Track Composite.
• Cons: requires a post-processing step (FFmpeg) and you’ll manage two egress jobs.
• Example merge command (mix two files into one, preserving longest duration):
◦ ffmpeg -i agent.wav -i sip.wav -filter_complex amix=inputs=2:duration=longest -c:a libmp3lame mixed-audit.mp3
• When to use: you need fine-grained mixing (gain, alignment) or want to avoid any Chrome compositor entirely.
Which to choose?
• If you want a continuous single file with minimal effort: Room Composite audio_only is simplest—just ensure you add the track-presence guard and webhook/stop-and-wait to avoid early aborts.
• If you need separate channels, precise mixing, or to avoid Chrome: record each side with Track Composite, then merge with FFmpeg.
If you want, I can provide:
• A Room Composite example (LiveKit Python) that records audio_only to GCS and includes the track-presence guard + stop-and-wait integration.
• Or a Track Composite flow that records two tracks and a small FFmpeg script to merge them into one continuous file (with an example of stereo mapping or mixed mono).
Which would you like me to generate next — the Room Composite (single-step) example or the Track Composite + FFmpeg merge example?
Sources: Universal Egress | How to set a fixed participant for agent audio processing
b
broad-teacher-94403
02/18/2026, 6:27 AMRoom Composite is not working, how about we merge two channels
t
tall-airline-16756
02/18/2026, 6:27 AMGreat — merging two separately-recorded tracks is a reliable approach if Room Composite isn’t behaving for you. Below I’ll give a concise, practical plan and copy‑pasteable recipes you can use right away:
Plan overview
1. Start two Track-based egress jobs (one for the SIP participant’s audio track, one for the agent’s audio track). Each job writes a file to GCS (or local tmp then upload).
2. Wait for both egress jobs to finish (poll or use webhooks).
3. When both files are available, run an FFmpeg merge to produce one continuous mixed file (mono mix or stereo mapping).
4. Upload/retain the merged file in your audit bucket and delete intermediate files.
Details and examples
A) Start per-track recording (Python, LiveKit)
• Use Track Composite egress (preferred) or Track Egress that writes to files. Replace SDK call names if yours differ.
Pseudo/Python outline (adapt to your client):
Copy code
# start two Track Composite egresses and return egress_ids and output paths
async def start_track_recordings(room_name, agent_track_id, sip_track_id, gcs_bucket):
# filepaths you will write to
agent_path = f"recordings/{session_id}/agent-{session_id}.ogg"
sip_path = f"recordings/{session_id}/sip-{session_id}.ogg"
lkapi = api.LiveKitAPI()
# start agent track egress
agent_req = api.TrackCompositeEgressRequest(
room_name=room_name,
audio_track_id=agent_track_id,
file_output=api.EncodedFileOutput(
file_type=api.EncodedFileType.OGG,
filepath=agent_path,
gcp=api.GCPUpload(bucket=gcs_bucket)
)
)
agent_info = await lkapi.egress.start_track_composite_egress(agent_req)
# start sip track egress
sip_req = api.TrackCompositeEgressRequest(
room_name=room_name,
audio_track_id=sip_track_id,
file_output=api.EncodedFileOutput(
file_type=api.EncodedFileType.OGG,
filepath=sip_path,
gcp=api.GCPUpload(bucket=gcs_bucket)
)
)
sip_info = await lkapi.egress.start_track_composite_egress(sip_req)
await lkapi.aclose()
return (agent_info.egress_id, agent_path), (sip_info.egress_id, sip_path)start_track_composite_egressstart_track_egress Copy code
# poll helper that waits for a given egress_id to reach terminal state
async def wait_egress_done(egress_id, timeout=60, poll_interval=2):
lkapi = api.LiveKitAPI()
elapsed = 0
while elapsed < timeout:
res = await lkapi.egress.list_egress(api.ListEgressRequest(egress_id=egress_id))
if res.items:
st = res.items[0].status
if st in (api.EgressStatus.EGRESS_COMPLETE, api.EgressStatus.EGRESS_FAILED, api.EgressStatus.EGRESS_ABORTED):
await lkapi.aclose()
return st
await asyncio.sleep(poll_interval)
elapsed += poll_interval
await lkapi.aclose()
return Noneegress_ended Copy code
ffmpeg -i agent.ogg -i sip.ogg -filter_complex \
"[0:a][1:a]amix=inputs=2:duration=longest:dropout_transition=0,volume=2.0" \
-c:a libopus -b:a 64k mixed.oggamixduration=longestlibopuslibmp3lamepcm_s16le Copy code
ffmpeg -i agent.ogg -i sip.ogg -filter_complex \
"[0:a]aformat=sample_fmts=fltp:channel_layouts=mono,pan=stereo|c0=c0|c1=0[a0]; \
[1:a]aformat=sample_fmts=fltp:channel_layouts=mono,pan=stereo|c0=0|c1=c0[a1]; \
[a0][a1]amerge=inputs=2,pan=stereo|c0<c0+c2|c1<c1+c3[out]" \
-map "[out]" -c:a libopus -b:a 64k mixed_stereo.ogg Copy code
ffmpeg -i agent.ogg -i sip.ogg -filter_complex "join=inputs=2:channel_layout=stereo" mixed_stereo.ogg-ar 16000EGRESS_COMPLETEamixduration=longest-aregress_endedb
broad-teacher-94403
02/18/2026, 6:35 AMWhere to use:
egress_endedt
tall-airline-16756
02/18/2026, 6:36 AMI don't have the answer you're looking for.
You could also try asking your question:
• in one of the other Slack channels
or
• to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code
If you find the answer, please post it here to help others!
b
broad-teacher-94403
02/18/2026, 6:37 AMhow to use the egress_ended webhook? should it be a sidecar or part of my application code?
broad-teacher-94403
02/18/2026, 6:39 AMhow to use the egress_ended webhook? should it be a sidecar or part of my application code?
3 Views