I have a workflow where I have a reservation platf...
# ask-ai
e
I have a workflow where I have a reservation platform that sets up a room and dials outwards using a trunk to call a person to confirm their restaurant reservation. The issue is that we now want to include a recording for auditing purposes so that restaurants can review the transcript + the recording. We have everything implemented but the recording but we note that the recording currently can only be forwarded to S3? Our AWS security does not allow us to record to an S3 bucket directly using AWS key ID as we need to use OIDC for SOC2 purposes. What are other ways we can operate to ensure we can get the recording - do we stream the call live and have a service that records and writes on our behalf?
t
Thanks for reaching out, Arthur. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — this is a great description of the problem. I’m responding to your question about alternatives to writing egress recordings directly to S3 with AWS access keys, and how you can meet SOC2/OIDC requirements. Short answer: You don’t have to record directly to S3 with long‑lived keys. The simplest options are (1) stream the audio out of LiveKit via WebSocket (Track Egress) to your OIDC‑authenticated backend and let that backend record/store the files, or (2) push the egress as an RTMP stream into your own ingest service which records and stores using your OIDC workflows. If you are an Enterprise customer, LiveKit also supports S3 uploads using AssumeRole (temporary credentials), which can meet your security needs. Options with pros/cons and what to consider • Track Egress -> WebSocket to your backend (recommended for audio-only use) ◦ What it is: LiveKit sends raw audio frames (pcm_s16le) over a WebSocket to an endpoint you control. ◦ How it meets your needs: Your backend (already OIDC‑authenticated) receives the stream, writes the file (WAV/MP3/etc.), runs transcription, and stores artifacts in whatever compliant storage you use. ◦ Pros: Full control of storage and credentials, recordings never require LiveKit to hold your AWS keys, easy to integrate with your OIDC flows, low latency for live transcription/processing. ◦ Cons: WebSocket egress currently supports audio tracks (not arbitrary video/composite formats). You must implement the receiver to assemble frames into a file and handle reconnection/formatting. • Room/Track Composite -> RTMP into your own ingestion service ◦ What it is: Egress outputs a single RTMP stream to an RTMP endpoint you host (media server/FFmpeg endpoint in your VPC). ◦ How it meets your needs: Your ingestion service receives RTMP, records (FFmpeg or media server), and stores using OIDC. Good when you need a single composed recording (mixed audio +/- video). ◦ Pros: Works for composite outputs, leverages existing media tools for recording/encoding, keeps storage inside your infrastructure. ◦ Cons: More infrastructure (RTMP server) to operate and scale. Slightly more complex to tie into real‑time processing pipelines. • Enterprise: S3 uploads via AssumeRole / temporary credentials ◦ What it is: LiveKit uses temporary credentials (via AWS AssumeRole) instead of your long‑lived access keys to upload egress artifacts to S3. ◦ How it meets your needs: Removes the need to provide LiveKit with long‑lived keys; aligns better with your security posture. ◦ Pros: Less code to run on your side; familiar S3 workflow. ◦ Cons: This is an Enterprise feature. If you are Enterprise, this is a good compromise; if not, you’ll need one of the earlier alternatives. • Client-side recording and upload ◦ Why not: It’s fragile for an audit trail (may miss one side, depends on client reliability), and it’s typically not recommended for strict compliance/auditing scenarios. Practical recommendations and next steps 1. If you only need audio and want full control: implement Track Egress -> WebSocket to your backend. ◦ Implement a secure WSS endpoint that accepts the binary PCM frames and writes to WAV/MP3. ◦ Use your existing OIDC flow to authenticate and authorize that backend for storage/transcription. ◦ Add logic to start egress when the outbound call is answered (listen for the call/participant Joined event or hook into your SIP/egress orchestration). 1. If you need a mixed audio/video or a precomposed stream: use RTMP egress into your ingestion service, and record with FFmpeg or a media server, then store via OIDC-authenticated processes. 2. If you’re an Enterprise customer and want simpler direct S3 uploads: enable S3 AssumeRole so LiveKit uses temporary credentials rather than your long‑lived keys. If you’d like, I can help with the next technical steps: • Outline a minimal WebSocket receiver (WSS) design that writes a WAV file from pcm_s16le frames and handles reconnections. • Example RTMP ingest flow with FFmpeg commands to record/segment the stream and upload to your OIDC‑protected storage. • Guidance on when to trigger egress so you only record once the outbound call is connected (webhook or SIP rule approach). Which approach sounds closest to what you’d prefer (WebSocket receiver, RTMP ingest, or Explore AssumeRole)? If you pick one I’ll give a short implementation sketch and key things to watch for. Sources: Universal Egress | server-sdk-js | livekit/livekit_composite | DeepWiki | examples/bank-ivr | livekit/livekit_composite | DeepWiki
e
How does the websocket approach work? Is it streamed from the WebRTC room or my agent? I'm asking as I want to understand what security structure I need to put into place
t
Thanks — that’s a great clarifying question. I’ll explain exactly where the audio comes from with Track Egress → WebSocket, and what security controls you should put in place. What is being streamed and where it originates • The audio is streamed from LiveKit’s egress service, which reads the audio track(s) from the WebRTC room (not from an individual agent’s device). In other words, LiveKit captures the room track(s) server-side and sends the raw audio frames over a WebSocket to the endpoint you provide. • You tell LiveKit which track or composite to egress; LiveKit then connects to your WSS endpoint and pushes binary frames containing raw PCM audio (pcm_s16le). Security implications — what you need to protect • Your WSS endpoint is receiving raw audio directly from LiveKit, so it must be trusted and protected the same way you protect any ingestion of sensitive data (recordings, PII). • Because LiveKit initiates the connection to your endpoint, you must ensure only legitimate egress connections are accepted and processed. Recommended security controls 1. Use TLS/WSS ◦ Serve your endpoint using TLS and require secure WebSocket (wss://). This ensures encryption in transit. 1. Authenticate and authorize the egress connection ◦ Require a short-lived token, HTTP header, or URL-signed credential that LiveKit includes when connecting. Validate it on your backend and reject unauthorized connections. ◦ If possible, rotate/expire these tokens frequently. Prefer short-lived credentials rather than static secrets. 1. Validate the caller and context ◦ Include room identifiers, egress job ID, and expected track IDs in the token or connection metadata. Your backend should verify these match an expected active session before accepting and persisting data. ◦ Optionally, verify the source IP ranges if LiveKit publishes them (if available to you), as an extra layer. 1. Mutual TLS (mTLS) or client certs (optional, high assurance) ◦ For higher assurance, use mutual TLS so both LiveKit and your ingestion service present certificates. This prevents unauthorized clients from connecting even if they have a token. 1. Ingress rate and size limits ◦ Enforce request size limits, timeouts, and maximum recording length per job to protect against resource exhaustion. 1. Secure storage and access controls ◦ Immediately store recordings in your OIDC‑protected storage with strict ACLs. ◦ Encrypt at rest and enforce least-privilege access to transcripts and recordings. 1. Audit logging and monitoring ◦ Log egress job start/stop, token usage, and any authentication failures. ◦ Alert on unusual activity (e.g., many egress starts for the same room). Operational considerations for assembly & reliability • Frame handling: LiveKit streams raw PCM frames. Your receiver must concatenate frames and write a correct container (WAV/RAW) or pipe into an encoder (FFmpeg) to produce MP3/MP4. • Start/stop timing: Start egress only when relevant participants are connected (e.g., when the outbound call is answered). Use your orchestration logic or webhooks to trigger egress. • Reconnects and retries: Implement logic to handle dropped connections, resume where possible, and mark incomplete files for retry or discard. • Metadata: Accept metadata (room name, participant ID, egress job ID) so you can link recordings to reservations and audit trails. Example minimal flow (high level) 1. Your orchestration triggers LiveKit Track Egress with the WebSocket URL: wss://your-ingest.example.com/egress?token=shortlived 2. LiveKit connects over WSS and starts sending binary PCM frames for the requested track. 3. Your service validates the token and expected egress parameters, opens a writer or pipes to FFmpeg, and writes/encodes the recording. 4. On egress completion or hangup, LiveKit closes the connection; your service finalizes storage, runs transcription, and records audit logs. If you’d like, I can: • Provide a minimal example (pseudo-code) for a secure WSS receiver that converts
pcm_s16le
frames into a WAV file. • Suggest the exact token/metadata fields to include for robust validation. Which would you prefer?
e
So websocket security I can figure out - the last thing is how do I know the room is closed and we can just trigger a compile job on all received frames? I don't want to use the agent end call hook as WarmTransfersTask means that the call can continue - does the WebRTC room have their own hooks that signal the event?
t
Great question — I’ll explain exactly how to know when it’s safe to finalize (compile) the recorded frames, and why you should rely on room lifecycle events instead of agent call hooks. Short answer • Don’t use agent end-call hooks for finalizing recordings — they can fire while the room is still active (e.g., warm transfers). • Use the room lifecycle webhook
room_finished
(fired when the room becomes empty and is closed). That is the reliable signal to trigger your compile job. What to listen for • room_finished: Sent when the last participant leaves and the room is closed. Use the room name or SID in this webhook to correlate with the in‑progress recording for that room. • Participant leave events are useful for progress/monitoring, but only
room_finished
guarantees the whole session is done. Why agent hooks aren’t sufficient • Agent-side hooks like an agent “end call” or session shutdown only reflect that agent’s connection. During warm transfers or other flows, the agent may disconnect while other participants remain in the room, so compiling at that point would be premature. Recommended implementation flow 1. Start ingest and collect frames (via WebSocket egress). 2. Correlate the incoming frames to a room identifier (room.name or room.sid) and an egress job ID. 3. Subscribe to LiveKit webhooks on your secure endpoint and validate signatures. 4. On receiving `room_finished`: ◦ Verify the room ID matches your recording session. ◦ Trigger your compile/finalize job for all frames associated with that room. ◦ Mark the job idempotently (save event id) to avoid double-processing on retries. 1. Finalize storage/transcription and write audit logs. Practical tips • Verify webhook authenticity (validate signature/JWT) before acting. • Respond quickly (2xx) to the webhook and run heavy processing asynchronously. • Use idempotency (store processed webhook IDs) to handle retries safely. • Include room metadata (room name/SID, egress job ID, timestamps) when you start egress so you can match the
room_finished
event precisely. • If you need to be extra cautious, you can also wait a short configurable grace period after
room_finished
before compiling, to allow any in-flight frames to arrive. If you’d like, I can: • Provide a short webhook handler example that validates the event and kicks off an async compile job. • Show how to correlate WebSocket egress connections to room IDs and persist in-progress state until
room_finished
. Which would help you more — a code example for the webhook handler or a template for mapping incoming frames to room/egress IDs? Sources: Server-side | livekit/livekit_composite | DeepWiki | Best practices for managing webhook event streams | LiveKit | server-sdk-js | livekit/livekit_composite | DeepWiki | Python SDKs | livekit/livekit_composite | DeepWiki | Job lifecycle | LiveKit Documentation
e
can a webhook url be registered at the time of room creation? or is it set at the organisation level? we have many environments meaning that we need to do this at room create or by agent
t
Thanks — good question. I’m answering whether webhook URLs can be set per room or per agent (at room creation), and how to handle multiple environments. Short answer • Webhook URLs are configured at the LiveKit project level and apply to all rooms in that project. You cannot register a different webhook URL at room creation or per agent. What that means for multi-environment setups • If you need different webhook endpoints per environment (staging, prod, tenant, etc.), you have two practical options: ◦ Create separate LiveKit projects for each environment. ▪︎ Each project can have its own webhook URL and credentials. ▪︎ This is the cleanest separation for security and routing. ◦ Use a single project and route inside your webhook consumer. ▪︎ Keep one global webhook endpoint. ▪︎ Encode environment/tenant context into the room name or room metadata when you create the room. ▪︎ In your webhook handler inspect
room.name
or
room.metadata
and forward/process events based on that value. Other options to consider • Agent-driven callbacks: Have the agent (or your orchestration service) call environment-specific endpoints directly when it’s appropriate. This is useful for agent-specific workflows but should not replace room lifecycle signals for finalizing recordings. • Idempotency & retries: Webhooks are delivered at‑least‑once; use event IDs to dedupe and process asynchronously to avoid blocking LiveKit retries. Recommended approach • If you can, use separate LiveKit projects per environment. If that’s not possible, put environment/tenant IDs into the room metadata and handle routing in your centralized webhook service. If you want, I can: • Show how to structure room names/metadata for routing patterns, or • Provide a minimal webhook handler example that routes events based on room metadata and demonstrates idempotent processing. Which would be most helpful? Sources: Best practices for managing webhook event streams | LiveKit | Server-side | livekit/livekit_composite | DeepWiki | examples/survey | livekit/livekit_composite | DeepWiki
e
This isn't feasible as we were told that we need to pay for additional plans per environment and therefore we have resorted to using one project for now. Is an alternate way to trigger the event on websocket close? I assume that if there is a reconnection event to write frames we can just recompile when that next connection closes?
t
Thanks — good constraints to call out. I’ll answer directly: yes, using the WebSocket lifecycle is a feasible fallback when you cannot rely on per-room webhooks, but you need to implement it carefully to be reliable and audit-safe. Short answer • You can use the WebSocket close event from LiveKit’s Track Egress → WebSocket to trigger finalization of the recorded frames. • Do not finalize on every disconnect; only finalize on a confirmed end-of-stream (clean egress close) and handle reconnections so you don’t miss frames or double-process. How to do this reliably (recommended pattern) 1. Correlate streams up front ◦ Include
egressId
,
roomName
(or room SID), and an egress job ID in the WebSocket URL or headers (e.g., `wss://ingest.example.com/egress?egressId=xxx&room=myroom`). ◦ Persist an in‑progress record keyed by that egressId/room so you know which frames belong to which job. 1. Treat WebSocket open/close as lifecycle signals ◦ On `open`: mark egress as active. ◦ On `message`: append frames to your buffer/store and persist sequence/timestamp metadata. ◦ On `close`: ▪︎ If close is clean (e.g., code 1000) and the connection includes metadata indicating LiveKit finished sending, treat as end-of-stream and finalize (assemble frames → encode → store → transcribe). ▪︎ If close is abnormal (e.g., 1006) or network error, do NOT immediately finalize. Wait for a reconnection window (see next step). 1. Handle reconnections and partial delivery ◦ Implement a short grace/reconnect window (configurable, e.g., 20–60s) after abnormal close to allow LiveKit to reconnect and continue streaming. ◦ If LiveKit reconnects with the same
egressId
(or provides consistent metadata), resume appending frames to the same job. ◦ If no reconnection occurs within the window, treat the stream as ended and finalize. 1. Make finalization idempotent ◦ Ensure your compile job is idempotent (check if egressId is already processed) so multiple close events or retries don’t create duplicate artifacts. 1. Use robust logging and monitoring ◦ Log open/close codes, timestamps, bytes received, and egressId. ◦ Alert on repeated abnormal closes or incomplete files. 1. Optional stronger signals ◦ If you can, also watch higher-level room/participant state via your orchestration (participant list, token grants) to cross-check whether the room is likely finished. ◦ If you can call
stopEgress
(server API) when you know the room should end, that will reliably stop egress and produce the expected final state. Caveats and limits • WebSocket egress is a delivery channel—there’s no guaranteed at‑least‑once complete delivery of every frame. Plan for possible partial losses. • LiveKit may retry connections; don’t treat the first close as definitive unless it’s a clean end and you validated metadata. • If you can later enable
egress_ended
or
room_finished
webhook handling (single shared webhook with routing), that is still the most authoritative server-side signal that egress was completed. Practical checklist to implement now • Add `egressId`/`roomName` to WSS handshake. • Persist in-progress state keyed by
egressId
. • Finalize only on clean close or after a reconnect timeout. • Make finalization idempotent. • Add monitoring and an alert for abnormal close rates. If you want, I can: • Provide a minimal WebSocket receiver sketch that shows how to tag frames, persist state, and finalize idempotently. • Suggest sensible default reconnect/window values and exact fields to pass in the handshake. Which would you prefer — a code sketch for the receiver or recommended reconnect window values and state model? Sources: Universal Egress | server-sdk-js | livekit/livekit_composite | DeepWiki | Best practices for managing webhook event streams | LiveKit | Universal Egress | server-sdk-ruby | livekit/livekit_composite | DeepWiki
e
I'm reading the egress request and I see that it is done on a participant level. How do I ensure that we start tracks for both the participant / the agent (and all subsequent joiners) and how do we ensure that we only trigger a job to join all tracks when all participants have left?
t
Thanks — that’s a clear, actionable question. I’ll explain how to start track egress for all participants (agent + joiners) and reliably trigger the final join/compile job only after everyone has left. High-level approach • Start a track egress job for each published participant track (agent and others) when that participant publishes. • Track active participants server-side using webhooks (participant_joined / participant_left). • Trigger your final merge/compile job only when the room becomes empty — use the room_finished webhook (or confirm all egress jobs ended via egress_ended) before compiling. Step-by-step pattern 1) Start egress for every participant track • Listen for the track/participant publish event (e.g.,
track_published
or
participant_joined
+ check published tracks). • When a participant publishes a track, call the server-side API to start a Track Egress for that track ID (start one egress job per track). • Persist mapping: egressId -> trackId -> roomName (so you can find all files later). Why: Track Egress records each participant’s track independently and gives you per-track artifacts you can merge later. 2) Maintain active participant state • Subscribe to
participant_joined
and
participant_left
webhooks. • Maintain an in-memory or DB record of currently active participants for each room (list or counter). • Update that record atomically on each webhook (store webhook event id for dedupe). Why: This gives you a reliable view of who’s still in the room independent of agent-side hooks. 3) Detect when the room is fully finished • Rely on
room_finished
webhook — it fires when the last participant leaves and the room is closed. • When you receive
room_finished
for a room: ◦ Confirm all expected egress jobs for that room have reached
egress_ended
(or poll/list egress jobs if needed). ◦ Only then trigger your compile/merge job. Why: Agent end-call events are insufficient (warm transfers, etc.).
room_finished
is the authoritative server-side signal the session is done. 4) Use
egress_ended
to ensure recordings are ready •
egress_ended
indicates an individual egress job has finalized and the file(s) are available. • Before merging, either: ◦ Wait for
egress_ended
for each egressId you started, or ◦ After
room_finished
, check/list egress job statuses and wait for any in-progress egress to finish. Why: Ensures you don’t merge partial files. 5) Make finalization idempotent and robust • Deduplicate webhooks using their unique event id. • Mark jobs processed by roomName + egressId so retries don’t re-run merges. • Return 2xx quickly from webhook handlers and do heavy work asynchronously. Operational tips and edge cases • Late joiners: If a new participant joins after you started merging, your
participant_joined
logic should start a new track egress and you should postpone finalization until
room_finished
again. • Tracking correlation: Put roomName, participantSid, and egressId into your DB record at egress start so you can map incoming
egress_ended
events to the right merge job. • Partial delivery or reconnects (if using WebSocket egress): Don’t treat a temporary WebSocket close as definitive — use reconnection windows and only finalize when egress ends or the room_finished event arrives. • Scale: If many participants publish/unpublish frequently, debounce start/stop operations and keep short-lived egress jobs where sensible (or start egress as soon as track is published to avoid missing early audio). Short example flow 1. Participant A publishes track -> your service calls StartTrackEgress(trackId=A) and saves egressId_A. 2. Participant B (agent) publishes -> StartTrackEgress(trackId=B) -> save egressId_B. 3. During the call, your webhook consumer updates active participant list on join/leave. 4. Last participant leaves -> LiveKit sends
room_finished
. 5. On
room_finished
, your service: ◦ Confirms
egress_ended
for egressId_A and egressId_B (or waits for them). ◦ When all egress jobs are ended, runs your merge/compile job (FFmpeg or similar) to combine per-track files into the final artifact(s). ◦ Stores artifacts in your OIDC-compliant storage and logs audit metadata. If you want, I can: • Provide a minimal webhook handler sketch showing join/leave state tracking and dedupe logic, or • Show an example of how to map egress job results to an FFmpeg merge command for per-track files. Which help would you like next — a webhook state-tracking example or the post-processing merge example? Sources: Universal Egress | Best practices for managing webhook event streams | LiveKit | Backup Egress Recordings | Why is there extra audio at the end of my recordings of an agent session? |...
e
I have used egress - but I am setting tmp tokens - how long should these tokens last? How long does it take for egress to start the bucket upload?
t
Thanks — good questions. I’ll address both: how long your short‑lived (tmp) tokens should last, and when LiveKit begins uploading egress output to storage. Short answers • LiveKit’s egress service begins uploading to your configured storage as soon as the egress session starts (there isn’t a long startup delay). • If you’re issuing temporary credentials that LiveKit (or an intermediary service) will use for uploads, make them valid for the expected egress duration plus a safety buffer — typically at least the maximum expected call length + 30–60 minutes. Details and recommendations 1) When upload starts • Uploads commence effectively when the egress job starts: the egress process boots, captures the media, and begins writing/streaming the output to the configured destination immediately. Plan for upload activity to begin right away once egress is initiated. 2) Token lifetime guidance • If you are handing temporary credentials that are used by some service to store recordings, the credentials must remain valid for the entire time that egress might be active and for a short period afterward while finalization/commit completes. • Practical recommendation: ◦ Token lifetime = expected maximum call/recording length + 30–60 minutes buffer. ◦ Example: if recordings can run up to 2 hours, issue tokens valid for ~2.5–3 hours. • Why the buffer: finalization, encoding, multipart uploads, retries, or post-processing can extend beyond the raw media capture time. 3) If LiveKit is doing the upload directly • You generally do not need to hand LiveKit ephemeral tokens for each job. LiveKit uses the storage credentials or role configuration you provide (or an enterprise AssumeRole setup) to perform uploads. Ensure whatever credentials/role LiveKit uses are valid for the job lifetime. • For enterprises using temporary credentials (AssumeRole), ensure the role/session duration covers the egress job plus the same buffer above. 4) Failure and retry handling • Design for retries and transient failures: if a token expires unexpectedly while an upload is in progress, multipart uploads or retries can fail. Having a buffer prevents this. • Log and monitor egress failures related to authorization so you can detect if tokens are expiring too soon. If you want, I can: • Suggest concrete token durations based on your typical and worst‑case call lengths, or • Describe how to implement an automated token/AssumeRole issuance flow that guarantees coverage for running egress jobs. Which would help you more — a recommended table of token durations for common call lengths, or a sketch of an automated credential refresh/AssumeRole flow? Sources: Security | LiveKit | Universal Egress | LiveKit