Thanks — this is very clear and helpful. I’ll respond to the two asks separately and then ask one quick clarification to ensure the example fits your environment. Summary of what you reported • livekit-agents 1.3.5 + livekit-plugins-aws 1.3.5 uses awscrt for Transcribe streaming and on Linux/amd64 the awscrt C-extension repeatedly holds the GIL during active Transcribe streaming, causing 0.5–6s asyncio event-loop stalls every ~16s. • Disabling STT removes the stalls, so you must replace Transcribe streaming with a pure-Python async client (aiobotocore/aioboto3) while staying on AWS. • You want a drop-in STT subclass for AgentSession that uses TranscribeStreamingClient (aiobotocore) and emits SpeechEvent objects (START_OF_SPEECH, INTERIM_TRANSCRIPT, FINAL_TRANSCRIPT, END_OF_SPEECH) correctly. • You also asked whether WorkerOptions supports process isolation (num_workers or similar) so awscrt blocking could be isolated. Quick clarification • Do you have the livekit-agents source code available locally (so you can confirm exact class names / imports), or should I provide a best-effort example based on the livekit-agents 1.3.5 public API surface (typical base-class names:

STT

,

SpeechEvent

,

SpeechEventType

,

AgentSession

)? If you prefer, I can return a small patch you can adapt to your repo. What I can deliver next • A minimal, focused example of a custom STT subclass that: ◦ Implements the STT constructor and required async methods (init and streaming/recognize entry points), ◦ Uses aiobotocore/aioboto3 TranscribeStreamingClient to open a streaming session, ◦ Reads PCM audio frames pushed by AgentSession and sends them to Transcribe, ◦ Receives interim and final transcripts and emits SpeechEvent objects into the pipeline with the correct SpeechEventType values, ◦ Demonstrates clean session start/reconnect and shutdown handling to avoid resource leaks. Notes up front about constraints and tradeoffs • The example will assume: ◦ audio sample rate and encoding you gave (sample_rate=8000), and that AgentSession delivers PCM frames suitable for Transcribe (Transcribe expects 16-bit PCM; if AgentSession provides a different encoding, conversion will be necessary). ◦ IAM credentials from IMDS are usable by aiobotocore (they are). ◦ aiobotocore/aioboto3 streaming client is present in your environment and works with your Transcribe region. • You’ll still need to test for latency/throughput; a pure-Python client avoids awscrt’s GIL grabs but may have different performance characteristics. Process-isolation / WorkerOptions • From the information available, there is no confirmed documented

num_workers

or built-in process-isolation option that will automatically offload an STT session into a separate process. If you want to isolate awscrt’s GIL grabs from your main event loop, two viable alternatives are: ◦ Run the STT work in a separate process (e.g., use multiprocessing or spawn a small worker process that handles the awscrt-based Transcribe client and communicates with the main agent via IPC or a socket). That isolates GIL/C-extension blocking completely. ◦ Use a pure-Python async client (aiobotocore/aioboto3) as you plan; that should avoid the awscrt C-extension and therefore the GIL blocking. • If you prefer, I can provide a short example showing how to spawn a subprocess worker for STT and forward audio/events over an asyncio Stream or a simple UNIX socket. Proposed minimal STT subclass example (what I will provide if you confirm) 1. STT.*init*(self, *, sample_rate, ...): ◦ store config (sample rate, language, aio session/client config) ◦ set up internal queues: one asyncio.Queue for incoming audio frames and one for shutdown signaling 1. STT.stream / STT._recognize_impl (async entrypoint called by AgentSession): ◦ start aiobotocore TranscribeStreamingClient in an async context ◦ concurrently: a. read audio frames from self._audio_queue and send them to Transcribe as a stream of AudioEvent bytes b. read events/responses from Transcribe and convert them to SpeechEvent objects: • on first non-empty audio received / on first transcript hypothesis -> push a SpeechEvent with SpeechEventType.START_OF_SPEECH • on interim hypotheses -> push SpeechEvent with SpeechEventType.INTERIM_TRANSCRIPT (set partial=True if needed) • on final transcripts -> push SpeechEvent with SpeechEventType.FINAL_TRANSCRIPT (include text and confidence if available) • when Transcribe session ends -> push SpeechEventType.END_OF_SPEECH and return • ensure graceful shutdown on cancellation or AgentSession stop 1. Pushing SpeechEvent into pipeline: ◦ the STT implementation should call the same method AgentSession expects for receiving SpeechEvent objects (usually a callback or a provided event sink). In the example I will show pushing into an async callback named

event_sink(event)

which you’ll wire to AgentSession’s receiver (if AgentSession expects a different method, I’ll adapt once you confirm the exact import path/signature). 1. Concurrency and backpressure: ◦ send audio frames as soon as they arrive into Transcribe. If AgentSession can produce faster than network, use a bounded

asyncio.Queue

and drop/compact frames or apply flow control to avoid memory growth. 1. Example will use aioboto3/aiobotocore syntax for TranscribeStreamingClient with async generator style for sending audio. Next step Please confirm one of the following so I can produce the concrete code: • A) Proceed with a best-effort minimal STT subclass example targeting livekit-agents 1.3.5 public API names (STT base class, SpeechEvent/SpeechEventType), and using an async

event_sink(event)

callback to deliver SpeechEvent objects; OR • B) Provide the path to your local livekit-agents STT base class (or paste the relevant base-class method signatures for

STT

) so I can match method names exactly to your code; OR • C) You’d rather see a small subprocess-isolation example (spawned worker that runs awscrt Transcribe) instead of the aiobotocore approach. Once you pick A, B, or C I’ll return the minimal code example (async class, aiobotocore usage, and precise guidance on emitting SpeechEvent objects) ready for you to adapt.