Thanks — I hear your concern about the LLM node in LiveKit Inference stopping/responding and effectively crashing your voice agent pipeline. I’ll summarize what is known, likely causes, and practical next steps you can take to diagnose and mitigate this. What’s known • There is no documented, inherent LiveKit bug that causes the LLM node itself to stop responding and crash the entire voice agent pipeline under normal operation. • The LLM node performs inference as part of the STT → LLM → TTS pipeline and returns text/ChatChunk objects used by downstream nodes. • If an individual job process crashes, that job is terminated but the agent server and other jobs remain running. If the agent server process crashes, all child jobs (including the voice pipeline) will be terminated. Possible causes that can look like an LLM crash • LLM model errors (for example, some Gemini models can return

MAX_TOKENS

or related errors when

max_output_tokens

is set too low). That will result in a failed response which can appear as the node “stopping.” • Resource exhaustion or worker pool overload on the agent server can cause job failures or unexpected terminations of child processes. • Upstream STT failures (e.g., aggressive noise cancellation) may suppress speech input and make the pipeline appear stuck even though the LLM node is functioning. • A crash of the agent server process will terminate all running jobs, making it seem like the LLM node brought down the pipeline. Practical diagnostic steps 1. Check the agent server and job logs around the time of the failure. ◦ Look for job-level crash traces versus agent-server-level errors. If the agent server crashed, you’ll see that parent process termination. 1. Inspect LLM responses and error messages. ◦ Look for token-limit errors (e.g.,

MAX_TOKENS

) or other model errors. If present, increase or omit

max_output_tokens

. 1. Monitor resource usage on the agent server. ◦ CPU, memory, and process counts can reveal worker exhaustion or out-of-memory crashes. 1. Verify STT input quality. ◦ Disable or test without noise cancellation to ensure the STT is receiving speech; missed utterances can halt downstream flow. 1. Reproduce with a minimal test job. ◦ Run an isolated STT→LLM→TTS job with logging enabled to capture where the pipeline stops. Mitigations • Adjust LLM parameters: increase or remove

max_output_tokens

to avoid hidden token limits (known with some Gemini models). • Scale or tune the agent server worker pool and host resources to reduce job failures due to resource exhaustion. • Temporarily disable noise suppression to confirm STT is not dropping speech. • Add robust error-handling/retries for LLM node failures so a single failed inference doesn’t block the whole pipeline. If you’d like, I can help further — please tell me: • Are you seeing specific error messages (logs) when the LLM node “stops”? Paste a short excerpt if possible. • Is the entire agent server restarting, or only individual jobs failing? • Which LLM model and

max_output_tokens

settings are you using? With that info I can suggest targeted fixes or steps to reproduce and isolate the issue. Sources: Pipeline nodes and hooks | LiveKit Documentation | Self-hosted deployments | LiveKit Documentation | Troubleshooting STT not picking up utterances | LiveKit