microscopic-jackal-49366
05/05/2025, 6:13 AMmicroscopic-jackal-49366
05/05/2025, 6:20 AM<https://github.com/livekit/livekit/tree/master|master>
by boks1971
<https://github.com/livekit/livekit/commit/2f002388b642c656dc82f1f095d0c8d44e81424c|2f002388>
- Use participant close reason in remove. (#3639)
livekit/livekitmicroscopic-jackal-49366
05/05/2025, 10:46 AMmicroscopic-jackal-49366
05/05/2025, 10:52 AM<https://github.com/livekit/livekit/tree/master|master>
by boks1971
<https://github.com/livekit/livekit/commit/8d3902af3647e9985064bfffb8354bb73af537f7|8d3902af>
- Protocol to pick up cgroups v2 memory path fix (#3640)
livekit/livekitmicroscopic-jackal-49366
05/05/2025, 11:21 AMmicroscopic-jackal-49366
05/05/2025, 11:26 AM<https://github.com/livekit/livekit/tree/master|master>
by boks1971
<https://github.com/livekit/livekit/commit/a1f4e88edbd46bc3c12fef3fc0ab56fac2d3328e|a1f4e88e>
- Update protocol to latest, got bit by tag (#3641)
livekit/livekitmicroscopic-jackal-49366
05/05/2025, 1:27 PMmicroscopic-jackal-49366
05/05/2025, 1:40 PM<https://github.com/livekit/livekit/tree/master|master>
by boks1971
<https://github.com/livekit/livekit/commit/b83190a3790bc5e46b6d1a6d6ce499af7b1e952d|b83190a3>
- protocol update to fix memory stats path (#3642)
livekit/livekitmicroscopic-jackal-49366
05/05/2025, 5:17 PMmicroscopic-jackal-49366
05/05/2025, 11:07 PMmicroscopic-jackal-49366
05/06/2025, 4:13 AMmicroscopic-jackal-49366
05/06/2025, 4:21 AM<https://github.com/livekit/livekit/tree/master|master>
by boks1971
<https://github.com/livekit/livekit/commit/d4ab1142cba9cc5f61d96954c8c60ef60373521f|d4ab1142>
- Redact address (#3643)
livekit/livekitmicroscopic-jackal-49366
05/06/2025, 5:50 AMmicroscopic-jackal-49366
05/06/2025, 6:17 AMNode A ── LiveKit server
Node B ── LiveKit server (receives the room request)
Node C ── LiveKit server (worker registers here)
---
## Logs
## Node A
2025-05-06T05:42:55.411Z INFO livekit service/roomallocator.go:164 selected node for room {"room":"<room_name>","selectedNodeID":"<livekit_node_id>"}
2025-05-06T05:42:55.423Z INFO livekit.api service/twirp.go:124 API RoomService.CreateRoom {"service":"RoomService","method":"CreateRoom","room":"<room_name>","request":{"name":"<room_name>","metadata":"<metadata>"},"duration":"16.212838ms","status":"200"}
2025-05-06T05:42:55.596Z INFO livekit.webhook webhook/url_notifier.go:124 sent webhook {"event":"room_started","id":"<event_id>","webhookTime":1746510175,"room":"<room_name>","roomID":"<room_id>","url":"<webhook_url>","queueDuration":"42.041µs","sendDuration":"173.904564ms"}
2025-05-06T05:42:55.646Z INFO livekit agent/client.go:161 failed to send job request {"error":"no workers with sufficient capacity","namespace":"","jobType":"JT_ROOM","agentName":""}
2025-05-06T05:42:56.493Z INFO livekit service/roommanager.go:405 starting RTC session {"room":"<room_name>","roomID":"<room_id>","participant":"<participant_id>","pID":"<process_id>","remote":false,"nodeID":"<livekit_node_id>","numParticipants":0,"participantInit":{"Identity":"<participant_id>","Client":{"sdk":"<sdk>","version":"<sdk_version>","os":"<os>","osVersion":"<os_version>","deviceModel":"<device_model>","browser":"<browser>","browserVersion":"<browser_version>"}}}
2025-05-06T05:42:56.496Z INFO livekit service/roommanager.go:938 created TURN password {"username":"<turn_username>","password":"<turn_pass>"}
2025-05-06T05:42:56.808Z INFO livekit.transport rtc/transport.go:546 ice reconnected or switched pair {"room":"<room_name>","roomID":"<room_id>","participant":"<participant_id>","pID":"<process_id>","remote":false,"transport":"PUBLISHER","existingPair":{"localProtocol":"udp","localCandidateType":"host","localAddress":"<local_ip>","localPort":"<local_port>","remoteProtocol":"udp","remoteCandidateType":"prflx","remoteAddress":"<remote_ip>","remotePort":"<remote_port>"},"newPair":{"localProtocol":"udp","localCandidateType":"host","localAddress":"<local_ipv6>","localPort":"<local_ipv6_port>","remoteProtocol":"udp","remoteCandidateType":"host","remoteAddress":"<remote_ipv6>","remotePort":"<remote_ipv6_port>"}}}
2025-05-06T05:42:56.922Z INFO livekit.pub rtc/participant.go:1826 mediaTrack published {"room":"<room_name>","roomID":"<room_id>","participant":"<participant_id>","pID":"<process_id>","remote":false,"kind":"audio","trackID":"<track_id>","webrtcTrackID":"<webrtc_track_id>","rid":"","SSRC":"<ssrc>","mime":"audio/red","trackInfo":{"sid":"<track_id>","type":"AUDIO","source":"MICROPHONE","mimeType":"audio/red","mid":"1","stream":"camera"},"fromSdp":true}
2025-05-06T05:42:56.957Z INFO livekit.webhook webhook/url_notifier.go:124 sent webhook {"event":"track_published","id":"<event_id>","webhookTime":1346510176,"room":"<room_name>","roomID":"<room_id>","participant":"<participant_id>","pID":"<process_id>","url":"<webhook_url>","queueDuration":"14.513µs","sendDuration":"32.059281ms"}
2025-05-06T05:42:57.704Z INFO livekit rtc/room.go:469 participant active {"room":"<room_name>","roomID":"<room_id>","participant":"<participant_id>","pID":"<process_id>","remote":false,"publisherCandidates":"<redacted>","subscriberCandidates":"<redacted>","connectionType":"udp"}
2025-05-06T05:42:57.744Z INFO livekit.webhook webhook/url_notifier.go:124 sent webhook {"event":"participant_joined","id":"<event_id>","webhookTime":1346510177,"room":"<room_name>","roomID":"<room_id>","participant":"<participant_id>","pID":"<process_id>","url":"<webhook_url>","queueDuration":"34.951µs","sendDuration":"39.824442ms"}
### Node B
2025-05-06T05:42:55.632Z WARN livekit.agents service/agentservice.go:401 failed to assign job to worker {"jobID": "<job_id>", "namespace": "", "agentName": "", "jobType": "JT_ROOM", "room": "<room_name>", "roomID": "<room_id>", "workerID": "<OLD_WORKER_ID_1>", "retry": true, "error": "worker not available"}
2025-05-06T05:42:55.639Z WARN livekit.agents service/agentservice.go:401 failed to assign job to worker {"jobID": "<job_id>", "namespace": "", "agentName": "", "jobType": "JT_ROOM", "room": "<room_name>", "roomID": "<room_id>", "workerID": "<OLD_WORKER_ID_2>", "retry": true, "error": "worker not available"}
2025-05-06T05:42:55.644Z WARN livekit.agents service/agentservice.go:401 failed to assign job to worker {"jobID": "<job_id>", "namespace": "", "agentName": "", "jobType": "JT_ROOM", "room": "<room_name>", "roomID": "<room_id>", "workerID": "<OLD_WORKER_ID_3>", "retry": true, "error": "worker not available"}
2025-05-06T05:42:55.644Z WARN livekit.agents service/agentservice.go:391 no worker available to handle job {"jobID": "<job_id>", "namespace": "", "agentName": "", "jobType": "JT_ROOM", "room": "<room_name>", "roomID": "<room_id>", "error": "no workers with sufficient capacity"}
### Node C
2025-05-06T05:42:44.567Z INFO livekit.agents service/agentservice.go:292 worker registered {"namespace": "", "jobType": "JT_ROOM", "agentName": "", "workerID": "<CORRECT_WORKED_ID>"}
---
## Expected behavior
• Node B should be able to dispatch the JT_ROOM
job to the worker already
registered on Node C, or the cluster should transparently proxy the request.
## Actual behavior
• Node B fails to find any available worker and eventually the allocator
selects Node A, which creates the room without an Agent worker.
---
## Reproduction steps
1. Run three LiveKit nodes (A, B, C) backed by the same Redis/state store (running in sentinel with one master node).
2. Start one Agent worker only on Node C (job type JT_ROOM
).
3. Send a CreateRoom
request that the allocator routes to Node B.
4. Observe worker-not-available warnings on Node B and room creation on Node A.
---
## Environment
| Component | Version |
| -------------- | --------------------------------------- |
| LiveKit server | v2.8.0 |
| LiveKit Agent | v0.x (worker) |
| Redis | 7.2 |
| Deployment | Docker Compose (one container per node) |
| OS | Ubuntu 22.04 |
---
## Question
Why can’t Node B delegate the job to the registered worker on Node C?
Is worker discovery limited to the local node by design, or is there an extra
configuration flag required to make workers globally visible across the cluster?
Any insight or pointers would be greatly appreciated!
livekit/livekitmicroscopic-jackal-49366
05/06/2025, 9:00 AM.unspecified
position
livekit/client-sdk-swiftmicroscopic-jackal-49366
05/06/2025, 11:06 AMTrack
vs Room
attributes
• Keeps LocalTrack
identity while recording/publishing (required by the above ⬆️)
• Adds misc checks for robustness (e.g. don't send to non-agent participants)
• Adds new scoped API for easier cleanup on errors
livekit/client-sdk-swiftmicroscopic-jackal-49366
05/07/2025, 12:07 PMmicroscopic-jackal-49366
05/07/2025, 2:09 PMmicroscopic-jackal-49366
05/07/2025, 4:45 PMmicroscopic-jackal-49366
05/07/2025, 5:00 PMmicroscopic-jackal-49366
05/07/2025, 5:06 PM<https://github.com/livekit/livekit/tree/master|master>
by boks1971
<https://github.com/livekit/livekit/commit/2fff36cb35364e6cccea3df2e46d902cb9944e82|2fff36cb>
- Stub MoveParticipant so that cloud can include the latest protocol. (#3646)
livekit/livekitmicroscopic-jackal-49366
05/07/2025, 5:44 PMprocess exited with non-zero exit code -4
This error is triggered when the supervised process terminates with SIGILL (signal 4), which indicates an illegal instruction was executed. The error happens in supervised_proc.py
around line 280-283:
if self._exitcode != 0 and not self._kill_sent:
logger.error(
f"process exited with non-zero exit code {self.exitcode}",
extra=self.logging_extra(),
)
## Technical Analysis
The error appears to be related to the Foreign Function Interface (FFI) used for audio processing. When audio input is detected from the microphone:
1. Audio data is captured and passed to the audio processing modules
2. This data is processed through AudioProcessingModule
in apm.py
3. The module calls native code via FFI in _ffi_client.py
4. When attempting to execute CPU instructions in the native code, a SIGILL is triggered
5. The supervised process is terminated before any user state change events can be emitted
The most likely cause is a CPU architecture mismatch between the compiled native FFI library (`liblivekit_ffi.so`/`.dylib`/`.dll`) and the current hardware, or the use of CPU instructions not supported by the current processor.
## Impact
• Audio processing fails completely when users attempt to speak
• Speech-to-text functionality is unavailable
• Real-time voice communication is broken
## Additional Information
The error only occurs when audio input is first processed, suggesting it's related to the initialization of audio processing components or the first execution of specialized CPU instructions for audio processing.
## Possible Solutions to Investigate
1. Provide compatible builds of the LiveKit FFI libraries for different CPU architectures
2. Add runtime detection of CPU capabilities and use fallback implementations when specific instructions aren't available
3. Set the LIVEKIT_LIB_PATH
environment variable to point to a compatible library version
4. Implement more robust error handling in the audio processing pipeline to prevent crashes
## Logs and Diagnostics
Agent state changed: from thinking to speaking
Metrics collected
type='metrics_collected' metrics=TTSMetrics(type='tts_metrics', label='livekit.plugins.google.tts.TTS', request_id='06a4e98e246e', timestamp=1746639797.2195292, ttfb=0.22073470099712722, duration=0.2774173969955882, audio_duration=2.112833333333335, cancelled=False, characters_count=37, streamed=False, speech_id='speech_ccc7eb5cfab0')
2025-05-07 174317,474 - httpcore.http11 - DEBUG - receive_response_body.complete
2025-05-07 174317,474 - httpcore.http11 - DEBUG - response_closed.started
2025-05-07 174317,474 - httpcore.http11 - DEBUG - response_closed.complete
Metrics collected
type='metrics_collected' metrics=LLMMetrics(type='llm_metrics', label='livekit.plugins.openai.llm.LLM', request_id='chatcmpl-BUd5YRsHyNdWIy7pl3OJbeMX7uc7V', timestamp=1746639797.4746995, duration=1.7644396450050408, ttft=1.0211968020012137, cancelled=False, completion_tokens=62, prompt_tokens=2076, prompt_cached_tokens=0, total_tokens=2138, tokens_per_second=35.13863462290481, speech_id='speech_ccc7eb5cfab0')
Metrics collected
type='metrics_collected' metrics=TTSMetrics(type='tts_metrics', label='livekit.plugins.google.tts.TTS', request_id='ec7f88af0a11', timestamp=1746639797.9792652, ttfb=0.40468507799960207, duration=0.7448762679996435, audio_duration=7.512416666666596, cancelled=False, characters_count=156, streamed=False, speech_id='speech_ccc7eb5cfab0')
Metrics collected
type='metrics_collected' metrics=TTSMetrics(type='tts_metrics', label='livekit.plugins.google.tts.TTS', request_id='66994be7d3e4', timestamp=1746639798.5305398, ttfb=0.24503060399729293, duration=0.5509810129951802, audio_duration=5.972833333333295, cancelled=False, characters_count=116, streamed=False, speech_id='speech_ccc7eb5cfab0')
Agent state changed: from speaking to listening
Chat History updated
["assistant: Hello Himesh, it's great to meet you! Before we dive into the technical questions, could you please tell me a bit about what motivated you to pursue a degree in Computer Science and Engineering? \n\nAdditionally, what have been some of your most memorable experiences or achievements during your time at university?"]
Metrics collected
type='metrics_collected' metrics=STTMetrics(type='stt_metrics', label='livekit.plugins.deepgram.stt.STT', request_id='', timestamp=1746639812.839311, duration=0.0, audio_duration=0.05, streamed=True)
Metrics collected
type='metrics_collected' metrics=VADMetrics(type='vad_metrics', label='livekit.plugins.silero.vad.VAD', timestamp=1746639813.7896044, idle_time=18.08289429100114, inference_duration_total=0.01760541099793045, inference_count=32)
Metrics collected
type='metrics_collected' metrics=VADMetrics(type='vad_metrics', label='livekit.plugins.silero.vad.VAD', timestamp=1746639814.8101447, idle_time=19.103434784003184, inference_duration_total=0.01616592599748401, inference_count=32)
2025-05-07 174336,281 - livekit.agents - ERROR - process exited with non-zero exit code -4
2025-05-07 174336,281 - ERROR livekit.agents - process exited with non-zero exit code -4 {"pid": 37275, "job_id": "AJ_XTg747wkhjik"}
livekit/livekitmicroscopic-jackal-49366
05/08/2025, 3:30 AMmicroscopic-jackal-49366
05/08/2025, 4:58 AM<https://github.com/livekit/livekit/tree/master|master>
by cnderrauber
<https://github.com/livekit/livekit/commit/793b383a5292b23df0a1e79fcd989b0af404b8fa|793b383a>
- Add Moving participant to another room (#3648)
livekit/livekitmicroscopic-jackal-49366
05/08/2025, 6:01 AMmicroscopic-jackal-49366
05/08/2025, 9:54 AM@session.on("conversation_item_added")
, I've noticed that the user messages are not caught by the event.
Server
• Version: 1.0.6
• Environment: local
• any other information about your deployment setup
Client
• SDK: Agents Playground & Local console
• Version: [0.x.x]
To Reproduce
Steps to reproduce the behavior:
1. Set up Gemini Realtime model (google.beta.realtime.RealtimeModel)
2. Add an event handler for session for "conversation_item_added"
3. Check logs and see only agent events, no user events
Expected behavior
I expected the user messages to be invoked as well, as it's mentioned in the documentation by this: "A ConversationItemAddedEvent is emitted when a item is committed to the chat history. This event is emitted for both user and agent items.".
Additional context
Code:
import logging
import os
from dotenv import load_dotenv
from livekit import agents
from livekit.agents import (
Agent,
AgentSession,
AutoSubscribe,
ConversationItemAddedEvent,
JobContext,
RoomInputOptions,
)
from livekit.agents.llm import AudioContent, ImageContent
from livekit.plugins import google, noise_cancellation
from mem0 import AsyncMemoryClient
# Setup logging
logger = logging.getLogger("agent")
load_dotenv()
# Initialize Mem0 client
memory_client = AsyncMemoryClient(api_key=os.getenv("MEM0_API_KEY"))
# Voice configurations
INTRO_VOICE = "Leda" # Friendly, welcoming voice
TUTOR_VOICE = "Puck" # More authoritative, teaching voice
class TestAssistant(Agent):
def __init__(self) -> None:
super().__init__(instructions="You are a helpful voice AI assistant.")
async def entrypoint(ctx: JobContext):
<http://logger.info|logger.info>("Starting entrypoint function")
try:
await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
<http://logger.info|logger.info>("Connected to room successfully")
await ctx.wait_for_participant()
session = AgentSession(
llm=google.beta.realtime.RealtimeModel(
model="gemini-2.0-flash-exp",
voice=INTRO_VOICE,
temperature=1.0,
),
)
@session.on("conversation_item_added")
def on_conversation_item_added(event: ConversationItemAddedEvent):
print(
f"Conversation item added from {event.item.role}: {event.item.text_content}. interrupted: {event.item.interrupted}"
)
# Start with the IntroAgent
await session.start(
room=ctx.room,
agent=TestAssistant(),
room_input_options=RoomInputOptions(
noise_cancellation=noise_cancellation.BVC()
),
)
except Exception as e:
logger.error(f"Error in agent setup or execution: {e}", exc_info=True)
if __name__ == "__main__":
agents.cli.run_app(agents.WorkerOptions(entrypoint_fnc=entrypoint))
livekit/livekitmicroscopic-jackal-49366
05/08/2025, 3:34 PMmicroscopic-jackal-49366
05/08/2025, 3:43 PM<https://github.com/livekit/livekit/tree/master|master>
by paulwe
<https://github.com/livekit/livekit/commit/aee34ffefad89f135378bf11b8461e866d83e01b|aee34ffe>
- log request for agent dispatch api (#3650)
livekit/livekitmicroscopic-jackal-49366
05/08/2025, 5:06 PM2025-05-08T15:45:29.935Z WARN livekit service/auth.go:95 error handling request {"status": 401, "method": "GET", "path": "/rt
c/validate", "error": "invalid token: eyJhbGciOiJIUzI1<redact-shortted>MF00w, error: go-jose/go-jose: err
or in cryptographic primitive"}
livekit/livekitmicroscopic-jackal-49366
05/08/2025, 5:25 PM# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 206
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0
livekit/livekit