Hi everyone, I’m integrating Gemini Live API with LiveKit Agents to build a real-time AI audio/video calling assistant. However, I'm running into several issues and I'm not sure whether they are caused by my code or by the Livekit/Gemini Live integration behavior. Problems I'm experiencing 1. Very slow response time 2. – Sometimes the AI takes a long time to respond, or never responds at all. 3. Audio glitches during AI speech 4. – I hear noticeable pops/clicks in the synthesized audio output. 5. Can't interrupt the AI by speaking 6. – While the AI is talking, my speech does not interrupt it (no turn-taking). 7. – It seems VAD / turn detection is not triggering. Dependency versions I’m using

livekit-agents[silero,turn-detector,google,images,mcp] ~= 1.35
livekit-plugins-noise-cancellation ~= 0.25

Here is my agent code,deploy in livekit cloud.

import copy
import logging
import os
from typing import Any

from dotenv import load_dotenv
from livekit import rtc
from livekit.agents import (
    Agent,
    AgentServer,
    AgentSession,
    JobContext,
    cli,
    mcp,
    room_io,
)
from livekit.agents.llm.mcp import MCPTool
from livekit.plugins import google, noise_cancellation

logger = logging.getLogger("agent")

load_dotenv(".env.local")

class VideoAssistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a helpful voice AI assistant",
            llm=google.realtime.RealtimeModel(
                model="gemini-2.5-flash-native-audio-preview-09-2025",
                voice="Kore",
                temperature=0.8,
            ),
        )


server = AgentServer()

@server.rtc_session()
async def my_agent(ctx: JobContext):
    ctx.log_context_fields = {
        "room": ctx.room.name,
    }

    session = AgentSession()

    await session.start(
        agent=VideoAssistant(),
        room=ctx.room,
        room_options=room_io.RoomOptions(
            video_input=True,
            audio_input=room_io.AudioInputOptions(
                noise_cancellation=lambda params: noise_cancellation.BVCTelephony()
                if params.participant.kind == rtc.ParticipantKind.PARTICIPANT_KIND_SIP
                else noise_cancellation.BVC(),
            ),
        ),
    )

    await ctx.connect()

    await session.generate_reply(instructions="please introduce yourself.")
    

if __name__ == "__main__":
    cli.run_app(server)

My Question Based on the issues above: Is there any mistake in my code or configuration? Or are these limitations/bugs related to the current Gemini Live realtime API integration (latency, turn detection, or TTS stability)? Any help or guidance would be greatly appreciated!

hey, i have tried your code, it looks good to me (at least it s working for me in dev mode). Is it possible to know what region you are in and is this mainly telephony so I can test on this scenario too

Thanks for the reply! My agent is deployed in eu-central. I also tested it in dev mode, and the performance was still quite poor. For testing, I used https://agents-playground.livekit.io/ from Sydney, Australia. Our final use case is real-time audio/video calling inside a mobile app (similar to GPT Live or Gemini Live). Our customers are located in Sydney, and we are currently in the technical validation stage.

ya we have had some issues with gemini live specificlly, hopefully i have some times to look at this today

1. Currently, we still have two major issues: 2. The AI voice response is almost impossible to interrupt, which greatly affects the natural conversational experience. Since Gemini enables Automatic VAD by default, I’m not sure how to properly handle turn interruption. 3. The AI response delay is still over 3 seconds, which negatively impacts real-time interaction.

As of late, I have also experienced Gemini realtime turn detection being very poor and I just can't interrupt the AI voice. @bored-egg-9547 Can you try using livekit's turn detection as shown here. https://docs.livekit.io/agents/models/realtime/plugins/gemini/#turn-detection Let me know if lk turn detection works for you. @bored-egg-9547 Incase you're having ay issues with gemini model calling/using tools, I have found that this gemini model

gemini-live-2.5-flash-preview

performed well with tool calling.

session = AgentSession( turn_detection=MultilingualModel(), llm=google.realtime.RealtimeModel( model="gemini-2.5-flash-native-audio-preview-09-2025", voice="Kore", temperature=0.8, realtime_input_config=types.RealtimeInputConfig( automatic_activity_detection=types.AutomaticActivityDetection( disabled=True, ), ), input_audio_transcription=None, stt="assemblyai/universal-streaming", ), mcp_servers=mcp_servers, ). The AI stopped responding after the modification.😅

sorry for getting back to you late, can you confirm on the agent version and see if updating to the latest version helps, i notice some warnings in my console but they are gone now so wondering if that could help

用的是最新的1.35

@bored-egg-9547 i have DM'ed you requesting more information