does gemini live with modalities = [TEXT] actually work? It crashes every time for me

Thanks for reaching out, Trevor. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Thanks — good to know you're on

gemini-3.1-flash-live-preview

. That model has known limitations with mid-session methods which can cause failures if the session calls unsupported APIs. To pinpoint the crash, could you share the following: • The exact error message or stack trace you see (copy/paste). • Whether you configured a native-audio model or a non-native-audio model. • Runtime environment: LiveKit Cloud or self-hosted, and language/runtime (Python or Node.js) and versions. • A minimal code snippet showing how you create the session and set

modalities = [TEXT]

(redact secrets). Once I have that I’ll identify the likely cause and recommend the next steps. , with interactive elements

example: """ POC: minimal LiveKit agent using gemini-3.1-flash-live-preview in TEXT modality with Cartesia TTS for voice output. Run: python src/poc_live_text.py console Requires in .env.local: GOOGLE_API_KEY CARTESIA_API_KEY LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET """ from future import annotations from dotenv import load_dotenv from google.genai import types as genai_types from livekit.agents import Agent, AgentSession, JobContext, WorkerOptions, cli from livekit.plugins import cartesia from livekit.plugins.google.realtime.realtime_api import ( RealtimeModel, RealtimeSession, ) load_dotenv(".env.local", override=True) class _TextRealtimeSession(RealtimeSession): """ Workaround: the plugin unconditionally attaches speech_config. In TEXT mode the backend rejects that with 1007. Strip it when AUDIO isn't in the modalities list. """ def _build_connect_config(self) -> genai_types.LiveConnectConfig: conf = super()._build_connect_config() if genai_types.Modality.AUDIO not in (conf.response_modalities or []): conf.speech_config = None return conf class TextRealtimeModel(RealtimeModel): def session(self) -> _TextRealtimeSession: sess = _TextRealtimeSession(self) self._sessions.add(sess) return sess INSTRUCTIONS = ( "You are a friendly assistant on a live phone call. " "When the user speaks, respond briefly and naturally. " "Keep responses to one or two short sentences." ) class POCAgent(Agent): def __init__(self) -> None: super().__init__( instructions=INSTRUCTIONS, llm=TextRealtimeModel( model="gemini-3.1-flash-live-preview", temperature=0.7, modalities=[genai_types.Modality.TEXT], ), tts=cartesia.TTS( model="sonic-3", voice="1fc31370-81b1-4588-9c1a-f93793c6e01d", ), ) async def entrypoint(ctx: JobContext) -> None: await ctx.connect() session = AgentSession() await session.start(agent=POCAgent(), room=ctx.room) if name == "__main__": cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

Thanks — I see your POC and that you’re running

gemini-3.1-flash-live-preview

in TEXT mode with Cartesia TTS. That model has known mid-session limitations which can cause failures when unsupported fields or methods are used (your workaround to strip

speech_config

is a good sign you’re hitting those boundaries). To diagnose the crash quickly, please paste: • The exact error message or full stack trace you get when it crashes. • Your Python version and whether you’re running on LiveKit Cloud or self-hosted. • Confirm the

.env.local

keys you set (no secrets) show you’re using a non‑native‑audio model (i.e., not a native-audio variant). If you can’t paste the full trace, tell me whether the crash happens: • Immediately on session start, or • After a user speaks / after the agent calls any mid-session methods (like

generate_reply()

,

update_instructions()

, etc.). With those details I’ll recommend the precise workaround or next steps. , with interactive elements

@red-dentist-96118 in case you didn’t already come to the conclusion, the Gemini models don’t support TEXT only output: https://github.com/googleapis/python-genai/issues/1780#issuecomment-4136452732

thanks, yeah too bad.. google voices aren't quite as good as cartesia's but their live models are lovely