I am not able to use gemini 3.1 flash live model with other external TTS

Thanks for reaching out, N. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i am using

gemini-3.1-flash-live and modality is se t to text

Thanks — that’s helpful to confirm you’re on

gemini-3.1-flash-live

with

modalities

set to

["text"]

. I know it’s frustrating when it still doesn’t behave as expected. I’ll give a concise example of the minimal setup to run Gemini in text-only mode and forward its output to an external TTS, then ask a couple of targeted questions so I can pinpoint what’s going wrong for you. Minimal example (conceptual) 1. Start the realtime/agent session with the model forced to text-only: ◦ Ensure the model is initialized with

modalities: ["text"]

so it does not attempt to produce audio itself. 1. Capture the model’s text responses and send them to your TTS: ◦ Listen for the model’s text response events (websocket, stream, callback). ◦ When a text response arrives, pass that text to your external TTS API and play the returned audio to the client. Example flow (pseudo-code) 1. Initialize session: ◦ model:

gemini-3.1-flash-live

◦ modalities:

["text"]

1. On model text event: ◦ receive

response_text

◦ call your TTS:

tts_audio = externalTTS.synthesize(response_text)

◦ play

tts_audio

to the user (via WebRTC or audio output path) Common causes when integration still fails • The session wasn’t actually created with

modalities

applied (typo, config in the wrong place). • The model variant in your deployment does not fully support text-only mode (some live builds differ). • Your code isn’t correctly handling the model’s text event (listening on the wrong channel or missing the callback). • The external TTS call succeeds but the playback path to the user is broken (wrong audio format/codec, or not attached to the peer connection). Next, to debug quickly I need two things from you: 1. The exact code/config snippet you use to start the realtime session (show where

modalities

is set). 2. Which external TTS provider and integration method you’re using (API name, and how you play the returned audio back to the client). If you prefer, paste any error messages or logs you see when the session starts or when responses arrive. With those I’ll give a tailored fix (including a small runnable example for Node or Python if that would help). , with interactive elements

tts=elevenlabs.TTS( _# voice_id="ZpTI0aqcB2Rtm1QT8B2W", # Yoshi voice_ _# voice_id="uMV6YgE1ZWYMf689bcBg", # Koika voice_ _voice_id_="TWZoRfAjoUgBZ0WXZhrd", # prasad voice _# voice_id="SZfY4K69FwXus87eayHK",_ model="eleven_multilingual_v2" ), llm=google.realtime.RealtimeModel( model="gemini-3.1-flash-live-preview", modalities=[Modality.TEXT] ),

Thanks — this is great detail. You’re using

gemini-3.1-flash-live-preview

with

modalities=[Modality.TEXT]

and ElevenLabs

eleven_multilingual_v2

with

voice_id="TWZoRfAjoUgBZ0WXZhrd"

. I’ll walk through the most likely causes and give focused checks and a minimal flow to verify the integration. What I’m addressing: why Gemini still isn’t working with ElevenLabs TTS even though

modalities

is set to

["text"]

. Quick checklist (please verify each) • Confirm the model actually started with text-only modality: ◦ Check the session startup logs or the response from the realtime API to ensure it shows

modalities: ["text"]

and no audio negotiation. • Ensure you are receiving the model’s text output: ◦ Verify your event handler (websocket/callback) receives plain text responses. If you’re not getting text events, the pipeline to your handler may be wrong. • Confirm ElevenLabs TTS call succeeds and returns playable audio: ◦ Log the HTTP response from ElevenLabs (status code, headers). ◦ Ensure the returned audio format matches what you expect (e.g., WAV/MP3) and is not empty. • Verify playback path to the user: ◦ If you stream audio into a WebRTC connection, confirm you convert the ElevenLabs audio to the required codec/sample rate and inject it on the correct track. ◦ If you play audio locally in the browser, ensure the client receives the audio buffer and calls play() without cross-origin or autoplay issues. Minimal functional flow to test locally 1. Start Gemini session with

modalities: ["text"]

. 2. On each text response event: ◦ Log the exact text payload you receive (timestamp + content). ◦ Call ElevenLabs TTS with that text and log the response (status, content-length). 1. Save the returned audio to disk (or serve it) and play it back locally to verify ElevenLabs produced valid, audible audio before routing it into your application. 2. If audio is valid locally, inject it into your app’s playback path (WebRTC track or HTML audio element) and confirm playback. Targeted debugging questions for you 1. Do you see the model’s text responses in your logs? If yes, paste one (sanitized) example. 2. Does the ElevenLabs API call return success (200) and a non-zero audio payload? If not, paste the error/response. 3. When you try to play returned audio in the client, what happens? (no sound, error, silent file, blocked by browser) 4. Are you streaming the ElevenLabs audio into a WebRTC peer connection or using an audio element on the client? Which? Common pitfalls specific to ElevenLabs + live models • The model is text-only but your client still expects an audio track negotiated at session start; that mismatch can lead to your code waiting on audio that never arrives. • ElevenLabs may return an audio format or sample rate incompatible with your WebRTC pipeline; a transcode step (e.g., to 16 kHz PCM or Opus) is often required before injecting into a peer connection. • Autoplay/browser policies can block playback unless the user interacted with the page first. If you share: • A short snippet of the event handler that receives Gemini responses, and • The code you use to call ElevenLabs and how you attempt to play the returned audio (WebRTC injection or client playback), I’ll provide a small, concrete example for Node or browser showing the exact steps to fetch ElevenLabs audio and play it into a WebRTC path (including a recommended transcode if needed). , with interactive elements

websockets.exceptions.ConnectionClosedError: received 1011 (internal error) Internal error encountered.; then sent 1011 (internal error) Internal error encountered. i am getting following error when i use modality text

Thanks — I’m sorry you’re hitting that 1011 internal error when you set

modalities

to

["text"]

. I know that’s frustrating. I’ll explain the most likely causes and give clear next steps and checks so we can narrow this down quickly. What the error suggests • A websocket 1011 (internal error) often means the realtime service encountered an unexpected condition while negotiating or starting the session. • One common reason here is that the specific live model build you’re using (

gemini-3.1-flash-live-preview

) may not fully support running in text-only mode. If the model cannot switch off native audio generation, the service can fail during session startup. Immediate checks to run 1. Confirm the session startup response/logs ◦ When you create the realtime session, capture and paste the full startup response or server logs around the time of the websocket close. Look for any fields mentioning supported modalities or audio negotiation errors. 1. Try a non-preview model or alternate build ◦ If you can, attempt the same call with a different Gemini live model variant (non-preview if available) or a known non-audio model. If that succeeds, it points to incompatibility with the preview build. 1. Validate your request payload ◦ Paste the exact JSON or code snippet you send to open the realtime session (including the

modalities

field). A misplaced field or typo can cause a server-side validation error. 1. SDK / client version ◦ Confirm you are using the latest realtime SDK or client library. Older SDKs could trigger server errors with newer model builds. 1. Minimal test: open a text-only session without TTS ◦ Start the realtime session with

modalities: ["text"]

and do nothing else (no ElevenLabs calls or playback). If that alone fails with 1011, the problem is the session creation/ model support rather than your TTS integration. What I can do next once you share one of these: • If you paste the exact session request payload and the startup logs around the error, I’ll point out any problematic fields and suggest a corrected payload. • If you can confirm trying a different model name and tell me the result, I’ll recommend the best workaround (use a model that supports text-only or run the half-cascade pattern with a supported model). • If the minimal test (text-only session alone) still fails, I’ll give a concise example payload that is known to be correct for text-only sessions so you can test it directly. A short checklist to send back (pick any of these to start): • The exact session creation code/payload you’re using (sanitized). • Server/log lines around the websocket 1011 close. • Result of trying a different Gemini model (if you tried). • Your realtime SDK/client version. Send one of those and I’ll provide the exact next step or a corrected payload you can copy-paste to test. , with interactive elements