Hi! I’m running a voice agent on LiveKit, using Deepgram STT plus the English turn detector to manage user speech. On the assistant side I rely on OpenAI’s realtime model. The issue is that the conversation history the realtime session produces never includes any of the user turns—only the assistant’s responses. From what I can tell, Deepgram is handling transcription for turn detection, but that text never reaches the realtime conversation log, so the history we pull at the end of a call is missing the user side. I’d love guidance on the recommended way to keep turn detection with an external STT while still getting complete user transcripts in the realtime session history.

Thanks for reaching out, Sebastian. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

session = AgentSession( stt=deepgram.STT(model="nova-3", language="en-US"), llm=openai.realtime.RealtimeModel( model="gpt-realtime", turn_detection=None, input_audio_transcription=None, voice=config.voice, temperature=config.temperature ), vad=ctx.proc.userdata["vad"], turn_detection=EnglishModel(),

I'm glad you're looking to include user transcripts in the session history while using Deepgram STT and OpenAI's real-time model! Let's walk through the steps together to make sure everything works smoothly: 1. Enable Interim Transcription: Start by configuring Deepgram STT with

interim_results=True

. This lets you stream both partial and final transcripts in real time, which is super helpful. 2. Integrate Turn Detection: Use a turn detection model like

EnglishModel

or

MultilingualModel

within your

AgentSession

. Just double-check that the language settings match between Deepgram and the turn detector to keep things aligned. 3. Disable Turn Detection in OpenAI: To avoid any conflicts with LiveKit's turn detection, set

turn_detection=None

and

input_audio_transcription=None

in the OpenAI real-time model. 4. Log Transcripts: Make use of the

on("user_input_transcribed")

event to capture and log user transcripts. Once finalized, these will automatically be added to your session history. 5. Optional Tuning: If you want to fine-tune things a bit, adjusting parameters like

min_endpointing_delay

and

max_endpointing_delay

can help improve turn detection accuracy. Here's a sample configuration to guide you:

session = AgentSession(
    stt=deepgram.STT(model="nova-3", language="en-US", interim_results=True),
    llm=openai.realtime.RealtimeModel(
        model="gpt-realtime",
        turn_detection=None,
        input_audio_transcription=None,
        voice=config.voice,
        temperature=config.temperature
    ),
    vad=ctx.proc.userdata["vad"],
    turn_detection=EnglishModel(),
    min_endpointing_delay=0.8,
    max_endpointing_delay=5.0
)

This setup should help you capture user transcripts in real time and keep them nicely included in your session history. If you have any questions or need a hand with anything else, please don’t hesitate to reach out—I’m here to help! Sources: Deepgram STT integration guide | LiveKit Docs | examples/voice_agents/basic_agent.py | LiveKit turn detector plugin | LiveKit Docs | Realtime model integrations | LiveKit Docs | Gemini Live API integration guide | LiveKit Docs

none of this helps me.

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!