careful-monitor-57946
05/19/2026, 11:18 PM@livekit/agents@1.4.2, @livekit/agents-plugin-openai@1.4.2
- Single voice.AgentSession({ llm: new openai.realtime.RealtimeModel({ model: 'gpt-realtime', voice: 'marin', ... }) }) — no separate STT/TTS (Realtime model is the whole pipeline)
- SIP inbound calls via LiveKit
- Each company has a configured first-line greeting (e.g. "Hi, thank you for calling Chucky Cheese, this is Tony — you are on a recorded line. How can I help?") that we need spoken verbatim when the call connects, before any user input.
What I've tried:
1. session.say(greeting, { allowInterruptions: true }) → throws "trying to generate speech from text without a TTS model" because we don't configure a separate TTS pipeline (the realtime model owns audio).
2. session.generateReply({ instructions: greeting }) → the model hallucinates a different greeting (e.g. says "Welcome to fun zone" for the Chucky Cheese prompt). instructions is a system-prompt-style directive for the turn, not text to speak.
3. session.generateReply({ instructions: "With an upbeat tone, immediately say "${greeting}". Do not say it multiple times." }) → mostly works, but the model occasionally leaks fragments of the wrapper meta-text (e.g. callers hear "with an upbeat attitude…" before the actual greeting). Security/UX issue for us since we're customer-facing.
Is there a canonical pattern for "Realtime agent speaks this exact text first, then proceeds normally"? I've thought about:
- Pre-populating the Agent.chatCtx with a synthetic ChatMessage({ role: 'assistant', content: greeting }) and then calling session.generateReply() with no args — but I'm not sure the realtime model will actually speak a context-only item.
- Mounting a separate tts: on the session purely so session.say() works — feels wasteful since the realtime model already does audio.
- Adding voice.Agent.onEnter() override — but unclear what the right hook is for "speak this verbatim."
Is there a method on voice.AgentSession or realtime.RealtimeModel I'm missing for "play this exact text through the realtime model's audio" without going through the LLM-instruction path? Pointers to a sample repo would be great.tall-airline-16756
05/19/2026, 11:18 PM