https://livekit.io logo
Join Slack
Powered by
This message was deleted.
# ask-ai
s
This message was deleted.
g
To use different models in a 
VoicePipelineAgent
while maintaining context, you can leverage the flexibility of the agent's architecture. The 
VoicePipelineAgent
allows you to modify the chat context dynamically and switch models as needed. You can achieve this by using the 
before_llm_cb
callback to adjust the context or model parameters before each LLM call. This approach ensures that the context is preserved while allowing you to change models based on the conversation's progression. For example, you can define a function that updates the LLM model or its parameters based on certain conditions or stages in the conversation. This function can be set as the 
before_llm_cb
callback, which is executed before the LLM processes the chat context. This way, you can switch models or adjust their settings dynamically without losing the conversation context. Would you like more detailed guidance or a code example on implementing this approach? Sources: - VoicePipelineAgent Documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
Here's a code sample demonstrating how to use the 
before_llm_cb
callback in a 
VoicePipelineAgent
to switch models dynamically while maintaining context:
Copy code
from livekit.agents import llm, VoicePipelineAgent
from livekit.plugins import cartesia, deepgram, openai, silero

# Initial chat context
initial_ctx = llm.ChatContext().append(role="system", text="Initial prompt")

# Define a callback to switch models based on conversation context
async def switch_model(agent: VoicePipelineAgent, chat_ctx: llm.ChatContext):
    # Example condition to switch models
    if "switch to advanced model" in chat_ctx.messages[-1].text:
        agent.llm = openai.LLM(model="gpt-4o-advanced")
    elif "switch to basic model" in chat_ctx.messages[-1].text:
        agent.llm = openai.LLM(model="gpt-3.5-turbo")

# Create the VoicePipelineAgent
agent = VoicePipelineAgent(
    vad=silero.VAD.load(),
    stt=deepgram.STT(model="nova-2-general"),
    llm=openai.LLM(model="gpt-3.5-turbo"),
    tts=cartesia.TTS(),
    chat_ctx=initial_ctx,
    before_llm_cb=switch_model,  # Set the callback
    allow_interruptions=True,
    interrupt_speech_duration=0.5,
    interrupt_min_words=0,
    min_endpointing_delay=0.5,
)

# Start the agent for a particular room and participant
agent.start(room, participant)
This code sets up a 
VoicePipelineAgent
with an initial LLM model and uses the 
before_llm_cb
callback to switch models based on specific conditions in the conversation. The callback checks the latest message in the chat context and updates the LLM model accordingly. Sources: - VoicePipelineAgent Documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
Open in Slack
PreviousNext
Powered by