@millions-hair-24956 How to send the history of transcriptions to an agent?

To send the history of transcriptions to an agent, you can utilize the

user_speech_committed

and

agent_speech_committed

events. These events capture the final transcriptions of both user and agent speech. You can store these transcriptions in a session variable or database and then send them to the agent as needed. This approach ensures that the agent has access to the full conversation history, which can be useful for maintaining context in ongoing interactions. Would you like more detailed guidance on implementing this process? Sources: - LiveKit Slack Discussion 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@millions-hair-24956 give an example of both on node ts

It seems likely that you can use the

user_speech_committed

and

agent_speech_committed

events in a Node.js environment to handle transcriptions. Here's a basic example in TypeScript that demonstrates how to listen for these events and store the transcriptions:

import { VoicePipelineAgent } from 'livekit-agents';
import { JobContext } from 'livekit-agents';
import { ChatMessage } from 'livekit-agents';

async function entrypoint(ctx: JobContext) {
  const agent = new VoicePipelineAgent();

  // Connect to the room
  await ctx.connect();

  // Listen for user speech committed event
  agent.on('user_speech_committed', (msg: ChatMessage) => {
    console.log(`User said: ${msg.content}`);
    // Store or process the user transcription
  });

  // Listen for agent speech committed event
  agent.on('agent_speech_committed', (msg: ChatMessage) => {
    console.log(`Agent said: ${msg.content}`);
    // Store or process the agent transcription
  });

  // Start the agent
  agent.start(ctx.room);
}

entrypoint(new JobContext());

This example sets up a

VoicePipelineAgent

and listens for the

user_speech_committed

and

agent_speech_committed

events. When these events are triggered, the transcriptions are logged and can be stored or processed as needed. Would you like more detailed guidance on this example? Sources: - https://docs.livekit.io/agents/openai/multimodal-agent/ - https://docs.livekit.io/agents/openai/multimodalagent/ Note: I wasn’t able to find highly relevant data sources, but above are a few potentially relevant links. 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.