agent-js v1.05 - how to obtain a summary of the conversation at end of session (so I can send it somewhere)

Thanks for reaching out, Steve. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I need a summary of the conversation - NOT the conversation verbatim (as you have provided). Probably needs an LLM call to summarise. If it helps, using the RealTime OpenAI model.

I'm happy to guide you through generating a summary of your conversation using an LLM like OpenAI with the RealTime model in agent-js v1.05. Just follow these simple steps: 1. Initialize the OpenAI LLM: Start by creating an instance of the OpenAI LLM. Be sure to specify the model (for example,

gpt-4o-mini

) and include your API key:

import * as openai from '@livekit/agents-plugin-openai';

const llmInstance = new openai.LLM({
  model: 'gpt-4o-mini',
  apiKey: process.env.OPENAI_API_KEY,
});

1. Access and Filter Conversation History: Next, use

session.chat_ctx.copy()

to grab and filter your conversation history, making sure to exclude any non-essential messages for a cleaner summary:

const ctxCopy = session.chat_ctx.copy({
  excludeEmptyMessage: true,
  excludeInstructions: true,
  excludeFunctionCall: true,
});

1. Format Messages for the LLM: Now, prepare the conversation history as a string that the LLM can easily understand for summarization:

let prevConvo = "";
for (const msg of ctxCopy.items) {
  if (msg.role === "user") {
    prevConvo += `Customer: ${msg.text_content}\n`;
  } else {
    prevConvo += `Assistant: ${msg.text_content}\n`;
  }
}

1. Generate the Summary: Finally, use the LLM to create a summary of the conversation:

const chatCtx = new llm.ChatContext();
chatCtx.addMessage({
  role: 'user',
  content: `Summarize the following conversation:\n${prevConvo}`,
});

const stream = llmInstance.chat({ chatCtx });
for await (const chunk of stream) {
  console.log(chunk?.text || '');
}

This method should help you get a clear and concise summary using the OpenAI RealTime model. If anything feels unclear or if you’d like me to walk you through any part again, just let me know—I’m here to help! Sources: Agent-assisted warm transfer | LiveKit Docs | Large language model (LLM) integrations | LiveKit Docs | examples/voice_agents/realtime_load_chat_history.py | OpenAI Realtime API integration guide | LiveKit Docs | OpenAI LLM integration guide | LiveKit Docs

what is 'llmInstance' ?

llmInstance

is basically your connection to the OpenAI Language Model that you set up in your code to chat with the OpenAI API. It’s created using the OpenAI plugin for LiveKit Agents. Here’s a simple way to define it:

import * as openai from '@livekit/agents-plugin-openai';

const llmInstance = new openai.LLM({
  model: 'gpt-4o-mini', // Choose the model you want to work with
  apiKey: process.env.OPENAI_API_KEY, // Your OpenAI API key goes here
});

This instance lets you send requests to the OpenAI model, like summarizing a chat or generating helpful responses. If anything’s unclear or you want me to walk you through it more, just let me know—I’m here to help!

getting errors in

for await (const chunk of stream) {
  console.log(chunk.text || '');
}

it says:

.text

does not exist on

chunk

.

Thanks for reaching out! The error you're seeing happens because the

chunk

object in the OpenAI LLM chat stream doesn’t have a direct

.text

property. Instead, you’ll want to access the content using

chunk.delta.content

. Here’s a little tweak to your code that should do the trick:

for await (const chunk of stream) {
  if (chunk.delta && chunk.delta.content) {
    console.log(chunk.delta.content);
  }
}

This way, you can safely grab the content from each chunk without running into errors. If anything’s unclear or you’d like me to walk you through it, just let me know—I’m here to help! Sources: plugins/openai/src/realtime/realtime_model.ts | livekit-plugins/livekit-plugins-anthropic/livekit/plugins/anthropic/llm.py | pipeline-llm/llm_powered_content_filter.py