i have telphony setup using twilio. can we get transcripts of conversation between agent and person in a session . I am using Node JS

Thanks for reaching out, Anirban. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i am already using the livekit agent SDK nodejs

Great — since you’re already using the LiveKit Agent SDK for Node.js, here’s a focused guide to capture and persist transcripts from a Twilio call routed through LiveKit. What to capture • User speech (STT): emitted as

user_input_transcribed

events (final transcripts flagged with

is_final

). • Agent speech (TTS): available via the agent’s

transcriptionNode

stream. Examples 1) Capture finalized user transcripts (log, save, or forward)

// session is your LiveKit Agent session
session.on('user_input_transcribed', (event) => {
  if (event.is_final) {
    const text = event.transcript;
    const metadata = {
      participantId: event.participant?.participant_id,
      timestamp: Date.now(),
      source: 'user',
    };
    // Example: log or save
    console.log('User transcript:', text, metadata);
    // saveTranscript(text, metadata);
  }
});

2) Capture agent (TTS) transcripts via transcriptionNode

async transcriptionNode(
  text: ReadableStream<string>,
  modelSettings: voice.ModelSettings,
): Promise<ReadableStream<string> | null> {
  return new ReadableStream({
    start(controller) {
      const reader = text.getReader();
      const pump = async () => {
        const { done, value } = await reader.read();
        if (done) {
          controller.close();
          return;
        }
        const transcript = value;
        const metadata = { participantId: 'agent', timestamp: Date.now(), source: 'agent' };
        console.log('Agent transcript:', transcript, metadata);
        // saveTranscript(transcript, metadata);
        controller.enqueue(value);
        pump();
      };
      pump();
    }
  });
}

Simple storage options • File: append lines to a file with timestamps. • Database: insert rows into Postgres/Mongo with participant, timestamp, session_id. • Webhook/stream: POST each finalized transcript to an external API in real time. Implementation tips • Only persist when

is_final

is true for user transcripts to avoid duplicates/partials. • Add session/room IDs and participant IDs to your saved records for later lookup. • If you need ordered conversation logs, store an incrementing turn index or use timestamps and sort on read. • For compliance/retention, consider redaction or retention policies before saving. Would you like: • a complete example that writes transcripts to a file or to Postgres? • code that forwards transcripts to a webhook in real time? Tell me which storage option you prefer and I’ll provide the exact snippet.

i want to save transcripts to DB only after the session has ended. how to achieve that

Thanks — since you’re using the LiveKit Agent SDK for Node.js, the simplest reliable pattern is: • Buffer transcripts during the session (in memory). • Register a shutdown callback that runs after the session ends and persist the buffered transcripts to your DB there. Minimal example (save on session end)

import { voice } from '@livekit/agents';

export default defineAgent({
  entry: async (ctx) => {
    const transcriptBuffer: { role: 'user'|'agent'; content: string; timestamp: string }[] = [];

    // capture finalized user STT
    ctx.session.on('user_input_transcribed', (ev) => {
      if (ev.is_final) {
        transcriptBuffer.push({
          role: 'user',
          content: ev.transcript,
          timestamp: new Date().toISOString(),
        });
      }
    });

    // collect agent replies later from agent.chatContext or capture on conversation events
    const agent = new voice.Agent({ instructions: 'You are a helpful assistant.' });

    await ctx.session.start({ agent, room: ctx.room });

    // shutdown hook runs AFTER session ends — persist here
    ctx.addShutdownCallback(async () => {
      // pull assistant messages from chatContext
      const agentReplies = agent.chatContext.messages
        .filter(m => m.role === 'assistant' && !m.type)
        .map(m => ({
          role: 'agent',
          content: Array.isArray(m.content) ? m.content.join(' ') : String(m.content),
          timestamp: new Date().toISOString(),
        }));

      const allTranscripts = [...transcriptBuffer, ...agentReplies];

      if (allTranscripts.length === 0) return;

      try {
        // replace with your DB logic (transactional insert recommended)
        await saveTranscriptsToDatabase(allTranscripts);
      } catch (err) {
        console.error('Failed to save transcripts on shutdown:', err);
        // optionally emit to logging/alerting for later retry
      }
    });

    await ctx.connect();
  },
});

Key tips • Only persist

user_input_transcribed

items when

is_final

is true to avoid partials. • Use

agent.chatContext

at shutdown to get the agent’s final replies (avoids reconstructing streamed pieces). • Ensure your process has enough memory for buffering; for long calls or high volume, consider a temporary on-disk buffer or short-lived persisted queue. • Wrap DB writes in try/catch and log failures for retries — shutdown callbacks have limited time to complete. If you want, I can: • Provide a full example that writes to Postgres (Prisma/pg) or MongoDB. • Show how to include session/room/participant IDs and an ordered turn index in the saved records. Which DB and format would you like the example for? Sources: Job lifecycle | LiveKit docs

Argument of type '"user_input_transcribed"' is not assignable to parameter of type 'keyof AgentSessionCallbacks'.ts(2345)

Thanks — that TypeScript error happens because the SDK uses typed event names. Use the AgentSession event constants (PascalCase) instead of the snake_case string. Quick fix — replace the string event name with

voice.AgentSessionEventTypes.UserInputTranscribed

and use

voice.AgentSessionEventTypes.Close

to persist on session end:

import { voice } from '@livekit/agents';

const buffer: { role: 'user'|'agent'; content: string; timestamp: string }[] = [];

// user STT (only finalize on isFinal)
session.on(
  voice.AgentSessionEventTypes.UserInputTranscribed,
  (ev) => {
    if (ev.isFinal) {
      buffer.push({ role: 'user', content: ev.transcript, timestamp: new Date().toISOString() });
    }
  }
);

// session end — persist buffered transcripts
session.on(
  voice.AgentSessionEventTypes.Close,
  async () => {
    if (buffer.length === 0) return;
    try {
      await saveTranscriptsToDatabase(buffer); // replace with your DB logic
    } catch (err) {
      console.error('Failed saving transcripts on Close:', err);
    }
  }
);

Notes: • The event constant names are typed; using them prevents the TS error. • Gather agent replies from

agent.chatContext

at shutdown if you didn’t buffer agent messages during the session. • Ensure

saveTranscriptsToDatabase

is async-safe and handles retries/logging inside the Close handler. Would you like a full example that also includes pulling assistant messages from

agent.chatContext

and inserting into Postgres (Prisma) or MongoDB? Sources: Agents (Node.js) | livekit/livekit_composite | DeepWiki | Agent session | LiveKit docs

what about the agent replies ?

Good question — you can capture agent replies two ways and then merge them with buffered user transcripts at session end. Options to capture agent replies • Buffer agent text as it’s produced: listen for conversation items or events that include assistant messages and push them into your in-memory buffer during the session. • Read agent final replies at shutdown: after the session ends, read

agent.chatContext.messages

and extract assistant messages — this is simplest and avoids streaming reconstruction. Recommended pattern (concise example)

import { voice } from '@livekit/agents';

const buffer: { role: 'user'|'agent'; content: string; timestamp: string }[] = [];

// User transcripts (final only)
session.on(voice.AgentSessionEventTypes.UserInputTranscribed, (ev) => {
  if (ev.isFinal) {
    buffer.push({ role: 'user', content: ev.transcript, timestamp: new Date().toISOString() });
  }
});

// Create agent
const agent = new voice.Agent({ instructions: 'You are a helpful assistant.' });
await session.start({ agent, room: ctx.room });

// On session end, pull agent replies from chatContext and save all
session.on(voice.AgentSessionEventTypes.Close, async () => {
  // extract assistant messages
  const agentReplies = agent.chatContext.messages
    .filter(m => m.role === 'assistant' && !m.type) // text-only messages
    .map(m => ({
      role: 'agent',
      content: Array.isArray(m.content) ? m.content.join(' ') : String(m.content),
      timestamp: new Date().toISOString(),
    }));

  const allTranscripts = [...buffer, ...agentReplies];

  if (allTranscripts.length === 0) return;

  try {
    await saveTranscriptsToDatabase(allTranscripts); // your DB logic
  } catch (err) {
    console.error('Failed saving transcripts on Close:', err);
  }
});

Notes and best practices • Using

agent.chatContext

at shutdown is reliable: it contains finalized assistant messages, so you won’t need to reconstruct streamed fragments. • If you require immediate ordering, add a turn index when buffering user items and when mapping assistant messages. • For large sessions or high volume, consider writing buffered items to a temporary store (disk or cache) rather than keeping everything in memory. • Always guard DB writes in try/catch and log failures for retry. Would you like a ready-to-run example that writes these combined records into Postgres (Prisma) or MongoDB?

Property 'chatContext' does not exist on type 'Agent<unknown>'.ts(2339) any

Thanks — the error happens because

Agent

doesn't expose

chatContext

. Use the session's conversation history instead. Session exposes

session.history

(or

session.chatContext

on the session object) which contains all conversation items (user and assistant messages). Buffer user STT via the typed events, then on session Close read

session.history

to get assistant replies and persist everything. Concise example (TypeScript):

import { voice } from '@livekit/agents';

type TranscriptItem = { role: 'user'|'agent'; content: string; timestamp: string };

const buffer: TranscriptItem[] = [];

// buffer finalized user transcripts
session.on(
  voice.AgentSessionEventTypes.UserInputTranscribed,
  (ev) => {
    if (ev.isFinal) {
      buffer.push({ role: 'user', content: ev.transcript, timestamp: new Date().toISOString() });
    }
  }
);

// start agent as usual
const agent = new voice.Agent({ instructions: 'You are a helpful assistant.' });
await session.start({ agent, room: ctx.room });

// on session end, read session.history and persist
session.on(
  voice.AgentSessionEventTypes.Close,
  async () => {
    // session.history contains conversation items (messages, tool calls, etc.)
    const assistantItems = (session.history ?? [])
      .filter(item => item.type === 'message' && item.role === 'assistant')
      .map(item => ({
        role: 'agent',
        content: Array.isArray(item.content) ? item.content.join(' ') : String(item.content),
        timestamp: (item.created_at ?? new Date().toISOString()) as string,
      }));

    const all = [...buffer, ...assistantItems];

    if (all.length === 0) return;

    try {
      await saveTranscriptsToDatabase(all); // implement DB insert here
    } catch (err) {
      console.error('Failed to save transcripts on Close:', err);
    }
  }
);

Notes: • Use

voice.AgentSessionEventTypes.UserInputTranscribed

and

Close

to avoid TS type errors. •

session.history

contains finalized assistant messages; no need for

agent.chatContext

. • If

session.history

is undefined in your SDK version, upgrade to v1.2.0+ or read conversation items via

conversation_item_added

events during the session and buffer them. • Add a turn index or timestamps if you need strict ordering. Want a full example that writes these records to Postgres (Prisma) or MongoDB? Which DB do you use? Sources: Agent session | LiveKit docs | Agents and handoffs | LiveKit docs | Agents (Node.js) | livekit/livekit_composite | DeepWiki

Property 'filter' does not exist on type 'ChatContext'.ts(2339) any

Thanks — that error means you’re treating

ChatContext

like an array.

session.chatContext

(or

session.history

) is an object that contains a

messages

array. Use

.messages

and then filter/map that array. Corrected TypeScript example (collect users during session, read assistant messages at shutdown):

import { voice } from '@livekit/agents';

type TranscriptItem = { role: 'user'|'agent'; content: string; timestamp: string };

const buffer: TranscriptItem[] = [];

// buffer finalized user transcripts
session.on(
  voice.AgentSessionEventTypes.UserInputTranscribed,
  (ev) => {
    if (ev.isFinal) {
      buffer.push({ role: 'user', content: ev.transcript, timestamp: new Date().toISOString() });
    }
  }
);

// start agent
const agent = new voice.Agent({ instructions: 'You are a helpful assistant.' });
await session.start({ agent, room: ctx.room });

// on session end, read session.chatContext.messages (or session.history) and persist
session.on(
  voice.AgentSessionEventTypes.Close,
  async () => {
    // prefer session.chatContext.messages if available
    const messages = (session.chatContext?.messages ?? session.history ?? []) as any[];

    const assistantItems = messages
      .filter(item => item.role === 'assistant' && item.type === 'message')
      .map(item => ({
        role: 'agent',
        content: Array.isArray(item.content) ? item.content.join(' ') : String(item.content),
        timestamp: (item.created_at ?? new Date().toISOString()) as string,
      }));

    const all = [...buffer, ...assistantItems];

    if (all.length === 0) return;

    try {
      await saveTranscriptsToDatabase(all); // your DB logic
    } catch (err) {
      console.error('Failed to save transcripts on Close:', err);
    }
  }
);

Notes: • Use

session.chatContext?.messages

—

chatContext

is an object;

messages

is the array. • If your SDK version lacks

chatContext

,

session.history

is an alternative that contains conversation items. • Ensure you check item

type === 'message'

(some items are tool calls/metadata). • If you want, tell me which SDK version and DB you’re using and I’ll give a runnable example for Prisma/Postgres or MongoDB.