https://livekit.io logo
Join Slack
Powered by
is it possible to make outgoing calls for the agen...
# ask-ai
l
is it possible to make outgoing calls for the agent in nodeJS
t
Thanks for reaching out, Anirban. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — you're asking whether an agent can make outgoing (SIP) calls using Node.js. Short answer: No — outbound calling for agents is only available in Python. Node.js agents can join rooms and handle media, but the SIP outbound API (creating a SIP participant / dialing out) is only supported in the Python SDK. If you’d like, I can: • Show the Python example for creating an outbound SIP participant, or • Help explore alternative Node.js approaches for your use case (e.g., two-step flow where a Python service initiates the SIP call and your Node.js agent handles the LiveKit room). Which would you prefer? Sources: Agents telephony integration | LiveKit docs
l
Secondly is it possible to provide pdf's or knowledge banks to the voice agent in NodeJS
t
Responding to your question about providing PDFs or knowledge banks to a voice agent in Node.js. Yes — you can make PDFs or other knowledge sources available to a Node.js voice agent, but it requires custom integration. Key options: • Preload documents at startup: parse PDFs (e.g., with 
pdf-parse
or 
pdf-lib
) into text, index them into a vector DB (Pinecone, Weaviate, etc.), and include retrieved content in the agent’s prompt context when generating replies. • On-demand ingestion via frontend byte streams: accept file uploads, send bytes to your backend, parse and store them, then run retrieval at each user turn. • Use lifecycle hooks (for example 
on_user_turn_completed
) to query your knowledge store and inject relevant passages into the LLM context before the agent responds. • Combine with RAG (retrieval-augmented generation) so the agent returns grounded answers and can cite or summarize PDF content. I can: • Share a concise Node.js example for parsing a PDF and storing embeddings, or • Outline how to implement RAG using a specific vector DB and LLM provider. Which would you like me to provide next? Sources: Pipeline nodes and hooks | LiveKit docs | Vision | LiveKit docs
l
export default defineAgent({ prewarm: async (proc: JobProcess) => { console.log('Prewarming agent...'); // Load Silero VAD model for voice activity detection proc.userData.vad = await silero.VAD.load(); console.log('VAD model loaded successfully'); }, entry: async (ctx: JobContext) => { const agentId = process.env.AGENT_ID || 'unknown'; const agentName = process.env.AGENT_NAME || '{{AGENT_NAME}}'; await ctx.connect(); console.log('Connected to LiveKit room'); // Wait for a participant to join const participant = await ctx.waitForParticipant(); console.log(
Participant joined: ${participant.identity}
); let currentCallId: string | null = null; const callerNumber = participant.identity || 'unknown'; try { const call = await prisma.call.create({ data: { agent_id: agentId, callerNumber, status: 'INITIATED', startTime: new Date(), endTime: new Date(), duration: 0, }, }); currentCallId = call.id; console.log(
Call record created: ${currentCallId}
); } catch (error) { console.error('Failed to create call record:', error); } const transcriptBuffer: Array<{ role: 'user' | 'agent'; content: string; timestamp: string; }> = []; const usageCollector = new metrics.UsageCollector(); const agent = new voice.Agent({ instructions: '{{INSTRUCTIONS}}', tools: { getCurrentTime, }, }); // Preloaded VAD const vad = ctx.proc.userData.vad as silero.VAD; const infomaniakSTT = new InfomaniakSTT({ apiToken: process.env.INFOMANIAK_API_TOKEN!, productId: process.env.INFOMANIAK_PRODUCT_ID!, pollingInterval: 500, maxPollingAttempts: 60, }); // This is required for non streaming STT like Infomaniak or OpenAI Whisper const streamAdapter = new stt.StreamAdapter(infomaniakSTT, vad); const session = new voice.AgentSession({ stt: streamAdapter, llm: new inference.LLM({ model: '{{LLM_MODEL}}', }), tts: new inference.TTS({ model: '{{TTS_MODEL}}', voice: '{{TTS_VOICE}}', }), vad: vad, }); session.on(voice.AgentSessionEventTypes.MetricsCollected, async(ev) => { metrics.logMetrics(ev.metrics); usageCollector.collect(ev.metrics); const metricsData = { type: ev.metrics.type, timestamp: new Date().toISOString(), data: ev.metrics, }; await saveMetricsToDatabase(agentId, metricsData); }); session.on( voice.AgentSessionEventTypes.UserInputTranscribed, (ev) => { if (ev.isFinal) { transcriptBuffer.push({ role: 'user', content: ev.transcript, timestamp: new Date().toISOString() }); } } ); const logUsage = async () => { const summary = usageCollector.getSummary(); console.log(
Usage: ${JSON.stringify(summary)}
); }; ctx.addShutdownCallback(logUsage); await session.start({ agent, room: ctx.room, }); console.log('AgentSession started'); if (currentCallId) { try { await prisma.call.update({ where: { id: currentCallId }, data: { status: 'IN_PROGRESS' }, }); console.log('Call status updated to IN_PROGRESS'); } catch (error) { console.error('Failed to update call status:', error); } } await session.generateReply({ instructions: '{{GREETING_INSTRUCTIONS}}', }); session.on( voice.AgentSessionEventTypes.Close, async () => { if (!currentCallId) { console.warn('No call ID available, skipping save'); return; } try { const messages = (session.chatCtx?.items ?? session.history ?? []) as any[]; const agentReplies = messages .filter(item => item.role === 'assistant' && item.type === 'message') .map(item => ({ role: 'agent', content: Array.isArray(item.content) ? item.content.join(' ') : String(item.content), timestamp: (item.created_at ?? new Date().toISOString()) as string, })); const allTranscripts = [...transcriptBuffer, ...agentReplies].sort( (a, b) => new Date(a.timestamp).getTime() - new Date(b.timestamp).getTime() ); if (allTranscripts.length === 0) { console.log('No transcripts to save'); return; } const endTime = new Date(); const call = await prisma.call.findUnique({ where: { id: currentCallId }, }); const duration = call ? (endTime.getTime() - call.startTime.getTime()) / 1000 : 0; await prisma.call.update({ where: { id: currentCallId }, data: { status: 'COMPLETED', endTime, duration, }, }); await prisma.callSummary.create({ data: { call_id: currentCallId, extractedData: { transcripts: allTranscripts, totalTurns: allTranscripts.length, userTurns: transcriptBuffer.length, agentTurns: agentReplies.length, sessionMetadata: { agentId, agentName, roomName: ctx.room.name, participantIdentity: participant.identity, duration, }, } as any, }, }); console.log('Session data saved successfully'); } catch (error) { console.error('Failed to save session data on Close:', error); } } ); }, }); based on my code show me sample code in node js to integrate pdf information
t
Thanks — I see you want a Node.js example that parses PDF content and injects it into your existing LiveKit voice agent flow so the agent can use the PDF as a knowledge source. Below is a concise, practical example showing: • Parsing PDFs on startup (prewarm) and indexing embeddings into a vector store (using a generic vector DB API). • Retrieving relevant passages with a simple cosine-retrieval step during a user turn. • Injecting retrieved passages into the LLM prompt before generating a reply. Adjust integrations (vector DB client, embedding/LLM provider) to your stack. 1) Install suggested packages
Copy code
npm install pdf-parse @pinecone-database/pinecone node-fetch cosine-similarity
# or replace Pinecone with your vector DB client and cosine-similarity with your chosen utils
2) Example code (integrate into your agent file)
Copy code
import fs from 'fs';
import pdf from 'pdf-parse';
import fetch from 'node-fetch';
import { PineconeClient } from '@pinecone-database/pinecone';
import cosine from 'cosine-similarity';

// --- Helpers: embedding (replace with your provider) ---
async function embedTexts(texts: string[]): Promise<number[][]> {
  // Example using an external embedding API; replace with your provider call
  const resp = await fetch(process.env.EMBEDDING_API_URL!, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${process.env.EMBEDDING_API_KEY}` },
    body: JSON.stringify({ inputs: texts }),
  });
  const data = await resp.json();
  return data.embeddings; // expect [[...], [...]]
}

// --- Helpers: simple local similarity search against Pinecone or in-memory ---
async function upsertToPinecone(indexName: string, vectors: { id: string; values: number[]; metadata: any }[]) {
  const pc = new PineconeClient();
  await pc.init({ apiKey: process.env.PINECONE_API_KEY!, environment: process.env.PINECONE_ENV });
  const index = pc.Index(indexName);
  await index.upsert({ upsertRequest: { vectors } });
}

async function queryPinecone(indexName: string, queryVector: number[], topK = 4) {
  const pc = new PineconeClient();
  await pc.init({ apiKey: process.env.PINECONE_API_KEY!, environment: process.env.PINECONE_ENV });
  const index = pc.Index(indexName);
  const q = await index.query({ queryRequest: { topK, vector: queryVector, includeMetadata: true } });
  return q.matches ?? [];
}

// --- PDF parsing & chunking ---
async function parsePdfBuffer(buffer: Buffer) {
  const data = await pdf(buffer);
  // simple split by paragraph (tune as needed)
  const paragraphs = data.text.split(/\n{2,}/).map(p => p.trim()).filter(Boolean);
  return paragraphs;
}

function chunkText(text: string, maxLen = 800) {
  const words = text.split(/\s+/);
  const chunks: string[] = [];
  let cur = [];
  let curLen = 0;
  for (const w of words) {
    cur.push(w);
    curLen += w.length + 1;
    if (curLen > maxLen) {
      chunks.push(cur.join(' '));
      cur = [];
      curLen = 0;
    }
  }
  if (cur.length) chunks.push(cur.join(' '));
  return chunks;
}

// --- Prewarm: load PDFs, embed, and index ---
export default defineAgent({
  prewarm: async (proc: JobProcess) => {
    console.log('Prewarming agent and indexing PDFs...');

    const indexName = process.env.PINECONE_INDEX || 'agent-knowledge';

    // Example: load all PDFs from a folder
    const pdfFiles = fs.readdirSync('./pdfs').filter(f => f.endsWith('.pdf'));
    const allTexts: string[] = [];
    const metadataList: any[] = [];

    for (const file of pdfFiles) {
      const buffer = fs.readFileSync(`./pdfs/${file}`);
      const paragraphs = await parsePdfBuffer(buffer);
      for (const para of paragraphs) {
        const chunks = chunkText(para, 800);
        for (const chunk of chunks) {
          allTexts.push(chunk);
          metadataList.push({ source: file });
        }
      }
    }

    if (allTexts.length === 0) {
      console.log('No PDF content found for indexing');
      proc.userData.knowledgeIndexed = false;
      return;
    }

    // get embeddings in batches
    const batchSize = 16;
    const vectors: { id: string; values: number[]; metadata: any }[] = [];
    for (let i = 0; i < allTexts.length; i += batchSize) {
      const batch = allTexts.slice(i, i + batchSize);
      const embeddings = await embedTexts(batch);
      for (let j = 0; j < embeddings.length; j++) {
        vectors.push({
          id: `pdf_${i + j}`,
          values: embeddings[j],
          metadata: { text: batch[j], ...metadataList[i + j] },
        });
      }
    }

    await upsertToPinecone(indexName, vectors);
    proc.userData.knowledgeIndexed = true;
    proc.userData.knowledgeIndexName = indexName;
    console.log(`Indexed ${vectors.length} chunks into ${indexName}`);
  },

  entry: async (ctx: JobContext) => {
    await ctx.connect();
    // ... your existing flow ...
    const vad = ctx.proc.userData.vad as silero.VAD;
    // create session similar to your code
    const session = new voice.AgentSession({ /* ... */ });

    // Helper: retrieve relevant docs for a query
    async function retrieveRelevant(query: string, topK = 4) {
      if (!ctx.proc.userData.knowledgeIndexed) return [];
      const qVec = (await embedTexts([query]))[0];
      const matches = await queryPinecone(ctx.proc.userData.knowledgeIndexName, qVec, topK);
      return (matches || []).map(m => ({ score: m.score, text: m.metadata.text, source: m.metadata.source }));
    }

    // Example: on each user final transcription, run retrieval and inject into LLM prompt
    session.on(voice.AgentSessionEventTypes.UserInputTranscribed, async (ev) => {
      if (!ev.isFinal) return;
      const userText = ev.transcript;
      // retrieve
      const docs = await retrieveRelevant(userText, 3);
      if (docs.length === 0) return;

      // Build a short context block
      const contextBlock = docs.map((d, i) => `Source ${i+1} (${d.source}): ${d.text}`).join('\n\n');

      // Inject context before next reply
      await session.generateReply({
        instructions: `Use the following documents as reference when answering. If the answer isn't in the documents, say so.\n\n${contextBlock}\n\nUser question: ${userText}`,
      });
    });

    // start session
    await session.start({ agent: new voice.Agent({ instructions: '{{INSTRUCTIONS}}' }), room: ctx.room });
    // ... rest of your flow ...
  },
});
Notes and adjustments • Embeddings/LLM: Replace 
embedTexts
and LLM calls with your provider (OpenAI, Anthropic, etc.). Ensure embedding dimensions match your vector DB. • Vector DB: I used Pinecone as an example; swap for Weaviate, Milvus, or an in-memory store for small datasets. • Security: Sanitize PDFs and limit indexing to allowed files. • Latency: Pre-index at prewarm to avoid runtime delays. For uploaded PDFs at runtime, parse, embed, and upsert then use retrieval on subsequent turns. • Prompt design: Keep retrieved passages concise; use attribution lines (source) to help the agent cite correctly. If you want, I can: • Provide a sample for handling runtime user PDF uploads (frontend byte stream → backend parse → upsert → immediate retrieval), or • Swap the Pinecone example for Weaviate or an in-memory cosine search snippet. Which do you prefer?
l
Help explore alternative Node.js approaches for your use case (e.g., two-step flow where a Python service initiates the SIP call and your Node.js agent handles the LiveKit room
t
Thanks — I’ll show a clear two-step architecture and concise example code snippets so you can keep your Node.js agent for room/agent logic while a small Python service handles outbound SIP dialing. Overview (what I’m responding to) • You asked for an alternative Node.js approach where Python makes the SIP outbound call and Node.js runs the LiveKit voice agent and session logic. Two-step architecture (high level) 1. Python dialing service ◦ Uses the Python-only SIP outbound API to create the SIP participant and dial the PSTN/SIP number. ◦ Returns the created call/participant identifier and any SIP metadata (callId, sipParticipantId, bridge info). 1. Node.js agent service ◦ Runs your LiveKit agent (joins room, runs STT/LLM/TTS, session lifecycle). ◦ Receives notification that the outbound call is active and can connect to the same LiveKit room or handle media for that call. 1. Coordination ◦ Communicate via a small API (HTTP webhook/REST), message queue (Redis, RabbitMQ), or pub/sub (Postgres LISTEN/NOTIFY) so Python notifies Node.js when call is created/answered/ended. ◦ Share a room name or unique session ID so both sides attach to the same LiveKit room context and share participant identities. Benefits • Keeps SIP outbound logic where it’s supported (Python). • Keeps your Node.js agent logic and integrations (PDF/RAG, metrics, DB) intact. • Minimal runtime coupling — services talk via light messages. Concrete flow (sequence) 1. UI/Operator requests outbound call (or agent initiates). 2. Node.js sends request to Python Dialer API: POST /dial { to: '+1234', room: 'room-abc', metadata: {...} }. 3. Python dials via SIP API, creates SIP participant and bridges it to LiveKit room 'room-abc'. 4. Python responds with { callId, sipParticipantId, room } to Node.js and optionally calls Node.js webhook: /call-updated. 5. Node.js receives notification, verifies participant joined (or waits for participant via ctx.waitForParticipant()), and starts the AgentSession for that room. 6. During call, Node.js runs STT/RAG/LLM/TT S and persists call data; Python monitors SIP state and informs Node.js on disconnects to close session and finalize DB records. Minimal example snippets 1) Python dialer (pseudo/minimal)
Copy code
# python_dialer.py (very small Flask example)
from flask import Flask, request, jsonify
from livekit_sip_client import create_sip_participant  # pseudocode, use LiveKit Python SIP API

app = Flask(__name__)

@app.route('/dial', methods=['POST'])
def dial():
    body = request.json
    to_number = body['to']
    room = body['room']
    # create SIP participant via Python SDK (example shape)
    sip_participant = create_sip_participant(room=room, to=to_number, some_meta=body.get('meta', {}))
    # respond with IDs and room info
    return jsonify({
        'callId': sip_participant.call_id,
        'sipParticipantId': sip_participant.id,
        'room': room
    })
2) Node.js initiator call (request Python dialer)
Copy code
// callInitiator.js
import fetch from 'node-fetch';

export async function requestDial(toNumber, roomName, metadata = {}) {
  const resp = await fetch(process.env.PY_DIALER_URL + '/dial', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ to: toNumber, room: roomName, meta: metadata }),
  });
  const data = await resp.json();
  return data; // { callId, sipParticipantId, room }
}
3) Node.js agent: wait for call and start session (integrate into your existing entry)
Copy code
// inside your entry: after ctx.connect()
const dialResult = await requestDial('+123456789', ctx.room.name);
console.log('Dial result', dialResult);

// Wait for participant to join (SIP participant should join the room)
const participant = await ctx.waitForParticipant({ identity: dialResult.sipParticipantId });
// then start session as you already do
await session.start({ agent, room: ctx.room });
4) Notification/webhook pattern (optional) • Python can POST to Node.js /webhook/call-updated with { callId, state: 'ANSWERED' } to avoid polling. • Node.js webhook handler updates DB and triggers agent actions (start/stop). Practical tips and pitfalls • Synchronize identities: have Python set participant identity or metadata so Node.js can waitForParticipant by identity. • Room lifecycle: decide whether Python creates the room or Node.js; both must use the same room name and LiveKit API keys scoped appropriately. • Security: authenticate calls between services (API keys, mutual TLS, signed JWT). Validate webhooks. • Race conditions: If Node.js starts waiting before SIP participant arrives, use waitForParticipant with timeout and handle retries. • Media bridging: ensure Python SIP creation is configured to bridge audio into the same LiveKit room so Node.js hears the call. • Call teardown: ensure Python notifies Node.js on call end so Node.js can close AgentSession and persist final transcripts (your Close handler already handles saving). Next steps I can provide • A runnable Python example using the exact LiveKit Python SIP API snippet for create_sip_participant. • A webhook handler example in your Node.js agent to receive call state updates. • Example for secure service-to-service auth (JWT signing example). Which of those would help you next?
l
Create A runnable Python example using the exact LiveKit Python SIP API snippet for create_sip_participant.
t
Thanks — below is a runnable Python example that uses the LiveKit Python SDK to create an outbound SIP participant with 
create_sip_participant
. It shows initialization, the API call, and minimal error handling. Set the required environment variables before running. Save as create_sip_call.py:
Copy code
import asyncio
import os
from livekit import api, models

async def create_sip_call():
    lkapi = api.LiveKitAPI(
        url=os.getenv("LIVEKIT_URL"),
        api_key=os.getenv("LIVEKIT_API_KEY"),
        api_secret=os.getenv("LIVEKIT_API_SECRET")
    )

    try:
        req = models.CreateSIPParticipantRequest(
            room_name="my-room",                            # room to bridge the SIP call into
            sip_trunk_id=os.getenv("SIP_OUTBOUND_TRUNK_ID"),# your configured outbound trunk ID
            sip_call_to="+1234567890",                      # destination phone number (E.164)
            participant_identity="sip-caller-123",          # identity used for the SIP participant in the room
            wait_until_answered=True                        # wait until the call is answered before returning
        )

        sip_participant = await lkapi.sip.create_sip_participant(req)
        print("SIP participant created:")
        print(sip_participant)

    except Exception as e:
        print("Error creating SIP participant:", e)
    finally:
        await lkapi.aclose()


if __name__ == "__main__":
    asyncio.run(create_sip_call())
Environment variables to set: • LIVEKIT_URL — your LiveKit URL (e.g., https://your-livekit.example) • LIVEKIT_API_KEY — API key with appropriate permissions • LIVEKIT_API_SECRET — API secret • SIP_OUTBOUND_TRUNK_ID — the outbound SIP trunk ID configured for your LiveKit project Notes: • 
sip_trunk_id
must reference a valid outbound trunk in your project. • Phone numbers must be E.164 formatted (e.g., +1234567890). • 
wait_until_answered=True
causes the call to wait until answered; set to False if you prefer async dialing. • Ensure your SIP trunk/provider allows outbound calls and any required SIP features (e.g., REFER) are enabled. If you want, I can: • Provide a Flask/FastAPI wrapper around this snippet so your Node.js service can POST to it, or • Show how to return structured JSON and send a webhook to your Node.js agent when the call state changes. Which would you like? Sources: Agents telephony integration | LiveKit docs | sip | livekit/livekit_composite | DeepWiki
l
Provide a Flask/FastAPI wrapper around this snippet so your Node.js service can POST to it
t
Below are two runnable examples (Flask and FastAPI). Each exposes a POST /dial endpoint your Node.js service can call with { "to": "+123...", "room": "room-name", "identity": "caller-1", "wait": true }. They call LiveKit's Python SDK create_sip_participant and return JSON with call info or error. Set environment variables as listed. Flask example (save as dialer_flask.py):
Copy code
import os
import asyncio
from flask import Flask, request, jsonify
from livekit import api, models

app = Flask(__name__)

async def create_sip_part(room, to, identity, sip_trunk_id, wait):
    lkapi = api.LiveKitAPI(
        url=os.getenv("LIVEKIT_URL"),
        api_key=os.getenv("LIVEKIT_API_KEY"),
        api_secret=os.getenv("LIVEKIT_API_SECRET"),
    )
    try:
        req = models.CreateSIPParticipantRequest(
            room_name=room,
            sip_trunk_id=sip_trunk_id,
            sip_call_to=to,
            participant_identity=identity,
            wait_until_answered=wait,
        )
        resp = await lkapi.sip.create_sip_participant(req)
        return resp
    finally:
        await lkapi.aclose()

@app.route("/dial", methods=["POST"])
def dial():
    body = request.get_json() or {}
    to = body.get("to")
    room = body.get("room", "default-room")
    identity = body.get("identity", f"sip-{int(os.times()[4])}")
    wait = bool(body.get("wait", True))
    sip_trunk_id = os.getenv("SIP_OUTBOUND_TRUNK_ID")

    if not to:
        return jsonify({"error": "missing 'to' phone number"}), 400
    if not sip_trunk_id:
        return jsonify({"error": "SIP_OUTBOUND_TRUNK_ID not configured"}), 500

    try:
        resp = asyncio.run(create_sip_part(room, to, identity, sip_trunk_id, wait))
        # Convert response to JSON-serializable form if needed
        return jsonify({"success": True, "participant": resp}), 200
    except Exception as e:
        return jsonify({"success": False, "error": str(e)}), 500

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=int(os.getenv("PORT", 8080)))
FastAPI example (save as dialer_fastapi.py):
Copy code
import os
import asyncio
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from livekit import api, models
import uvicorn

app = FastAPI()

class DialRequest(BaseModel):
    to: str
    room: str = "default-room"
    identity: str | None = None
    wait: bool = True

async def create_sip_part(room, to, identity, sip_trunk_id, wait):
    lkapi = api.LiveKitAPI(
        url=os.getenv("LIVEKIT_URL"),
        api_key=os.getenv("LIVEKIT_API_KEY"),
        api_secret=os.getenv("LIVEKIT_API_SECRET"),
    )
    try:
        req = models.CreateSIPParticipantRequest(
            room_name=room,
            sip_trunk_id=sip_trunk_id,
            sip_call_to=to,
            participant_identity=identity,
            wait_until_answered=wait,
        )
        resp = await lkapi.sip.create_sip_participant(req)
        return resp
    finally:
        await lkapi.aclose()

@app.post("/dial")
async def dial(req: DialRequest):
    sip_trunk_id = os.getenv("SIP_OUTBOUND_TRUNK_ID")
    if not sip_trunk_id:
        raise HTTPException(status_code=500, detail="SIP_OUTBOUND_TRUNK_ID not configured")
    identity = req.identity or f"sip-{int(os.times()[4])}"
    try:
        resp = await create_sip_part(req.room, <http://req.to|req.to>, identity, sip_trunk_id, req.wait)
        return {"success": True, "participant": resp}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

if __name__ == "__main__":
    uvicorn.run("dialer_fastapi:app", host="0.0.0.0", port=int(os.getenv("PORT", 8080)), reload=False)
Environment variables required: • LIVEKIT_URL • LIVEKIT_API_KEY • LIVEKIT_API_SECRET • SIP_OUTBOUND_TRUNK_ID Node.js caller example:
Copy code
// callDialer.js
import fetch from 'node-fetch';
export async function requestDial(to, room, identity, wait = true) {
  const resp = await fetch(process.env.PY_DIALER_URL + '/dial', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ to, room, identity, wait }),
  });
  return resp.json();
}
Notes and recommendations: • Authenticate requests between Node.js and Python (API key, JWT) in production. • Ensure room name/participant identity is agreed so Node.js can waitForParticipant in the LiveKit room. • Use webhooks or POST back from Python to Node.js for call state updates (answered/ended) to avoid polling. • For large-scale use, run the Python dialer as an async service (FastAPI + Uvicorn) for concurrency. Would you like the FastAPI example extended to send a webhook to your Node.js service when the call is answered/ended?
l
what does an outbound trunk specify
t
An outbound trunk specifies how LiveKit connects outbound calls to a SIP/telephony provider. Key pieces it defines: • Connection details: SIP server/domain, port, transport (UDP/TCP/TLS) and any proxy/egress settings used to route calls. • Authentication: SIP credentials (username/password, token) or other auth used to register/authenticate with the provider. • Trunk ID / routing: A unique 
sip_trunk_id
used when creating SIP participants so LiveKit knows which trunk to use for dialing. • Caller ID / From number (optional): The originating phone number presented to recipients if the provider supports overriding the From header. • Region/geographic options (optional): Which region or egress point to use so calls originate from a desired locale or comply with local rules. • Call behavior features (optional): Per-trunk capabilities or defaults such as codecs, DTMF behavior, or audio-processing flags (e.g., noise-cancel settings applied to calls using that trunk). Practical notes: • When placing an outbound call you pass the trunk’s 
sip_trunk_id
to the create_sip_participant call so LiveKit routes the call through that provider. • Ensure the trunk is configured with a provider that allows outbound dialing and the required SIP features (caller ID override, REFER, etc.). • You can list configured outbound trunks with the CLI (
lk sip outbound list
) to find the correct 
sip_trunk_id
. If you want, I can show an example curl/CLI command or the exact Python snippet that references a 
sip_trunk_id
. Which would you prefer? Sources: sip | livekit/livekit_composite | DeepWiki | Outbound Calling Script | LiveKit docs | SIP Lifecycle Management Agent | LiveKit docs | Agents telephony integration | LiveKit docs
l
basically should a single voice agent have a single outbound trunk assigned to it ?
t
Short answer: No — a voice agent does not have to be limited to a single outbound trunk. Choose one or multiple trunks based on operational needs. Guidance to decide: • Use a single trunk when: ◦ You have one SIP provider that meets coverage, pricing, and regulatory needs. ◦ Simpler configuration, easier troubleshooting, and predictable caller ID behavior are priorities. ◦ You want fewer assets to manage for small deployments. • Use multiple trunks when: ◦ You need geographic coverage or local origination (different trunks per region to present local caller IDs). ◦ You require redundancy/failover if one provider is down. ◦ You want cost optimization or per-number routing rules (cheaper provider for specific destinations). ◦ Regulatory or carrier restrictions require different trunks for certain numbers or features. Operational considerations: • Routing logic: implement service-side logic (Python dialer or gateway) to pick trunk by region, cost, or availability. • Identity mapping: ensure participant identity/metadata includes trunk info so your Node.js agent can correlate calls and waitForParticipant correctly. • Failover: handle trunk failures by retrying on alternate trunks and surface state to the agent (webhook or messaging). • Caller ID: verify provider support for overriding From header if you need consistent caller ID across trunks. • Monitoring & billing: track per-trunk metrics and usage for invoicing and troubleshooting. • Complexity vs benefit: multiple trunks add operational overhead — use them only when benefits (coverage, redundancy, cost) justify it. If you want, I can: • Provide a small routing example (Python) that selects a trunk by destination country with failover, or • Sketch how to store trunk metadata in your call DB and surface it to the Node.js agent. Which would help most?
l
declare enum SIPTransport { /** * @generated from enum value: SIP_TRANSPORT_AUTO = 0; */ SIP_TRANSPORT_AUTO = 0, /** * @generated from enum value: SIP_TRANSPORT_UDP = 1; */ SIP_TRANSPORT_UDP = 1, /** * @generated from enum value: SIP_TRANSPORT_TCP = 2; */ SIP_TRANSPORT_TCP = 2, /** * @generated from enum value: SIP_TRANSPORT_TLS = 3; */ SIP_TRANSPORT_TLS = 3, } what protocol should i opt for outbound calls
t
Good question — here's a concise recommendation for which SIP transport to choose. • Start with SIP_TRANSPORT_AUTO. It lets LiveKit negotiate UDP/TCP/TLS and provides the best resilience across varied networks. Use this unless you have strict requirements. • Choose SIP_TRANSPORT_TLS when security/compliance matters (HIPAA, PCI, sensitive data) or when you want encrypted signaling; ensure your SIP provider and certificates support TLS (sips: URI / port 5061). • Choose SIP_TRANSPORT_TCP when UDP is unreliable or blocked but you don’t require encryption; TCP is more firewall-friendly than UDP. • Choose SIP_TRANSPORT_UDP only when you control both network endpoints, need minimal latency, and are sure UDP is allowed through firewalls/NAT. Practical tips: • Transport is typically set at the trunk level (use sips: to force TLS). • Verify your SIP provider supports the selected transport. • Use AUTO for general deployments, TLS for regulated/secure contexts, and TCP/UDP only for specific network requirements. Would you like a short example showing how to force TLS in the trunk URI (sips:) or code to detect and fallback between trunks? Sources: Firewall Tips | LiveKit Security | https://blog.livekit.io/introducing-livekit-phone-numbers-zero-to-ringing-in-60-seconds/#/portal/signup| | sip | livekit/livekit_composite | DeepWiki | Running LiveKit on AWS
l
<http://router.post|router.post>('/calls/initiate', async (req, res) => {
try {
const {
agentId,
trunkId,
fromNumber,
toNumber,
roomName,
displayName,
dtmf,
playDialtone,
waitUntilAnswered,
krispEnabled,
participantMetadata,
} = req.body;
// Validation
 
if (!agentId || !trunkId || !fromNumber || !toNumber) {
return res.status(400).json({
success: false,
message: 'Missing required fields: agentId, trunkId, fromNumber, toNumber',
});
}
// Validate phone number format (E.164)
 
const phoneRegex = /^\+\d{10,15}$/;
if (!phoneRegex.test(toNumber)) {
return res.status(400).json({
success: false,
message: 'Invalid phone number format. Use E.164 format (e.g., <tel:+12135550100|+12135550100>)',
});
}
// Validate DTMF if provided
 
if (dtmf && !/^[0-9*#wW]+$/.test(dtmf)) {
return res.status(400).json({
success: false,
message: 'Invalid DTMF format. Use digits 0-9, *, #, and w for delays',
});
}
const result = await outboundCallService.initiateCall(
agentId,
trunkId,
fromNumber,
toNumber,
{
roomName,
displayName,
dtmf,
playDialtone: playDialtone ?? false,
waitUntilAnswered: waitUntilAnswered ?? true,
krispEnabled: krispEnabled ?? true,
participantMetadata,
}
);
res.json({
success: true,
message: result.message,
data: result,
});
} catch (error: any) {
console.error('Failed to initiate call:', error);
// Check for SIP-specific errors
 
if (error.message.includes('SIP Error')) {
return res.status(400).json({
success: false,
message: 'Call failed',
error: error.message,
});
}
res.status(500).json({
success: false,
message: 'Failed to initiate call',
error: error.message,
});
}
});
what does teh above api call do
t
You're asking what the Express POST /calls/initiate endpoint does. Here's a concise breakdown. What it does • Receives a JSON request to start an outbound call and validates required fields. • Calls 
outboundCallService.initiateCall(...)
to perform the actual dialing/bridging work. • Returns success JSON with the service result, or an error JSON if initiation fails. Input it expects (from req.body) • Required: 
agentId
, 
trunkId
, 
fromNumber
, 
toNumber
• Optional: 
roomName
, 
displayName
, 
dtmf
, 
playDialtone
, 
waitUntilAnswered
, 
krispEnabled
, 
participantMetadata
Validation performed • Ensures required fields are present; returns 400 if missing. • Validates 
toNumber
is E.164 (
+
followed by 10–15 digits); returns 400 if invalid. • Validates 
dtmf
contains only digits, 
*
, 
#
, or 
w
(for delays); returns 400 if invalid. Call flow 1. After validation, it calls 
outboundCallService.initiateCall(agentId, trunkId, fromNumber, toNumber, options)
where options contains the optional parameters (with defaults for 
playDialtone
, 
waitUntilAnswered
, 
krispEnabled
). 2. 
outboundCallService
is responsible for creating the SIP participant / dialing via your telephony/trunk integration (not shown in this snippet). 3. On success, the endpoint responds with 
{ success: true, message, data }
where 
data
is the service result (call IDs, participant info, etc.). Error handling • Catches exceptions and logs them. • If the error message includes "SIP Error", returns 400 with the SIP-specific error. • Otherwise returns 500 with a generic failure message and the error text. Practical notes • 
trunkId
selects which outbound trunk/provider to use (routing, credentials, transport). • 
waitUntilAnswered: true
typically blocks until the call is answered; set false for async dialing. • Ensure 
outboundCallService.initiateCall
implements trunk usage, room/identity mapping, and webhook/notification callbacks so your Node.js agent can detect participant join/leave. If you want, I can: • Show a sample implementation of 
outboundCallService.initiateCall
(Node.js calling the Python dialer or directly calling LiveKit if available), or • Suggest request/response shapes to return from the service for easier integration. Which would you prefer?
l
does the call to the phone number happen after the createSIPparticipant is called
t
Short answer: Yes — calling the phone number is initiated when you invoke 
create_sip_participant
. That API tells LiveKit to create a SIP participant and instruct the configured outbound trunk to place the outbound call to the specified 
sip_call_to
number. Key details: • The outbound dialing is triggered by the 
create_sip_participant
request. • If you set 
wait_until_answered=true
, the API call will wait (block) until the remote party answers (or a timeout/failure). If 
wait_until_answered=false
, the API will return immediately while dialing continues asynchronously. • Trunk-level settings (transport, credentials, caller ID, etc.) determine how the SIP INVITE is sent to the provider; the API simply requests the trunk to place the call. • You should monitor participant/room events (participant joined, answered, disconnected) or use webhooks from your dialing service to detect when the call is answered, in-progress, or ended. • Handle errors from the API (SIP failure, trunk misconfiguration) — these can surface synchronously if 
wait_until_answered
is true, or via events/webhooks if false. If you want, I can show: • A short Python example of 
create_sip_participant
with 
wait_until_answered
both true and false, or • How to wire a webhook / event listener so your Node.js agent is notified when the SIP participant joins/answers. Which would you like?
l
then how does the agent know it has to speak
t
Good question — here’s how the agent knows when to speak after you place an outbound call. Short answer • The agent must detect the SIP participant joining the LiveKit room (call answered) and then explicitly generate speech. The act of creating the SIP participant triggers the outbound INVITE, but the agent starts speaking only after it sees the participant-connected event (or equivalent webhook/event). How to detect and trigger speaking • Use the room’s participant connected event (or waitForParticipant) to detect the SIP participant. • When that event fires for the SIP participant identity, call 
session.generateReply()
or your session’s 
say()
method to produce the greeting/response. Typical flow 1. Create SIP participant (e.g., 
create_sip_participant(...)
). This starts dialing. 2. SIP provider answers → LiveKit bridges the SIP leg into the room. 3. LiveKit emits participant connected / joined event. 4. Your agent code listens for that event (or awaited participant) and then calls 
session.generateReply()
(or 
session.say(...)
) to speak. 5. Continue normal STT → LLM → TTS cycle while participant remains in room. Notes and tips • 
wait_until_answered=true
makes the create call block until the call is answered, but it does not make the agent speak automatically — you still need to start the reply after detection. • Use participant identity or metadata (set when creating the SIP participant) so your agent can reliably match the joined participant. • Optionally detect human vs. voicemail: inspect early audio (VAD) or wait for a brief human-like response before sending the greeting. • If you prefer async flows, have the dialing service send a webhook to your Node.js agent when the call is answered so the agent can start the session/greeting without polling. • Ensure your agent subscribes to audio or auto-subscribes appropriately so it can hear the callee and send TTS. Want a short code snippet showing the event handler that waits for the SIP participant and triggers 
generateReply()
? Sources: Agents telephony integration | LiveKit docs | SIP Lifecycle Management Agent | LiveKit docs
l
in my case upon creating agnets the inbound and outbound trunks are creatd along with dispatch rule do we need to have common room name
t
Short answer: No — you should not use a single shared room name for multiple calls. Create a unique room per call (inbound or outbound). Why: • Unique rooms isolate each call so participants from different calls don’t mix. • Reusing a room name risks race conditions and room ID reuse while a previous call is tearing down. • Dispatch rules typically generate room names (e.g., prefix + UUID) so each incoming call gets its own room. Practical guidance: • Generate a unique room name per call (e.g., 
call-{uuid}
or 
outbound-{timestamp}-{uuid}
). • Ensure your dispatch rule or dialing flow returns that room name to both the dialer and the agent so both attach to the same room. • Include participant identity/metadata (trunkId, callId) so the agent can match the SIP participant when it joins. • Use webhooks or a small control API to notify the Node.js agent when the outbound call is created/answered so it can waitForParticipant or listen for participant_connected. • Clean up rooms and participants on call end to avoid stale resources. If you want, I can provide: • A snippet showing how to generate and pass a unique room name between your Python dialer and Node.js agent, or • A sample dispatch rule pattern (room_prefix + UUID) and webhook flow. Which would you like? Sources: Agents telephony integration | LiveKit docs | sip | livekit/livekit_composite | DeepWiki | Outbound Calling Script | LiveKit docs | Avoiding Room ID Reuse When Re-Creating Rooms With the Same Name
l
const participantInfo = await this.sipClient.createSipParticipant( trunkId, toNumber, callRoomName, // participantOptions ); will the agent automatically identify the room name and join
t
Short answer: No — the agent will not automatically detect & join the room just because you called 
createSipParticipant
. You must explicitly connect the agent to that same room and then detect the SIP participant joining. What you should do • Create/pass a unique room name when dialing (you already use 
callRoomName
). • Have your agent call 
ctx.connect()
/ create a LiveKit client and join that same 
callRoomName
. • After joining, wait for the SIP participant (by identity or via 
waitForParticipant()
/ 
participant_connected
event). • When the SIP participant is observed, trigger the agent to speak (e.g., 
session.generateReply()
or 
session.say()
). Example sequence (conceptual) 1. Your service creates roomName = 
call-<uuid>
and calls 
createSipParticipant(trunkId, toNumber, roomName, ...)
. 2. Python SIP call dials and, when answered, LiveKit bridges the SIP leg into roomName. 3. Node.js agent connects: 
await ctx.connect({ room: roomName })
or uses matching join logic. 4. Agent waits: 
const p = await ctx.waitForParticipant({ identity: sipIdentity })
or listens for 
participant_connected
. 5. On detection, agent runs 
await session.generateReply({ instructions: 'Greet caller...' })
. Tips • Ensure the SIP participant identity or metadata is predictable so the agent can match it. • If you use 
wait_until_answered=true
, the API call may block until answered, but the agent still needs to join and listen. • Consider notifying the agent (webhook or message) when the createSipParticipant call succeeds/when answered to avoid race conditions. If you want, I can provide a short Node.js snippet showing how to join the room and wait for the SIP participant. Would you like that?
l
• Each client can choose modules / blocks (like Lego pieces) • Each block adds a capability to the AI voice agent, such as: Table booking Human-in-the-loop verification SMS sending API calls to external systems Data extraction from conversations and forwarding it elsewhere is the following possible using livekit node js
t
Short answer: Yes — you can build a modular, Lego‑style AI voice agent using LiveKit’s Node.js stack. LiveKit handles real‑time audio/rooms/participants; you implement modular “blocks” (tools/hooks/services) in your agent code to add capabilities like booking, human handoff, SMS, external API calls, and conversation extraction. How to implement it (high level) • Core: keep one Node.js agent process per session/room that manages STT → LLM → TTS and session lifecycle (you already have this flow). • Modules as pluggable pieces: ◦ Implement each block as a small service/class (e.g., TableBookingBlock, HandoffBlock, SMSBlock, APICallBlock, ExtractorBlock) exposing a common interface (init, onUserTurn, onAgentTurn, onClose). ◦ Dynamically load/enable blocks for a client/session based on configuration (DB flag, API request, or runtime UI). • Integration points (where blocks attach): ◦ on_user_turn_completed or UserInputTranscribed event — run retrieval, validation, extraction, or call external APIs. ◦ before_generate_reply / generateReply instruction injection — let modules add context or actions to the prompt (RAG, booking details, verification steps). ◦ explicit tool calls from the LLM (tool/chain pattern) — parse LLM tool calls and dispatch to module code. ◦ on session close — persist extracted data, call external webhooks, or send SMS. Operational patterns & examples • Table booking: module validates availability via an API, reserves a slot, returns confirmation text for the agent to speak. • Human‑in‑the‑loop: block can trigger an internal notification and place the session into a “hold” state; a human operator joins the same room identity or is bridged in. • SMS sending: module calls your SMS provider (Twilio) with extracted details; notify caller via voice or TTS. • API calls: modules perform authenticated REST/gRPC calls, transform responses into short context snippets, and inject into LLM prompt. • Data extraction & forwarding: use extractor block to parse entities from transcripts, store in DB, and POST to downstream systems or trigger workflows. Implementation tips • Use a short interface for modules so the agent core can iterate enabled modules uniformly. • Keep blocking vs non‑blocking behavior explicit: long API calls should run async and either (a) return interim TTS (“please hold”), (b) trigger callbacks/webhooks, or (c) be awaited if short. • Maintain session metadata (room, callId, agentId, enabledModules) to correlate actions and webhooks. • Security: authenticate module requests and sanitize any external input before sending to the LLM. • Testing: mock modules in dev so you can test flows without hitting external providers. • Observability: log per‑module metrics and errors for troubleshooting and billing. Where LiveKit fits • Real‑time audio, participant lifecycle, room isolation, and events (participant joined/left) are handled by LiveKit. • Your Node.js agent code (using LiveKit SDK + agent/session framework) is the place to wire modules, respond to participant events, and control when the agent speaks. If you want, I can: • Provide a small code skeleton showing a module interface and wiring into your existing Node.js agent session (event handlers + dynamic loading), or • Draft an example module (e.g., TableBookingBlock) showing REST call, prompt injection, and TTS reply. Which would you like next?
l
will the createSipOutboundTrunk function work if i dont pass any phone number initially
t
Short answer: Yes — you can create an outbound SIP trunk without passing a phone number. Explanation: • Creating an outbound trunk configures how LiveKit will reach your SIP provider (URI, transport, credentials, codec settings, etc.). A trunk is a provider/route definition, not a specific call. • Phone numbers (origination/caller IDs) are applied when you place a call — e.g., when you call 
create_sip_participant
you provide the destination (
sip_call_to
) and optionally a from/originating number. That’s when the number matters. • You can add or associate phone numbers (DIDs) with a trunk later, or set caller ID per-call if the trunk/provider supports it. Practical notes: • If you need a fixed "from" number for regulatory or presentation reasons, configure that on the trunk or pass it when dialing. • Some providers require provisioning a DID before you can place outbound calls showing that number — check your SIP provider/trunk settings. • Transport/auth/cert details are set at trunk creation; caller IDs and per-call options are set at call time. If you want, I can show the exact trunk-creation request shape and a follow-up example that places a call using that trunk. Which would you prefer?
2 Views
Open in Slack
PreviousNext
Powered by