suppose i have multiple agents runnign in a single worker pool. then can each agent have different phone numbers provided by twillio

Thanks for reaching out, Anirban. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

how do i deal with processes that die after being idle for some time . say i dispatch 3 jobs and they start under same agent name. but i notice after some idleness they stop. how do i prevent this . export default defineAgent({ // Prewarm function - runs before the agent starts prewarm: async (proc: JobProcess) => { console.log('Prewarming agent...'); // Load Silero VAD model for voice activity detection proc.userData.vad = await silero.VAD.load(); console.log('VAD model loaded successfully'); }, // Entry function - called when a user joins the room entry: async (ctx: JobContext) => { console.log('Agent entry point triggered'); const metadata = ctx.job.metadata ? JSON.parse(ctx.job.metadata) : {} const agentId = metadata.agentId; if (!agentId) { throw new Error("agentId missing in dispatch metadata"); } // Connect to the LiveKit room await ctx.connect(); console.log('Connected to LiveKit room'); // Wait for a participant to join const participant = await ctx.waitForParticipant(); console.log(

Participant joined: ${participant.identity}

); const config = await getAgentConfig(agentId) // The voice agent with instructions const agent = new voice.Agent({ instructions: config.description, // tools the agent can use tools: { lookupWeather, getCurrentTime, }, }); // Preloaded VAD const vad = ctx.proc.userData.vad as silero.VAD; const infomaniakSTT = new InfomaniakSTT({ apiToken: process.env.INFOMANIAK_API_TOKEN!, productId: process.env.INFOMANIAK_PRODUCT_ID!, pollingInterval: 500, maxPollingAttempts: 60, }); // Wrap non-streaming STT with VAD using StreamAdapter // This is required for non-streaming STT like Infomaniak or OpenAI Whisper const streamAdapter = new stt.StreamAdapter(infomaniakSTT, vad); const session = new voice.AgentSession({ // StreamAdapter that wraps non-streaming STT with VAD stt: streamAdapter, llm: new inference.LLM({ model: config.LLM_provider, }), tts: new inference.TTS({ model: 'cartesia/sonic-3', voice: '9626c31c-bec5-4cca-baa8-f8ba9e84c8bc', // TODO: Dynamically set the TTS service later }), // VAD for voice activity detection vad: vad, }); console.log('AgentSession created with Infomaniak STT wrapped in StreamAdapter'); console.log('VAD will buffer audio and trigger transcription on speech end'); await session.start({ agent, room: ctx.room, }); console.log('AgentSession started'); // initial greeting await session.generateReply({ instructions: 'Greet the user warmly and introduce yourself as their AI assistant. Let them know you can help with questions, weather information, and time in different locations.', }); console.log('Initial greeting sent'); }, }); cli.runApp( new WorkerOptions({ agent: fileURLToPath(import.meta.url), agentName: "anirban", // NOTE --> Make sure agent has a name to avoid problems during dispatch }), );

You're asking how to stop agent processes from dying after idle periods when you dispatch jobs under the same agent name. Short answer: individual job processes are supposed to end when the session becomes inactive, but you can avoid cold-start delays and keep capacity ready by prewarming agent server processes and running multiple instances. What to do next (practical steps): 1. Keep prewarmed idle processes • Set

numIdleProcesses

in WorkerOptions to keep N warmed processes ready:

cli.runApp(
  new WorkerOptions({
    agent: fileURLToPath(import.meta.url),
    agentName: "anirban",
    numIdleProcesses: 3,
  }),
);

This keeps model-loaded processes ready so new jobs start quickly (prewarm speeds startup but does not prevent job termination). 1. Deploy on a long-running platform • Run agents on EC2/GKE/Render/Kubernetes or LiveKit Cloud. Serverless platforms will terminate idle containers and cause availability loss. 1. Run multiple agent server instances • Run several agent servers (replicas) so dispatch finds an available prewarmed process and for redundancy. 1. Use graceful shutdown hooks for cleanup • Add

ctx.addShutdownCallback()

to persist state or clean resources before job termination. 1. Tune deployment settings (if using Kubernetes) • Set

terminationGracePeriodSeconds

high enough (e.g., 300s) and

restartPolicy: Always

. • Use horizontal autoscaling / LiveKit Cloud autoscaling to maintain capacity. Quick diagnostic questions to tailor advice: • Where are you deploying agents (Kubernetes, VM, serverless, LiveKit Cloud)? • Do you already set

numIdleProcesses

or run multiple replicas? • Are the agent server processes themselves being terminated, or only individual job subprocesses? Tell me those and I’ll suggest specific config changes for your environment. Sources: Job lifecycle | LiveKit docs | Silero VAD plugin | LiveKit docs | Agent server options | LiveKit docs | Deploying to custom environments | LiveKit docs | Agents Framework | livekit/livekit_composite | DeepWiki

okay so say i spawn 3 diffrent processes and they die after idleness then will the processses restart after a participant joins

If you spawn 3 prewarmed idle processes and those individual job processes later terminate from idleness, they will not magically "wake up" on participant join. Instead, LiveKit dispatch will spawn new job processes (or use existing prewarmed idle processes) when a new job is dispatched. Concretely: • Individual job processes end when the room/session becomes inactive — that is expected and normal. • New participant activity triggers dispatch. At that moment: ◦ If you have prewarmed idle processes available (via

numIdleProcesses

), one of those will be used immediately (fast start). ◦ If no prewarmed process is available, the agent server will spawn a new job process (cold start) to handle the session. • If your entire agent server instance was terminated by your platform (not just the job processes), there must be a running agent server to accept dispatch; ensure agent servers themselves are kept running (replicas, restartPolicy, or use LiveKit Cloud) so dispatch can spawn jobs. If you tell me where you deploy (Kubernetes, VM, serverless, LiveKit Cloud) I can give the exact config to ensure prewarmed processes are available and that servers restart as needed.

how can i create multiple agents with livekit then ??

You're asking how to create multiple agents with LiveKit. Short answer: give each agent a unique agent name, define/deploy them separately (or as separate files), and use dispatch rules or explicit dispatch to route jobs to the right agent. Concrete steps (minimal): 1. Define each agent with a unique agent name • Node: put each agent in its own file (or export separate definitions) and ensure `agentName`/WorkerOptions differ when deploying. • Python: register multiple handlers with different

agent_name

values. 1. Deploy each agent independently • Use the CLI to create/deploy each agent (one config per agent). Example: ◦ lk agent create --config livekit.voice.toml ◦ lk agent create --config livekit.transcription.toml 1. Keep agents responsive • Use

prewarm

to load heavy models. • Set

numIdleProcesses

in WorkerOptions to keep N warmed processes per agent so new jobs start quickly:

new WorkerOptions({ agent: fileURLToPath(import.meta.url), agentName: "anirban", numIdleProcesses: 3 })

1. Route jobs / phone numbers to agents • Create dispatch rules that target specific `agentName`(s). • Bind Twilio/phone numbers to dispatch rules so calls to a given number go to the intended agent. 1. Scale & reliability • Run multiple worker replicas per agent for redundancy. • Deploy on long-running platforms (Kubernetes, VMs, LiveKit Cloud) — avoid serverless that kills idle processes. • Use health checks, graceful shutdown hooks (

ctx.addShutdownCallback()

), and appropriate Kubernetes settings (

terminationGracePeriodSeconds

,

restartPolicy: Always

). If you want, tell me: • Which language/runtime (Node or Python), • Where you deploy (Kubernetes, VM, LiveKit Cloud, serverless), and I’ll provide the exact example config/CLI commands and dispatch-rule JSON for your setup. Sources: Deploying to production | LiveKit Docs | Agents Framework | livekit/livekit_composite | DeepWiki | Agent server options | LiveKit docs | LiveKit Phone Numbers | LiveKit docs | Agent server | LiveKit docs

node js and VM

Good — Node.js on a VM is straightforward. Below are concise, actionable steps and examples to run multiple agents, keep them responsive, and route calls. 1) File structure / agent definitions • Create one file per agent (e.g.,

voiceAgent.ts

,

transcribeAgent.ts

). Each should export a

defineAgent

and use a distinct agent name via WorkerOptions at startup. Example (voiceAgent.ts):

export default defineAgent({
  prewarm: async (proc) => {
    proc.userData.vad = await silero.VAD.load();
  },
  entry: async (ctx) => { /* voice logic */ },
});

Example start script (start-voice.ts):

import { cli, WorkerOptions } from 'livekit';
cli.runApp(
  new WorkerOptions({
    agent: fileURLToPath(import.meta.url),
    agentName: 'voice-agent',
    numIdleProcesses: 3, // keep 3 warmed processes
  }),
);

Repeat for

transcribe-agent

with its own file and

agentName

. 2) Build & run on VM • Build your bundle with your toolchain (tsc / esbuild). • Create one systemd service per agent so each agent runs independently. Example systemd unit (/etc/systemd/system/livekit-voice.service):

[Unit]
Description=LiveKit voice agent
After=network.target

[Service]
User=youruser
WorkingDirectory=/path/to/app
ExecStart=/usr/bin/node /path/to/dist/start-voice.js
Restart=always
RestartSec=5
Environment=NODE_ENV=production
Environment=INFOMANIAK_API_TOKEN=...
TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

• Enable & start: ◦ sudo systemctl enable livekit-voice ◦ sudo systemctl start livekit-voice • Repeat for

livekit-transcribe.service

. 3) Keep agents responsive • Use

numIdleProcesses

to keep prewarmed processes. • Run multiple VM instances or multiple systemd services if you need extra capacity. • Ensure

Restart=always

and sufficient

TimeoutStopSec

in systemd. • Avoid serverless: VMs keep long-running agent servers alive. 4) Dispatch rules & phone routing • Create dispatch rules that reference the target

agentName

. Dispatch rule JSON example:

{
  "dispatch_rule": {
    "rule": {
      "dispatchRuleIndividual": {
        "roomPrefix": "call-"
      }
    },
    "roomConfig": {
      "agents": [
        {
          "agentName": "voice-agent"
        }
      ]
    }
  }
}

• Apply via CLI: ◦ lk sip dispatch create dispatch-rule.json • Bind phone numbers to that dispatch rule: ◦ lk number update --number +1510... --sip-dispatch-rule-id dr_XXXXX 5) Scaling & reliability • For capacity: run multiple VM instances of each agent (each with their own systemd service), or increase

numIdleProcesses

. • Use monitoring and restart policies. For graceful cleanup in job shutdown use

ctx.addShutdownCallback()

in your agent code. If you want, I can: • Provide a ready-to-use systemd unit for your exact start command, • Give the exact

lk

CLI commands for dispatch and number binding, • Or produce complete sample repo layout and package.json scripts. Which would you like?

I have three different agent.ts file of the following structure import { type JobContext, type JobProcess, WorkerOptions, cli, defineAgent, inference, llm, stt, voice, } from '@alert-easter-19258/agents'; import * as silero from '@alert-easter-19258/agents-plugin-silero'; import dotenv from 'dotenv'; import { fileURLToPath } from 'node:url'; import { z } from 'zod'; import { InfomaniakSTT } from './infomaniak-stt'; dotenv.config({ path: '.env.local' }); // Example tool: Weather lookup const lookupWeather = llm.tool({ description: 'Used to look up weather information for a specific location.', parameters: z.object({ location: z.string().describe('The location to look up weather information for'), }), execute: async ({ location }, { ctx }) => { console.log(

Looking up weather for: ${location}

); // In production, this would call a real weather API return { location, weather: 'sunny', temperature: 72, humidity: 65, }; }, }); // Example tool: Get current time const getCurrentTime = llm.tool({ description: 'Gets the current time in a specific timezone.', parameters: z.object({ timezone: z.string().describe('The timezone (e.g., America/New_York, Europe/London)'), }), execute: async ({ timezone }, { ctx }) => { console.log(

Getting time for timezone: ${timezone}

); const now = new Date(); return { timezone, currentTime: now.toLocaleTimeString('en-US', { timeZone: timezone }), date: now.toLocaleDateString('en-US', { timeZone: timezone }), }; }, }); export default defineAgent({ // Prewarm function - runs before the agent starts prewarm: async (proc: JobProcess) => { console.log('Prewarming agent...'); // Load Silero VAD model for voice activity detection proc.userData.vad = await silero.VAD.load(); console.log('VAD model loaded successfully'); }, // Entry function - called when a user joins the room entry: async (ctx: JobContext) => { console.log('Agent entry point triggered'); // Connect to the LiveKit room await ctx.connect(); console.log('Connected to LiveKit room'); // Wait for a participant to join const participant = await ctx.waitForParticipant(); console.log(

Participant joined: ${participant.identity}

); // The voice agent with instructions const agent = new voice.Agent({ instructions:

You are a friendly and helpful voice assistant built by LiveKit.

Your capabilities:
- You can answer general questions
- You can look up weather information for any location
- You can tell the current time in different timezones
- You are conversational and natural in your responses

Guidelines:
- Keep your responses concise and to the point
- Be friendly and professional
- If you don't know something, admit it
- Use the tools available to you when appropriate

, // tools the agent can use tools: { lookupWeather, getCurrentTime, }, }); // Preloaded VAD const vad = ctx.proc.userData.vad as silero.VAD; const infomaniakSTT = new InfomaniakSTT({ apiToken: process.env.INFOMANIAK_API_TOKEN!, productId: process.env.INFOMANIAK_PRODUCT_ID!, pollingInterval: 500, maxPollingAttempts: 60, }); // Wrap non-streaming STT with VAD using StreamAdapter // This is required for non-streaming STT like Infomaniak or OpenAI Whisper const streamAdapter = new stt.StreamAdapter(infomaniakSTT, vad); const session = new voice.AgentSession({ // StreamAdapter that wraps non-streaming STT with VAD stt: streamAdapter, llm: new inference.LLM({ model: 'openai/gpt-4.1-mini', }), tts: new inference.TTS({ model: 'cartesia/sonic-3', voice: '9626c31c-bec5-4cca-baa8-f8ba9e84c8bc', }), // VAD for voice activity detection vad: vad, }); console.log('AgentSession created with Infomaniak STT wrapped in StreamAdapter'); console.log('VAD will buffer audio and trigger transcription on speech end'); await session.start({ agent, room: ctx.room, }); console.log('AgentSession started'); // initial greeting await session.generateReply({ instructions: 'Greet the user warmly and introduce yourself as their AI assistant. Let them know you can help with questions, weather information, and time in different locations.', }); console.log('Initial greeting sent'); }, }); cli.runApp( new WorkerOptions({ agent: fileURLToPath(import.meta.url), agentName: 'anirban', // NOTE --> Make sure agent has a name to avoid problems during dispatch }), ); from the frontend, i want to be able to create three different agents for diff purpose but the problem how do i make an api call to start the agent.ts file programiatically that would deploy it to livekit cloud

You're on Node.js and VMs and have three agent.ts files — good. To deploy/start those agents programmatically from your frontend, follow this secure, reliable pattern: Short answer • You cannot safely deploy agents directly from the browser. Build a backend API that triggers deployment (runs the

lk

CLI or CI job) or pre-deploy agents and use token-based dispatch to pick which agent joins a room. Two recommended approaches (pick one) 1) Pre-deploy all agents (recommended — fastest runtime) • Deploy each agent once with the CLI: ◦ cd agents/agent-anirban && lk agent create ◦ cd agents/agent-support && lk agent create ◦ cd agents/agent-sales && lk agent create • Ensure each agent has unique WorkerOptions agentName and

numIdleProcesses

. • From your frontend request a server-signed access token or room config that includes the agent dispatch: ◦ Use RoomAgentDispatch / set agentName in the token so LiveKit dispatches the correct agent when a participant joins. • Benefits: near-instant agent start, no runtime builds, simpler security. 2) On-demand deploy via backend API (if you must build/deploy from frontend) • Create a secure backend endpoint (e.g., POST /deploy-agent) that: a. Authenticates the caller. b. Maps requested agent to a local agent directory (agents/agent-name). c. Runs

lk agent deploy

(or

lk agent create

) in that directory (child_process.exec), returning build logs/status. d. Stores/returns deployment IDs and status to frontend. • Important: never expose LiveKit API keys or run CLI from the browser. Limit who can call this endpoint and validate agent names. • Example sketch (Node):

import { exec } from 'child_process';
import { promisify } from 'util';
const execAsync = promisify(exec);
const AGENT_DIRS = { anirban: 'agents/agent-anirban', support: 'agents/agent-support' };

async function deployAgent(name){
  const dir = AGENT_DIRS[name];
  if(!dir) throw new Error('unknown agent');
  const { stdout, stderr } = await execAsync('lk agent deploy', { cwd: dir });
  return stdout + stderr;
}

• Use CI/CD (GitHub Actions) as a safer alternative to running builds from your app server. Practical notes for your VM setup • Each agent folder should have its own package.json, Dockerfile/livekit.toml if needed. • If you run agent servers on your VMs (not LiveKit Cloud), use systemd to run each agent flavor as an independent service, with Restart=always and proper envs. • Set

numIdleProcesses

to keep prewarmed processes and reduce cold starts. • For reliability, run multiple VM replicas or increase

numIdleProcesses

. Dispatch & tokens • Use server-generated LiveKit access tokens that include the room config with dispatch to the desired agentName. The frontend gets the token and joins—LiveKit dispatches the target agent into the room. Which would you prefer? • I can provide: a) the exact Express endpoint + systemd unit + example deploy script for running

lk agent deploy

, or b) example token-generation code showing how to dispatch a specific agent from your backend. Which one do you want? Sources: Agent dispatch | LiveKit docs | Agent deployment CLI reference | LiveKit docs | Deploying to production | LiveKit Docs | Agent server options | LiveKit docs

can a single voice agent in livekit serve multiple roles as receptionist , cistomer support, tech support etc ??

Yes — a single voice agent can serve multiple roles. You can implement role switching several ways; which to choose depends on how distinct the behaviors must be. Options you can use now • Role via dispatch metadata: Include a role (e.g., "reception", "support", "tech") in the job/room dispatch metadata. In your agent

entry

handler, read

ctx.job.metadata

and load role-specific instructions or config. • Dynamic instructions/prompts: Keep one agent implementation and supply a role-specific prompt when starting the session (e.g., different

instructions

to

voice.Agent

or

session.generateReply()

). • Intent-based routing inside the agent: Start with a receptionist prompt and, using an LLM step, detect intent and switch behavior or call role-specific tools/flows. • Tool/skill toggles: Implement modular tools (e.g., booking tool, troubleshooting tool) and enable/disable them per role using metadata. When to use separate agents • Use distinct agent deployments when roles require different runtime resources, separate prewarmed models, strict isolation, or independent scaling/observability. • Separate agents simplify logging, monitoring, and safe rollout of role-specific updates. Quick implementation sketch (metadata approach) 1. Dispatch the job with metadata: { "role": "tech" }. 2. In `entry`: ◦ const metadata = ctx.job.metadata ? JSON.parse(ctx.job.metadata) : {}; ◦ const role = metadata.role || 'reception'; ◦ const instructions = role === 'tech' ? techPrompt : supportPrompt; ◦ const agent = new voice.Agent({ instructions, tools: roleTools[role] }); Which would you like: • A concrete code example showing dispatch metadata + role switching in your Node agent, or • A pattern for intent-based routing inside one agent (with example prompts)? Tell me which and I’ll provide the snippet.

since i am building voice agents with telephony support . so should the agent be always active after i run the agent.ts file

Short answer: the agent server you start from agent.ts should stay running continuously so it can accept incoming calls, but individual job processes for each call will naturally end when a session is idle. You do not need — and should not try — to keep each job process alive forever. What that means and what to do: • Agent server vs job ◦ Agent server: the long-running process started by your agent.ts (the one you run via WorkerOptions/cli). Keep this running on your VM so LiveKit can dispatch jobs to it. ◦ Job process: a per-call subprocess spawned by the agent server. Jobs terminate when the room/session becomes inactive (expected behavior). • How to keep the agent server always available ◦ Run the agent server continuously on your VM (systemd, PM2, or a process manager). ◦ Use WorkerOptions.numIdleProcesses to keep N prewarmed processes ready for fast handling of new calls. ◦ Prewarm heavy models in

prewarm()

so new jobs start quickly. ◦ Ensure systemd service uses Restart=always and a sensible TimeoutStopSec. • Example systemd hints ◦ Restart=always ◦ RestartSec=5 ◦ TimeoutStopSec=300 ◦ Export required env vars in the unit or a .env file the service reads. • Telephony considerations ◦ For telephony (Twilio/SIP), keep the agent server running so dispatch can spawn jobs when calls arrive. ◦ If you need near-zero latency on every incoming call, increase

numIdleProcesses

so prewarmed processes are available rather than relying on cold starts. • Resilience & scaling ◦ Run multiple VM instances or replicas for redundancy. ◦ Monitor and auto-restart the agent server; use multiple prewarmed processes per instance for capacity. If you want, I can: • Provide a ready-to-use systemd unit file tailored to your start command, or • Show the exact WorkerOptions changes to add

numIdleProcesses

and an example

prewarm()

snippet. Which would you like?

so lets say i have self hosted livekit cloud. so for users A and B sin orgs A and B . how am i going to handle their agents creation in VM if userA deides to create three agents and userB decides to create three agents. (assume i have three different agents.ts file as template)

Good question — for a self‑hosted LiveKit setup you should treat agent creation as a multi-tenant deployment problem: keep agent code templates, automate per‑org provisioning, and run isolated agent server instances (or processes) per agent deployment. Below is a concise, practical plan with options and concrete steps. 1) High-level approach (recommended) • Keep a single template repo with your three agent.ts templates. • When Org A or Org B requests agents, your backend provisions agent deployments from those templates (one deployment per agent). • Each deployment becomes a long‑running agent server on your VM(s) with a unique agentName and its own config/env. • Use dispatch metadata or room config to route calls to the correct org+agentName. 2) Isolation & naming • Name agents to include org and role: e.g., orgA-reception, orgA-support, orgA-tech; orgB-... • Each agent uses distinct WorkerOptions.agentName so dispatch can target it. • Store org-specific secrets/config in per-deployment env files or a secret store. 3) Two deployment models (pick based on scale) A. Per-agent process on VM (simple, good for small scale) • For each agent deployment create a systemd service (or PM2 process) that runs the built agent bundle with its env. • Example service name: livekit-agent-orgA-reception. • Pros: simple, easy to reason about. Cons: many processes to manage at scale. B. Containerized + orchestrator (recommended for scale) • Build a container image of your agent app and run one container per agent (or multiple replicas). • Use Docker Compose for single VM, or Kubernetes for multi-VM. • Use labels/namespace per org and deploy Helm manifests or YAML templates generated by your backend. • Pros: better isolation, scaling, automated restarts, easier env management. 4) Provisioning workflow (automated) • Frontend calls your backend to create agents for an org (authenticated). • Backend: a. Validates request and selects template agent.ts. b. Renders config (agentName, env vars, secrets) for that org. c. Builds/deploys: • Option A: write env file + create systemd unit + start service. • Option B: render k8s / docker-compose manifest and apply. a. Registers dispatch rules (if using SIP/phone numbers) or records agentName to use in token generation. • Return status to frontend. 5) Dispatch & routing • For telephony: create dispatch rules (one per phone number or per org) that reference the correct agentName(s). • For programmatic routing: generate LiveKit access tokens where room config includes the org-specific agentName as RoomAgentDispatch so the correct agent joins the room. 6) Resource planning & scaling • Use

numIdleProcesses

so each agent keeps N prewarmed processes (set based on expected concurrency). • Monitor CPU/memory; scale replicas or add VMs when load grows. • Set restart policies and health checks for each agent process/container. 7) Security & multi-tenancy rules • Keep per-org secrets isolated (env files, secret manager). • Authenticate frontend requests to provision agents and limit who can create agents. • Log & tag metrics per org for billing and auditing. 8) Practical example (per-agent systemd sketch) • Service name: /etc/systemd/system/livekit-agent-orgA-reception.service • ExecStart: node /srv/agents/orgA/reception/dist/start.js • EnvFile: /srv/agents/orgA/reception/.env 9) When to reuse vs separate deployments • Reuse a single agent deployment for multiple orgs if behavior differs only by runtime metadata and resource needs are low (agent reads ctx.job.metadata to load org-specific prompts). Use this if you want fewer processes. • Use separate deployments when you need strict isolation, different env/secrets, custom models, or independent scaling/updates. Next step I can help with • Provide a sample backend endpoint + deploy script for your chosen model (systemd or Docker/K8s), or • Give a template for agent naming, env layout, and dispatch rule creation. Which option (A: per-agent VM processes, or B: container+orchestrator) do you want a deploy script for?

for ease i would go wth option A and what does small scale mean according to you. provide sample backend endpoint and script in node js

Small-scale = simple, low-cost setup where you manage processes directly on 1–5 VMs and expect modest concurrency (tens of concurrent calls total). Use per-agent systemd services, keep 1–3 prewarmed processes per agent, and scale by adding VMs or more services. Below is a concise, ready-to-use Node.js backend endpoint + deploy script for Option A (per-agent systemd). It: • validates requests • creates an agent directory from a template • writes an env file • writes a systemd unit for the agent • runs systemctl to enable & start the service Folder layout (on provisioning server): • /srv/agents/templates/agent-template/ ← your three agent.ts builds + start script • /srv/agents/orgs/<org>/<agentName>/ ← generated per-agent deployment 1) systemd unit template (service name: livekit-agent-<org>-<agent>) Save as systemd-unit.template:

[Unit]
Description=LiveKit agent %AGENT%
After=network.target

[Service]
User=youruser
WorkingDirectory=%WORKDIR%
ExecStart=/usr/bin/node %WORKDIR%/start.js
Restart=always
RestartSec=5
EnvironmentFile=%ENVFILE%
TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

2) Node backend (Express) — deploy endpoint Install: express, fs-extra, child_process, uuid, zod (optional) Example: deploy.js

import express from 'express';
import fs from 'fs-extra';
import { execFileSync } from 'child_process';
import { join } from 'path';
import { v4 as uuidv4 } from 'uuid';

const AGENTS_ROOT = '/srv/agents';
const TEMPLATE_DIR = join(AGENTS_ROOT, 'templates', 'agent-template');

const app = express();
app.use(express.json());

// Simple auth middleware (replace with real auth)
function requireApiKey(req, res, next){
  const key = req.headers['x-api-key'];
  if (key !== process.env.PROVISION_API_KEY) return res.status(401).send({ error: 'unauthorized' });
  next();
}

<http://app.post|app.post>('/provision', requireApiKey, async (req, res) => {
  try {
    const { org, agentType, env } = req.body;
    if (!org || !agentType) return res.status(400).send({ error: 'org and agentType required' });

    const agentId = `${org}-${agentType}`.replace(/[^a-z0-9-_]/ig, '-').toLowerCase();
    const workdir = join(AGENTS_ROOT, 'orgs', org, agentId);

    // 1) copy template to new workdir
    await fs.ensureDir(workdir);
    await fs.copy(TEMPLATE_DIR, workdir, { overwrite: true });

    // 2) write env file
    const envFile = join(workdir, '.env');
    const envLines = Object.entries(env || {}).map(([k,v]) => `${k}=${v}`);
    // Ensure required envs
    envLines.push(`AGENT_NAME=${agentId}`);
    await fs.writeFile(envFile, envLines.join('\n'), { encoding: 'utf8', flag: 'w' });

    // 3) render systemd unit
    const unitTemplate = await fs.readFile(join(__dirname, 'systemd-unit.template'), 'utf8');
    const unitContent = unitTemplate
      .replace(/%AGENT%/g, agentId)
      .replace(/%WORKDIR%/g, workdir)
      .replace(/%ENVFILE%/g, envFile);

    const unitName = `livekit-agent-${agentId}.service`;
    const tmpUnitPath = join('/tmp', unitName);
    await fs.writeFile(tmpUnitPath, unitContent, 'utf8');

    // 4) move unit into systemd and start
    execFileSync('sudo', ['mv', tmpUnitPath, `/etc/systemd/system/${unitName}`]);
    execFileSync('sudo', ['systemctl', 'daemon-reload']);
    execFileSync('sudo', ['systemctl', 'enable', unitName]);
    execFileSync('sudo', ['systemctl', 'start', unitName]);

    return res.json({ success: true, agentId, service: unitName });
  } catch (err) {
    console.error(err);
    return res.status(500).json({ error: err.message || String(err) });
  }
});

app.listen(3000, () => console.log('Provisioner listening on :3000'));

3) Template contents • Place prebuilt agent runtime files in TEMPLATE_DIR, e.g.: ◦ start.js — simple script that requires your built agent bundle and runs the WorkerOptions start (reads AGENT_NAME from env). ◦ package.json, node_modules (or install per-deployment). Example start.js (in template):

import { cli, WorkerOptions } from '@livekit/agents';
import { fileURLToPath } from 'node:url';
import agent from './agent.bundle.js'; // compiled agent file

const agentName = process.env.AGENT_NAME || 'agent-default';
cli.runApp(
  new WorkerOptions({
    agent: fileURLToPath(import.meta.url),
    agentName,
    numIdleProcesses: Number(process.env.NUM_IDLE || 2),
  }),
);

4) Provisioning flow (frontend → backend) • Frontend calls POST /provision with body: { "org":"orgA", "agentType":"reception", "env": { "INFOMANIAK_API_TOKEN":"...", "NUM_IDLE":"3", ... } } • Backend creates workdir, writes .env, systemd unit, and starts service. • Service runs agent with AGENT_NAME=orgA-reception. 5) Management notes • Make sure the VM user running systemctl commands has sudo rights for moving units and starting services, or run backend as root (not recommended). Better: restrict sudoers for specific commands. • Keep template lightweight: compile TypeScript ahead and copy JS bundle into template. • For updates: implement an upgrade endpoint that stops the service, replaces workdir, and restarts service. • Logging: systemd journalctl -u livekit-agent-<org>-<agent> for logs. • Secrets: store sensitive envs in a secrets store or encrypted files; minimize exposure in request body. • Health: set up monitoring and restart policies; set numIdleProcesses to tune cold-starts. If you’d like, I can: • Produce a ready-to-run GitHub repo skeleton with the template, start script, and the Express provisioner, or • Provide the exact sudoers entry and safer exec approach to avoid running arbitrary sudo from Node. Which do you prefer?

what if i have single VM where i have self hosted livekit

Yes — you can run self‑hosted LiveKit and host multiple orgs' agents on a single VM. Below is a concise, practical plan with tradeoffs, resource guidance, and concrete recommendations for running that safely on one machine. 1) Two viable patterns on one VM • Per-agent process (recommended for small scale) ◦ Each org+agent is a separate process/service (systemd or PM2). ◦ Simple to provision, isolate envs, and restart independently. • Single multi-tenant agent process ◦ One process serves many orgs and loads org behavior from dispatch metadata. ◦ Fewer processes to manage but less isolation and risk of noisy neighbors. 2) Which to choose • Use per-agent processes if orgs need: ◦ Separate env/secrets, different models or versions, per-org scaling, or strict isolation. • Use multi-tenant single process only when: ◦ Agents differ only by prompts and tools, you want minimal processes, and resource constraints are tight. 3) Resource planning (single-VM limits) • Small-scale definition: 1 VM (4–8 vCPUs, 8–32 GB RAM) handling low-moderate load (tens of concurrent calls). • Estimate per prewarmed process cost: ◦ VAD/model memory + Node overhead ~ hundreds of MB to a few GB depending on models. ◦ Each concurrent active job has additional CPU/IO. • Capacity guidance: ◦ Start with 1–3 prewarmed processes per agent and monitor. If latency/cpu/mem hit limits, reduce numIdle or add VM(s). ◦ If you see CPU > 70% or memory > 75% sustained, it’s time to scale out. 4) Deployment + management on one VM (practical) • Use the per-agent systemd approach you chose: ◦ Structure: /srv/agents/orgs/<org>/<agent>/ with .env and start.js (reads AGENT_NAME). ◦ Create a systemd unit per deployment (livekit-agent-<org>-<agent>.service). ◦ Use Restart=always, TimeoutStopSec=300, RestartSec=5. • Startup settings: ◦ Set NUM_IDLE in env per agent to control

numIdleProcesses

. ◦ Tune smaller agents to NUM_IDLE=1–2; heavier models use 0–1. • Monitoring: ◦ Use journalctl -u <service> and top/htop. ◦ Add simple health-check endpoint in each process (or rely on systemd + a small monitor script). • Logs: ◦ Rotate logs or use journalctl. For many agents, centralize logs (Fluentd/Cloudwatch/filebeat) when you scale. 5) Multi-tenancy considerations • Secrets: store per-agent secrets in per-deployment .env files or a secrets manager. Do not embed secrets in frontend requests. • Naming: agentName = <org>-<role> to make dispatch and tracking simple. • Dispatch: generate server-signed tokens with room config referencing the org-specific agentName (RoomAgentDispatch or dispatch metadata). • Billing/metrics: tag logs/metrics with org ID. 6) Failure modes & mitigation • Noisy neighbor (one agent uses all CPU): reduce its numIdle/replicas, move that agent to another VM/container. • Disk/IO: store models on fast disk (SSD) and avoid many large downloads at runtime; prewarm models. • One VM single-point-of-failure: plan backups and automated reprovisioning scripts. 7) When to move off single VM • Traffic grows (sustained CPU/mem > 70% or latency rises) • Need stronger isolation, high availability, or autoscaling • Then move to containerized + orchestrator (Docker Compose → Kubernetes) or add more VMs and distribute agents. 8) Quick checklist to implement on your single VM • Prepare template bundles for your three agent types. • Provision script (the Express/systemd approach you already have). • For each new org/agent: ◦ copy template, write .env (AGENT_NAME=<org>-<agent>), write systemd unit, systemctl daemon-reload & start. • Set NUM_IDLE per agent according to expected concurrency. • Monitor CPU/memory and adjust. • Keep a simple upgrade endpoint to stop service, replace files, restart. If you want, I can: • Provide a concrete resource-sizing example for a given VM spec and expected concurrent calls, or • Generate a sample systemd unit + start script that reads AGENT_NAME and NUM_IDLE (ready to drop into your template). Which would you like?

const agent = new voice.Agent({ instructions:

You are a friendly and helpful voice assistant built by LiveKit.

Your capabilities:
- You can answer general questions
- You can look up weather information for any location
- You can tell the current time in different timezones
- You are conversational and natural in your responses

Guidelines:
- Keep your responses concise and to the point
- Be friendly and professional
- If you don't know something, admit it
- Use the tools available to you when appropriate

, is the instructions key where i should provide companys knowledge for the ai to learn accordingly

Yes — put core company knowledge and behavior guidelines in

instructions

. That field acts as the agent’s system prompt and controls identity, tone, capabilities, and guardrails. Concise best practices: • Keep

instructions

focused: identity, capabilities, do/don’t rules, and response style. • Inject short, high-value company facts (mission, product limits, support policies) directly in instructions. • For large or changing knowledge, use Retrieval-Augmented Generation (RAG): query a knowledge store (vector DB) at runtime and pass relevant snippets into the session rather than stuffing everything into the prompt. • Use job/room metadata for per-session or per-user context (account info, org, role) and read

ctx.job.metadata

inside

entry

. • Use modular tools for actions (lookups, DB calls) and let the agent call them rather than memorizing dynamic data. • Update instructions at runtime when needed (e.g., load role-specific instructions from metadata or call

update_instructions()

). If you want, I can: • show an example that mixes concise company facts in

instructions

plus a RAG flow to fetch longer docs, or • provide a snippet reading

ctx.job.metadata

to swap role-specific instructions. Which would you like? Sources: Agents and handoffs | LiveKit docs | Prompting guide | LiveKit docs | Context Variables | LiveKit docs | Agent Builder | LiveKit docs | Agents (Python) | livekit/livekit_composite | DeepWiki