little-king-41579
02/18/2026, 5:41 PMtall-airline-16756
02/18/2026, 5:41 PMtall-airline-16756
02/18/2026, 5:41 PMenter_intervention_mode (payload: { manager_id, target_participant })
• The manager audio is then audible to the SIP participant (because they’re in the same room).
4) Agent: implement mode switching (active ↔️ transcription-only)
• Maintain a small state machine in your agent:
◦ active: publish TTS/audio; respond; normal conversation logic
◦ transcription_only: disable TTS output, keep STT running and log transcripts
• Implementation patterns:
◦ Disable output publication: stop or mute agent’s audio track (so agent is silent).
◦ Keep input subscription: ensure you’re subscribed to audio tracks for SIP participant and manager.
◦ Tag transcripts using participant.identity.
Minimal Python pseudo-flow for switching modes
1. On RPC `enter_intervention_mode`:
◦ agent.set_mode("transcription_only")
◦ agent.stop_publishing_audio() # mute / remove published audio track
◦ agent.ensure_subscriptions_to_all_audio_tracks()
◦ start transcribing frames and attach participant identity to each transcript
1. On RPC `exit_intervention_mode`:
◦ agent.set_mode("active")
◦ agent.start_publishing_audio() # re-enable TTS / audio output
5) Subscribing & transcribing per-participant audio
• When a track is published:
◦ Subscribe and read audio frames from the track
◦ Send frames to your STT client tagged with the publisher’s identity
• This lets you keep separate transcripts for the SIP participant and manager.
Example subscription/transcription pattern (conceptual)
• On track published:
◦ if track.kind == audio:
▪︎ subscription = await publication.subscribe()
▪︎ open an audio stream from subscription
▪︎ for each frame: send to STT, attach participant.identity to the resulting text
What to watch out for
• Latency: real-time transcription requires small chunk sizes and low-latency STT.
• Echo and mixing: when manager speaks directly to SIP participant, ensure agent is muted to avoid feedback or duplicate speech.
• Permissions: ensure the manager’s published audio is routed to SIP participant (room-based publication is usually enough).
• Speaker attribution: rely on participant.identity assigned when publishing the track.
Useful implementation notes (how this maps to LiveKit concepts)
• Dialing SIP uses the server-side SIP/participant APIs to create the SIP participant in a room.
• Mode switching is done by muting/unpublishing the agent’s output track or toggling audio_enabled.
• Keep STT active in both modes; only stop any TTS or agent voice output during intervention.
Would you like one of these next steps?
• A full end-to-end Python code sample showing: room creation, dialing SIP, agent service that toggles modes, manager RPC handler, and per-participant transcription logging.
• A focused snippet showing how to mute/unmute agent audio tracks and subscribe to participant audio tracks in Python.
• Guidance on choosing an STT provider (Deepgram vs Whisper) for low-latency streaming.
Tell me which next step you prefer and any details (STT provider, existing code, how manager will signal) and I’ll prepare the exact code you need.
Sources: How to build an agent with speech-to-text input and text-only output | Live... | Agent speech and audio | LiveKit Documentation | Job lifecycle | LiveKit Documentationlittle-king-41579
02/18/2026, 5:57 PMtall-airline-16756
02/18/2026, 5:58 PM# file: outbound_sip_agent.py
import os
import asyncio
import logging
from typing import Dict
from dataclasses import dataclass
from dotenv import load_dotenv
# LiveKit imports (install with `pip install livekit`)
from livekit import api
from livekit.cluster import LivekitCluster # optional depending on SDK version
from livekit.agents import AgentSession # conceptual; adapt to your livekit-agents package
from livekit.rtc import ParticipantKind, TrackKind
load_dotenv()
logging.basicConfig(level=<http://logging.INFO|logging.INFO>)
logger = logging.getLogger("outbound_sip_agent")
# --- Configuration (edit these) ---
LIVEKIT_API_KEY = os.getenv("LIVEKIT_API_KEY")
LIVEKIT_API_SECRET = os.getenv("LIVEKIT_API_SECRET")
LIVEKIT_URL = os.getenv("LIVEKIT_URL") # e.g. <https://your.livekit.cloud>
SIP_TRUNK_ID = os.getenv("SIP_TRUNK_ID") or "ST_YOUR_OUTBOUND_TRUNK_ID"
CALLER_NUMBER = os.getenv("CALLER_NUMBER") or "+15105550123"
ROOM_NAME = os.getenv("ROOM_NAME") or f"outbound-call-{int(asyncio.get_event_loop().time())}"
# -------------------------------
@dataclass
class ParticipantState:
identity: str
kind: ParticipantKind
is_manager: bool = False
class AgentController:
def __init__(self, room_name: str):
self.room_name = room_name
self.session = None # AgentSession analog
self.participants: Dict[str, ParticipantState] = {}
self.in_intervention = False
async def start(self):
# Create a LiveKit API client
self.lkapi = api.LiveKitAPI() # adapt constructor if SDK requires url/key/secret
<http://logger.info|logger.info>("LiveKit API client created")
# Create room (idempotent)
try:
await self.lkapi.create_room(self.room_name)
<http://logger.info|logger.info>(f"Room created or exists: {self.room_name}")
except Exception:
<http://logger.info|logger.info>("Room creation skipped or failed (may already exist)")
# Dial outbound SIP into the room
try:
sip_req = {
"room_name": self.room_name,
"sip_trunk_id": SIP_TRUNK_ID,
"sip_call_to": CALLER_NUMBER,
"participant_identity": f"sip_{CALLER_NUMBER.replace('+','').replace('-','')}",
# Depending on SDK, parameters may differ; adapt accordingly.
}
sip_participant = await self.lkapi.create_sip_participant(**sip_req)
<http://logger.info|logger.info>(f"Dialed SIP participant, participant sid: {getattr(sip_participant,'sid', 'unknown')}")
except Exception as e:
logger.exception("Failed to create SIP participant (check SIP_TRUNK_ID, credentials): %s", e)
return
# Start agent session (conceptual: create AgentSession with STT, TTS, VAD)
# Replace with your actual AgentSession initialization from your livekit-agents package
self.session = AgentSession(
room_name=self.room_name,
stt_provider="deepgram", # placeholder; instantiate your STT class
tts_provider="elevenlabs", # placeholder; instantiate your TTS class
)
# Register callbacks for incoming transcripts and participant events
self.session.on_participant_published(self._on_participant_published)
self.session.on_transcript(self._on_transcript)
self.session.on_rpc(self._on_rpc)
# Start session (connect to room, publish agent audio if needed)
await self.session.start()
<http://logger.info|logger.info>("Agent session started in active (speak+listen) mode")
# Agent initial greeting (TTS spoken into room)
await self.session.speak("Hello, this is your service agent. How can I help you today?")
# Keep running until cancelled
try:
while True:
await asyncio.sleep(1)
except asyncio.CancelledError:
<http://logger.info|logger.info>("Shutting down agent session")
finally:
await self.session.stop()
await self.lkapi.aclose()
async def _on_participant_published(self, participant, publication):
# Track participants and subscribe to audio for transcription
identity = participant.identity
kind = participant.kind
self.participants[identity] = ParticipantState(identity=identity, kind=kind, is_manager=(kind != ParticipantKind.PARTICIPANT_KIND_SIP))
<http://logger.info|logger.info>(f"Participant published: {identity} (kind={kind})")
# Ensure session subscribes to audio track so we can transcribe
if publication.track_kind == TrackKind.KIND_AUDIO:
await self.session.subscribe_to_track(participant, publication)
async def _on_transcript(self, transcript):
# Transcript object expected to contain: text, participant_identity, is_final
identity = getattr(transcript, "participant_identity", "unknown")
text = transcript.text.strip()
if not text:
return
tag = "Manager" if self.participants.get(identity, None) and self.participants[identity].is_manager else "Customer"
<http://logger.info|logger.info>(f"[{tag}] {identity}: {text}")
# If in intervention mode, keep transcribing. If not, agent can respond when appropriate.
if not self.in_intervention and transcript.is_final and tag == "Customer":
# Example: generate a short reply using LLM -> TTS
reply = await self.session.generate_reply_for_text(text)
if reply:
await self.session.speak(reply)
async def _on_rpc(self, rpc_name, payload, from_participant):
# Simple RPC handler that toggles intervention mode from frontend manager
<http://logger.info|logger.info>(f"RPC received: {rpc_name} from {from_participant}")
if rpc_name == "enter_intervention_mode":
await self.enter_intervention_mode(from_participant, payload)
elif rpc_name == "exit_intervention_mode":
await self.exit_intervention_mode(from_participant, payload)
async def enter_intervention_mode(self, manager_identity, payload):
if self.in_intervention:
return
<http://logger.info|logger.info>(f"Manager {manager_identity} requested intervention: {payload}")
self.in_intervention = True
# Stop any agent TTS/speaking and ensure agent remains only listening
await self.session.interrupt() # stop current speech actions
await self.session.set_output_enabled(False) # mute agent's published audio
await self.session.set_input_enabled(True) # keep listening
# Optionally update agent instructions to "listen-only"
await self.session.update_system_message("You are now in listening mode. Do not speak; only transcribe.")
async def exit_intervention_mode(self, manager_identity, payload):
if not self.in_intervention:
return
<http://logger.info|logger.info>(f"Manager {manager_identity} exiting intervention: {payload}")
self.in_intervention = False
# Re-enable agent output so it can speak again
await self.session.set_output_enabled(True)
await self.session.update_system_message("You may resume speaking to the caller.")
# --- Run the controller ---
async def run():
controller = AgentController(ROOM_NAME)
await controller.start()
if __name__ == "__main__":
try:
asyncio.run(run())
except KeyboardInterrupt:
<http://logger.info|logger.info>("Stopped by user")
How this maps to your requirements
• Outbound SIP call: the code calls the LiveKit API to create a SIP participant in the room (check the actual API parameters in your LiveKit SDK—method names may differ).
• Manager interrupt: manager triggers RPC enter_intervention_mode (this should be called from your frontend using LiveKit Data or RPC API). The agent reacts by muting its audio output and remaining listening-only.
• Agent transcription-only: STT remains enabled; transcripts are logged with participant identity and manager/customer tags.
• Agent resumes after manager exits intervention: manager triggers exit_intervention_mode (or you implement an automatic timeout/VAD-based end-of-speech detection).
Practical steps to complete and run this example
1. Install packages:
◦ pip install livekit python-dotenv
◦ Also install any STT/TTS/agent packages you use (e.g., livekit-agents, deepgram client, whisper, ElevenLabs SDK).
1. Fill environment variables:
◦ LIVEKIT_API_KEY, LIVEKIT_API_SECRET, LIVEKIT_URL, SIP_TRUNK_ID, CALLER_NUMBER
1. Adapt AgentSession:
◦ The sample uses a conceptual AgentSession API. Replace it with the actual classes/functions from your livekit-agents package and your chosen STT/TTS providers. Key Sources: Agents telephony integration | LiveKit Documentation | Outbound Calling Script | LiveKit Documentation | Making calls overview | LiveKit Documentation | examples/voice_agents | livekit/livekit_composite | DeepWiki | Text and transcriptions | LiveKit Documentation