I have a agent session integrated with telephony. ...
# ask-ai
l
I have a agent session integrated with telephony. using livekit python framework. now i want to handle transfer to human. where when transfering my agent didn't leave the room. it should listen to the room participants and make summary of call. how to do this
t
Thanks for reaching out, Rajadurai. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I understand you want the agent (using LiveKit Python + telephony) to transfer the caller to a human but stay in the room, continue listening to the conversation, and produce a summary. Below is a concise, practical plan and example code patterns to implement that behavior. What to do (high level) • Do not close or remove the agent participant from the room when you initiate transfer. Let the agent remain connected as a participant so it can keep receiving audio/transcripts and room events. • Use a cold SIP transfer (SIP REFER) to move the PSTN caller to a human. The transfer will redirect the caller while your agent stays in-room. • Continue collecting transcripts and/or recorded audio from the room after transfer so you can build a final call summary. • When you want the summary, aggregate the collected transcripts/audio and send them to your LLM/summary service. Key implementation steps (with example patterns) 1) Identify and transfer the SIP participant (cold transfer) • Find the SIP participant in the room and call the SIP transfer API. This issues a SIP REFER to your trunk provider and transfers the caller to the target phone number. The agent does not leave the room. Example pattern:
Copy code
# find SIP participant
sip_participant = None
for p in job_ctx.room.remote_participants.values():
    if p.kind == rtc.ParticipantKind.PARTICIPANT_KIND_SIP:
        sip_participant = p
        break

if not sip_participant:
    return "No SIP caller found to transfer."

# request cold transfer (tel: format)
await job_ctx.api.sip.transfer_sip_participant(
    api.TransferSIPParticipantRequest(
        room_name=job_ctx.room.name,
        participant_identity=sip_participant.identity,
        transfer_to=f"tel:{human_phone}",
    )
)
# agent remains in the room after this call
Notes: • Ensure your SIP trunk/provider supports SIP REFER (some trunks require an explicit setting like “Enable PSTN Transfer”). 2) Keep the agent connected and listening • Do not call session close/shutdown after transfer. Keep background listeners for events active. • Subscribe to transcript events and track events so the agent keeps receiving speech-to-text and notifications when the human joins. Example event handlers:
Copy code
@session.on("user_input_transcribed")
async def on_transcript(transcript):
    if transcript.is_final:
        call_history.append(transcript.transcript)

@session.on("track_published")
def on_track_published(track, participant):
    if participant.kind == rtc.ParticipantKind.PARTICIPANT_KIND_STANDARD:
        # human has joined — you can start special monitoring if needed
        print(f"Human agent {participant.identity} joined")
3) Collect conversation data for the summary • Maintain a running buffer (call_history) of final transcripts and optionally store audio samples or recordings if you need higher quality for the summary. • Include timestamps and participant identities if you want more structured summaries. 4) Generate the call summary with an LLM • When appropriate (e.g., after some inactivity or when the human signals handoff complete), aggregate the transcripts and request a summary from your LLM. Example pattern:
Copy code
async def generate_call_summary():
    full_transcript = "\n".join(call_history)
    response = await ctx.llm.chat(
        chat_ctx=[{"role": "user", "content": f"Summarize this call:\n{full_transcript}"}]
    )
    return response.choices[0].message.content
5) Optional: agent publishes audio or interjects • If you want the agent to speak (e.g., announce that it will summarize), keep it publishing an audio track and use TTS to interject into the room. Checklist and gotchas • Do not call
session.close()
or
ctx.shutdown()
if you want the agent to remain and listen. • Confirm your SIP trunk/provider supports SIP REFER / PSTN transfer features. • If you rely on transcripts, ensure your session is configured to receive
user_input_transcribed
events or the appropriate STT stream. • If you need verbatim accuracy, capture and store audio recordings in addition to transcripts. If you want, I can: • Show a full minimal example file wiring up the session lifecycle, transfer call, transcript collection, and summary generation. • Help design how to detect "transfer complete" or when to trigger summary generation (e.g., inactivity timeout, explicit human signal, or a webhook). Which would you like next: the full example code, or help deciding the best trigger to generate the summary? Sources: Agents (Python) | livekit/livekit_composite | DeepWiki | Agents telephony integration | LiveKit Documentation | Agent session | LiveKit Documentation | Company directory phone assistant | LiveKit Documentation