@millions-hair-24956 What is the best way to get the full number from the user, when the user dials in the voicepipelineagent through sip trunking?

To obtain the full number dialed by the user when using the

voice-pipeline-agent

through SIP trunking, you can access the SIP participant's attributes. Specifically, the

sip.trunkPhoneNumber

attribute provides the number that was dialed by the user. This attribute is part of the SIP participant's attributes and can be accessed in your agent code. For example, you can use the following code snippet to retrieve the dialed number:

participant = await ctx.wait_for_participant()
dialed_number = participant.attributes.get("sip.trunkPhoneNumber")

Would you like more details on how to implement this in your application? Sources: - https://livekit-users.slack.com/archives/C07FY8WHGPM/p1733245440.370219 - https://livekit-users.slack.com/archives/C07FRP66XGE/p1733245434.910139 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@millions-hair-24956 I don’t mean the number that the user dialed (which is the sip trunk number), I mean the user’s own number, how can I best get that?

To obtain the user's own number when they dial into the

voice-pipeline-agent

through SIP trunking, you should access the

sip.phoneNumber

attribute of the SIP participant. This attribute represents the caller's phone number. Here's a code snippet to retrieve the caller's number:

participant = await ctx.wait_for_participant()
caller_number = participant.attributes.get("sip.phoneNumber")

Would you like more details on how to implement this in your application? Sources: - https://livekit-users.slack.com/archives/C07FY8WHGPM/p1727971688.727619 - https://docs.livekit.io/sip/sip-participant/ 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@millions-hair-24956 I have the following code:

class AgentCallableFunctions(llm.FunctionContext):
    def __init__(self, phone_number: str, ctx: JobContext, participant: rtc.RemoteParticipant, user_phone_number: str):
        super().__init__()
        self.phone_number = phone_number
        self.ctx = ctx
        self.participant = participant
        self.user_phone_number = user_phone_number

    @llm.ai_callable(description="Called when getting one or more location information by user's phone number")
    async def get_location_information(
        self,
        phone: Annotated[
            str, llm.TypeInfo(description="The locations of the user based on their phone number")
        ]
    ):
        location_result = await self.location_service.get_location(self.user_phone_number)
        return location_result

It might happen that sometimes the user_phone_number is not set or undefined, in those cases the LLM should ask the user for a phone number and use the ‘phone’ parameter and pass that through the get_location function. How can I best address this in LiveKit?

It seems likely that you can handle the scenario where

user_phone_number

is not set by checking its value within the

get_location_information

function. If it's not set, you can prompt the LLM to ask the user for their phone number and use the

phone

parameter provided by the user. Here's a possible approach: 1. Check if

self.user_phone_number

is set. 2. If not, use the

phone

parameter to get the location information. Here's how you might implement this:

@llm.ai_callable(description="Called when getting one or more location information by user's phone number")
async def get_location_information(
    self,
    phone: Annotated[
        str, llm.TypeInfo(description="The locations of the user based on their phone number")
    ]
):
    # Use the provided phone number if user_phone_number is not set
    phone_number_to_use = self.user_phone_number or phone
    location_result = await self.location_service.get_location(phone_number_to_use)
    return location_result

Would you like more details on how to implement this in your application? Sources: - https://livekit-users.slack.com/archives/C025KM0S1CK/p1730739382.463779 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@millions-hair-24956 how does the above make it clear to use self.user_phone_number if that is set? Shouldn’t that be made clearer in the description?

I'm not able to find any data sources to answer your question. You might want to clarify how the logic prioritizes

self.user_phone_number

over the

phone

parameter in the function's description. If this doesn't help, please reach out to the Livekit support team for further assistance. 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@millions-hair-24956 let me rephrase. I get the participant phone number through sip.phoneNumber (see below).

user_phone_number = participant.attributes.get("sip.phoneNumber")
fnc_ctx = AgentCallableFunctions(phone_number, ctx, participant, user_phone_number)

The user_phone_number is in turn added to the funct context class, as seen below.

class AgentCallableFunctions(llm.FunctionContext):
    def __init__(self, phone_number: str, ctx: JobContext, participant: rtc.RemoteParticipant, user_phone_number: str):
        super().__init__()
        self.phone_number = phone_number
        self.ctx = ctx
        self.participant = participant
        self.user_phone_number = user_phone_number

    @llm.ai_callable(description="Called when getting one or more location information by user's phone number")
    async def get_location_information(
        self,
        phone: Annotated[
            str, llm.TypeInfo(description="The locations of the user based on their phone number")
        ]
    ):
        location_result = await self.location_service.get_location(self.user_phone_number)
        return location_result

How can I make sure that the voice agent uses the user_phone_number if it has a value set as part of the AgentCallableFunction context and asks for it if it’s not set? Do I have to do this through prompting or through another approach?

It might be possible to handle this scenario by checking if

self.user_phone_number

is set within the

get_location_information

method. If it's not set, you could prompt the user for their phone number using the LLM. Here's a potential approach: 1. Check if

self.user_phone_number

is set. 2. If not, use the

phone

parameter to prompt the user for their phone number. Here's how you might implement this:

@llm.ai_callable(description="Called when getting one or more location information by user's phone number")
async def get_location_information(
    self,
    phone: Annotated[
        str, llm.TypeInfo(description="The locations of the user based on their phone number")
    ]
):
    # Check if user_phone_number is set
    if not self.user_phone_number:
        # Prompt the user for their phone number
        self.user_phone_number = phone

    # Use the user_phone_number to get location information
    location_result = await self.location_service.get_location(self.user_phone_number)
    return location_result

This approach ensures that the

user_phone_number

is used if available, and prompts the user for their phone number if it's not set. Would you like more details on how to implement this in your application? Sources: - https://docs.livekit.io/sip/examples/sip-participant-attributes/ - https://docs.livekit.io/sip/sip-participant/ Note: I wasn’t able to find highly relevant data sources, but above are a few potentially relevant links. 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@millions-hair-24956 No I want the LLM/VoicePipelineAgent to see and know if the user_phone_number is set or not, and if not ask for it specifically from the user. Is that possible at all?

I'm not able to find any data sources to answer your question. You might want to specify how the LLM should handle cases where

user_phone_number

is not set, possibly by using a prompt to request the phone number from the user. If this doesn't help, please reach out to the Livekit support team for further assistance. 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@millions-hair-24956 can the VoicePipelineAgent through function calling read variables and parameters in the code>

Yes, the

VoicePipelineAgent

can interact with variables and parameters in your code through function calling. You can define functions within your agent's function context that the LLM can call. These functions can access and manipulate variables and parameters as needed. To achieve this, you should: 1. Define a class that inherits from

llm.FunctionContext

. 2. Decorate your functions with

@llm.ai_callable()

to make them available for the LLM to call. 3. Within these functions, you can access and modify variables and parameters as required. Here's a simple example:

class AgentCallableFunctions(llm.FunctionContext):
    def __init__(self, some_variable):
        super().__init__()
        self.some_variable = some_variable

    @llm.ai_callable(description="Example function to read and modify a variable")
    async def modify_variable(self, new_value: str):
        self.some_variable = new_value
        return f"Variable updated to: {self.some_variable}"

Would you like more details on how to implement this in your application? Sources: - https://livekit-users.slack.com/archives/C07FY8WHGPM/p1728124961.554009 - https://docs.livekit.io/python/livekit/agents/pipeline/index.html 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@millions-hair-24956 in the code below:

user_phone_number = participant.attributes.get("sip.phoneNumber")

How can I pass the user_phone_number to the function context for the voicepipelineagent?

It seems likely that you can pass the

user_phone_number

to the function context for the

VoicePipelineAgent

by including it as a parameter when you instantiate your function context class. Here's how you might do it: 1. Define your function context class to accept

user_phone_number

as an argument. 2. Pass

user_phone_number

when creating an instance of the function context. Here's an example:

class AgentCallableFunctions(llm.FunctionContext):
    def __init__(self, phone_number: str, ctx: JobContext, participant: rtc.RemoteParticipant, user_phone_number: str):
        super().__init__()
        self.phone_number = phone_number
        self.ctx = ctx
        self.participant = participant
        self.user_phone_number = user_phone_number

# Retrieve the user phone number from participant attributes
user_phone_number = participant.attributes.get("sip.phoneNumber")

# Create an instance of the function context with the user phone number
fnc_ctx = AgentCallableFunctions(phone_number, ctx, participant, user_phone_number)

# Pass the function context to the VoicePipelineAgent
agent = VoicePipelineAgent(
    vad=vad_instance,
    stt=stt_instance,
    llm=llm_instance,
    tts=tts_instance,
    chat_ctx=chat_context,
    fnc_ctx=fnc_ctx
)

This approach ensures that the

user_phone_number

is available within the function context for use by the

VoicePipelineAgent

. Would you like more details on how to implement this in your application? Sources: - https://docs.livekit.io/sip/examples/sip-participant-attributes/ - https://docs.livekit.io/agents/voice-agent/function-calling/ 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.