Hey team! I’m working on a voice agent that follow...
# agents
a
Hey team! I’m working on a voice agent that follows a script in the same way a human would. It should stick to the structure, say some lines exactly when needed, adapt when the user goes off track, and guide the conversation back to the script. We also need this to be modular since we handle many different scripts. Have you seen approaches or patterns for this type of behavior, or is this usually solved with custom logic?
s
if you want it to be very scripted, I've used tool calls + session.say since prompting is not deterministic
a
That's basically my approach but it struggle to go off script, so it seems kind of robotic sometimes
p
@ancient-judge-59849 break it up into multiple agents, create an evaluation pipeline to test it doesn’t go off script and iterate on the prompt (instructions and function tool schema) when it goes off script. Ideally, star with very basic and simple instructions and make incremental changes (change, test, repeat). There is a feature request in GitHub to improve generate_reply to make it more consistent. Have you already built something and are facing issues?
a
Thank you Mario from Mario Bros! So I started with the flow divided in 4, each part being an agent. Then, lots of clients started wanting to have exact messages in certain parts and the flush in between agents make a lot of latency, so i switched to just one agent that used a function tool to go into the next step of the conversation, similar to what marc said. I dynamically change the enum values of the function so it knows what are the possible next steps and use predefined Tasks in key points of data gathering but what im struggling right now is that the agent might not be able to go off rail without going crazy. For example, if i go to a point of the script by accident, the agent does not know how to go back to another part or it sometime struggles with simple questions about the conversation. It seem like a regular use case so i just wanted to know if the team has figured out a standard way to do this or if there is already some builtin functionality for something like that