Hi, I have a bunch of prerecorded conversations b...
# agents
s
Hi, I have a bunch of prerecorded conversations between Human agents and customers, I want to test my AI agent using the recordings by possibly streaming just the customer queries and comapring results between the human agent and AI agent responses. Are their any recommendations , pipelines to do this kind of testomg at scale? Thanks!
g
Livekti Agents Testing. Just check the docs. Otherwise there are a few platforms out there who make it easier
g
You can transcribe your voice call, using tools like deepgram. It will give also give transcriptions divided by speaker, then use the transcripts of speaker to test the AI Agent.
💯 1
s
@green-judge-63955 I can see that the docs link to 3p platforms when looking for Voice agents testing, the frameowkr mentioned for livekit is text agents only I understand, can you share inputs on levergaing existing recordings the test out agents by simulating the caller inputs
r
Can you tell me more about your testing? How will you know when there is success versus failure?
s
@refined-appointment-81829 As we essentially have these pre-recorded audio calls I was wondering if we can use the "Human agent" piece to evaluate how the AI agent is performing, maybe a semantic similarity match
r
@crooked-tiger-31253 has a nice solution for testing called https://getbluejay.ai/ As @green-judge-63955 mentions there is agent testing in LiveKit. As proof of concept I also built this basic test tool that may make a nice reference if you are trying to build your own: https://github.com/yepher/livekit_info/tree/main/simple_test_client
🙏 1
c
Hey @some-lamp-76309 I’m Faraz, co-founder @ Bluejay! happy to help here, we have a feature on our platform that allows you to replay customer responses and test your agent’s performance against them! I’ve DM’d you with more info!
🙌 1