​ To learn more, see the sources I considered: [[1] Manually test your agent](https://docs.retellai.com/test/llm-playground) [[2] Testing Overview](https://docs.retellai.com/test/test-overview) [[3] Discord message from @taafy on 2025-08-24](https://discord.com/channels/1186424376300679209/1186462578310910112/1409051754426798151) ​

### Answer The LLM Playground (manual chat testing) and phone/web call testing serve different purposes and may produce different results. ## Key Differences LLM Playground tests the LLM responses directly without the speech-to-text (STT) layer, making it ideal for rapid prototyping, debugging prompts, and testing function calling logic. [(1) ]() [(2) ]() Phone/Web Call Testing includes the full voice pipeline: speech-to-text transcription, turn-taking, background noise processing, and interruption handling. [(2) ]() ## Are Responses Consistent? Not necessarily. The main difference between a voice call and a text call is that voice calls use the STT layer. [(3) ]() Community feedback indicates that agent behavior can differ between chat and voice: - One user reported that "the consistency and accuracy of the agent in voice conversations is significantly worse when compared to chat conversations" - RAG/knowledge base retrieval that works consistently in manual chat may behave differently in voice calls ## Why the Difference? Transcription quality affects voice call performance. Factors to check include: - Transcription accuracy settings - Noise removal configuration - What chunks are being pulled from knowledge bases ## Recommended Approach Use the testing methods in phases: [(2) ]() 1. LLM Playground for development and prompt iteration 2. LLM Simulation Testing for batch quality assurance 3. Phone/Web Call Testing for final production validation with actual voice interactions If you notice discrepancies, compare the transcription logs in call history to see what's bei

ng captured versus what you're typing in chat. [(3) ]()

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.