Seen this happen before, its usually an environment difference between the UI and batch simulation rather than a straight up bug. Check your logs for any errors on the knowledge base API calls during simulation. If you want testing that doesnt rely on Retell's simulation at all give cekura.ai a shot, its built to run automated conversation tests in a controlled environment so retrieval works consistently every time.