Hello! I have a chatbot with a knowledge base of 100+ pages. But the response times are a little too slow, (10-20 seconds per answer). Is there anything i could do? The problem is that the website i use it for is essentially a big website with 200+ items listed. And its gonna keep growing too, so its not a static website either. Ive seen a solution to convert it to markdown but im not sure if thats very feasible to constantly do.

hi dash! if your knowledge bases are indexed then they are probably not the source of your delay, since vector kbs are incredibly fast. what model are you using to generate your answers?

Hey @fierce-student-26269 we were experiencing slowness this morning however things should be back to normal now! Can you check again? If the issue still persists it would be exactly as shen is saying above, using a faster LLM would be a better option for you!

At the moment im using gpt 4o, what llm would you suggest using instead?

Can you try 4o-mini?

4o unfortunately boasts a throughput of 34 tokens per second which is horrendously slow

I see, alright ill try that tomorrow!

Do drop an update when can!

Alright! thank you, it is definitely much faster using Gemini 2.5 flash! One quirk it had was that the chatbot started replying with external URLs instead of links from the website's own domain when using Gemini 2.5 Flash instead of GPT 4o. So I had to make the instructions a bit stricter on that end

that does tend to happen! models interpret instructions pretty differently from one another so there's that