Is there a way to add some filer words like "umm", "aah" to reduced perceived latency?

You might want to take a look at https://docs.livekit.io/agents/logic-structure/external-data/#-thinking-sounds. That doc specifically talks about external data access, when there is expected high latency, but you can also access the

thinking

event through https://docs.livekit.io/reference/other/events/#agentstate to perform some custom logic on state change.

There is also this example https://github.com/livekit/agents/blob/main/examples/voice_agents/fast-preresponse.py