Thanks for bringing this up! The issue happens because the Gemma model's chat template expects user and assistant messages to alternate strictly. When there are consecutive user messages, it can lead to server errors. A helpful way to fix this is by using the

toChatCtx

function in the Google provider format. This function smartly groups consecutive user messages into one message with multiple text parts, keeping the alternating pattern the model needs. So, for example, if you have two user messages in a row, they’ll be combined into a single message before being sent to the model. Also, to keep things running smoothly, the system limits combined messages to 1kB and truncates the chat context to a maximum of 40 messages. This helps avoid hitting model limits while still maintaining the flow of your conversation. By trying out this approach, you should find that the server errors caused by consecutive user messages are resolved. If you need any more help, feel free to ask—I’m here to support you! Sources: tests/test-chat-template.cpp | agents/src/llm/provider_format/google.test.ts | livekit-plugins/livekit-plugins-google/livekit/plugins/google/llm.py