What's wrong in this code block:

async def llm_node(
        self,
        chat_ctx: llm.ChatContext,
        tools: list[FunctionTool],
        model_settings: ModelSettings,
    ) -> AsyncIterable[llm.ChatChunk | str]:
        """Override llm_node to truncate chat history before sending to LLM"""
        
        print(f"[LLM NODE] Inside llm_node")
        
        # Truncate chat history to prevent context overflow
        MAX_MESSAGES = 8
        
        if len(chat_ctx.messages) > MAX_MESSAGES:
            # Simply keep the last 8 messages
            original_count = len(chat_ctx.messages)
            chat_ctx.messages = chat_ctx.messages[-MAX_MESSAGES:]
            print(f"[DEBUG] Truncated chat history from {original_count} to {len(chat_ctx.messages)} messages")
        
        print(f"[LLM NODE] Inside llm_node after truncation logic")
        # Call the default LLM node implementation - pass self as first argument to static method
        async for chunk in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
            yield chunk

Thanks for reaching out, Anthony. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

From my logs i get the following:

[LLM NODE] Inside llm_node
  ERROR:livekit.agents:Error in _llm_inference_task

I'm sorry you're encountering the 'Error in _llm_inference_task'. To help you better, could you please share a bit more information? For example, the full stack trace or any other log messages you see before or after this error would be really helpful. That way, we can work together to figure out what's going on.

I'm wondering the following, should I use

chat_ctx

.items instead of

chat_ctx.messages

Hi there! I wanted to gently point out that using

chat_ctx.items

instead of

chat_ctx.messages

is the way to go when truncating chat history. The

items

attribute is the correct one in the current LiveKit version, and it’s designed to handle richer conversation structures, including function calls and outputs. If you’ve been using

chat_ctx.messages

, that’s totally understandable—it was part of older versions. Updating your code to use

items

will help you stay in sync with the latest API design and make things run more smoothly. If you need any help with this update, I’m here for you! Sources: Agents 0.x migration guide | LiveKit Docs | livekit-agents/livekit/agents/llm/remote_chat_context.py