I am fetching composio tools using composio("user_id").session.tools(), and passing it to vercel AI SDK

async function streamChatResponse(
  validatedMessages: ChatUIMessage[],
  model: AiModel = DEFAULT_AI_MODEL,
  tools: ToolSet,
): Promise<ChatStreamResult> {
  try {
    const result: ChatStreamResult = streamText({
      model: getModelInstance(model),
      messages: await convertToModelMessages(validatedMessages),
      tools,
      stopWhen: stepCountIs(10),
      onFinish: (event) => {
        logger.info({ usage: event.usage }, `Stream finished`);
      },
    });
    return result;
  } catch (error) {
    logger.error({ error, model }, 'Failed to stream chat response');
    throw new Error('Failed to stream AI response', { cause: error });
  }
}

When i pass in

tools

the system tokens shoots up from 200 to 7000. Any way to cache them? I cant seem to find anything

Hey @assasin, you can use

COMPOSIO_SEARCH_TOOLS

to fetch tools based on use case. Doc for ref: https://docs.composio.dev/docs/tools-and-toolkits#meta-tools. Also, did you get a chance to checkout our chatbot on docs or Dashboard? It should really help you out. Thanks!

I saw the chatbot docs, it mentioned using the above method. Do you mean that I should use the

COMPOSIO_SEARCH_TOOLS

tool instead of session.tools() to pass the tools?

Hey @assasin, Your current workflow is intended. However, if you’re looking for alternative ways to pass tools to the LLM, you can use

tools.get

to obtain static tools and pass them to the LLM. This approach works well for deterministic scenarios and with a smaller number of tools. Another alternative is to manually execute

COMPOSIO_SEARCH_TOOLS

based on your specific use case to generate a list of relevant tools and pass it to the LLM. Also, did you get a chance to checkout the chatbot I've mentioned earlier, it can help you with quicker responses, happy to help if the issue remains unresolved.