how to skip llm tokenization in agentsession to im...
# ask-ai
a
how to skip llm tokenization in agentsession to improve ttft latency
✅ 1
t
Thanks for reaching out, Harshita. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’m responding to your question about skipping LLM tokenization in an AgentSession to improve TTFT latency. I don’t have details about your AgentSession implementation or which LLM/SDK you’re using. A few quick clarifying questions so I can give a precise recommendation: • Which LLM provider and SDK are you using (OpenAI, Anthropic, local model, etc.)? • Where does tokenization occur today — client-side before sending the request, or server-side inside your AgentSession? • Are you using streaming output from the model (SSE/grpc/WS)? • Do you control the AgentSession code (can you modify the request flow), or is it managed by a library/platform you cannot change? • Is your priority strictly lowest TTFT, or do you also need exact token counts/usage reporting? With those answers I’ll propose whether skipping tokenization is possible and concrete alternatives (e.g., client-side pre-tokenization, sending raw bytes if supported, using model APIs that accept pre-tokenized input, enabling streaming early, or caching/warmer requests). , with interactive elements