Hi! I can definitely help optimize this. Generally, latency starts to noticeably increase once prompts go beyond 2,500–3,000 tokens, though it also depends on context complexity and function calls.
For good performance, keeping your system prompt around 1,500–2,500 tokens is usually ideal, and offloading static info (like FAQs or company data) to external storage or retrieval calls can help keep latency low as your system grows..
If you want, I can review your current prompt and suggest a structure to maintain speed even as it expands.
@neerajjoshi_