https://www.boundaryml.com logo
🟡 Is there a reliable way to detect context limit...
# questions
a
Is there a reliable way to detect specifically errors for going over the context limit?
b
Hmm - this is different than BamlClientFinishReasonError with finish_reason max_tokens, I see
It doesn't seem like API providers have consistent behavior here: - Anthropic has model_context_window_exceeded https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons#model-context-window-exceeded - OpenAI has
finish_reason length
on /chat/completions, but I don't see a context limit exceeded option - OpenAI has
incomplete_details max_output_tokens
on the Responses API - we may need to do work to support this
a
yeah.. i ended up just checking if num output tokens == max output tokens.. would be cool if baml could expose this in a cleaner way
b
heard! will keep it in mind 🙂
2 Views