Hi, is there a way to indicate to promptfoo that the chat has a character limit so it limits how big are the prompts that it generates? Can I do that through purpose somehow or through a config setting?

Out of curiosity, what is your limit? Are you doing single or multi-turn conversations? Which strategies are you using? In the purpose you can specify short responses. if that is not working, we can build something for you.

Our API limits user inputs to 1000 characters per message, so my promptfoo runs are getting blocked by that and it's not representative of a good test suite. Ideally, I can indicate to promptfoo that the limit exists. I am using a mix of single and multi-turn strategies - crescendo, jailbreak, jailbreak composite, base64 and other encodings

As a workaround, I recommend trying jailbreak:meta and jailbreak:hydra. They are the most effective strategies. I also recommend putting in the purpose to keep prompts concise or short

+1 to telling it to keep prompts concise in the application details