Having a hard time getting optimize to do anything more than hang. I copied over the example from the docs: https://docs.boundaryml.com/guide/baml-advanced/prompt-optimization I set my environment so that

echo $ANTHROPIC_API_KEY

shows my key. I tried executing a test in the vs code playground, but it said

try enabling the proxy (See API Keys button)

which I fixed by copying my API key into the playgound keys (should that be necessary -- why can't it read

ANTHROPIC_API_KEY

from the env? But even after the tests pass in the playground, running

baml-cli optimize --beta --trials 3 --max-evals 10 --function ExtractSubject

shows the TUI but just hangs. What else should I check?

Hmm @greg (BAML) might be able to help, do you use python or typescript?

python

Thanks for all the details! Could you send a screenshot of the hanging state? That would help give me more context.

Also, running the TUI with

--no-ui

will give you some more debug info.

I was wondering if it was something like that 🙂

Another user made a great example that's a bit harder for the LLM to solve, thus optimization requires several iterations to get to 100%. I'll update the docs with that soon, too!

Thank you! Since you mentioned

--no-ui

, doing that way leads it to hang in a different way, just waiting for user input: https://cdn.discordapp.com/attachments/1452029311870828685/1452031216994750514/image.png?ex=694854f3&is=69470373&hm=ceac782f0783bbdefff60b9a5b559cfa5fea5661dcff80b940c6a70fb66e58b1&

Yeah, that's a great point, thanks! The order doesn't matter to the optimizer, except for the fact that, as you mentioned, failure is fast. So an early failure won't be able to give the optimizer information about failures that would have happened if execution had gone farther. I'll describe this behavior in the docs.