https://promptfoo.dev/ logo
Not recognising provider config
# ❓|questions
j
Overnight my code stopped working. Before I could run this command no problem, and today I get the error: API Error: Incorrect API key provided: $OPENAI_***_KEY. I am not trying to use openai. command: promptfoo eval —assertions x.yaml —model-outputs y.json —no-cache -o z.json -o z.html —config config.yaml —share where my config file contains: providers: - id: bedrock claude 3 config: temperature: 0.0 max_tokens: 350 top_p: 0.999 (formatted correctly with the correct model identifier)
m
Hi @jimmy_soup , thanks for reaching out and providing those details! To help us investigate further, could you please let me know the version of Promptfoo you're using? Additionally, are you using any model-graded metrics in your setup? It would also be helpful to know if the issue started after any recent changes to your config file or environment. If you’ve made any modifications recently or noticed any other details, feel free to share those as well.
j
i think 0.84.2, i didnt do an upgrade in between it working and failing though, unless it was automatic somehow. And yes, I fixed it by moving the provider into the assert file and having it under every llm rubric assert. So the config file stopped being picked up right - i really don't think anything changed in my environment, but it couldve