American Boy
06/03/2024, 4:51 PMgte.py to take care of the similarity scores. Again, I attach the pic to this post.
In the promptfooconfig.yaml I wrote down this:
defaultTest:
options:
provider:
embedding:
id: 'python:gte.py'
Which leads to it saying its not appropriate embedding provider and reverting to the OpenAI API:
Provider python:gte.py is not a valid embedding provider for 'similarity check', falling back to default
Provider python:gte.py is not a valid embedding provider for 'similarity check', falling back to default
...
[FAIL] Error: OpenAI API key must be set for similarity comparison
Error: OpenAI API key must be set for similarity comparison
at OpenAiEmbeddingProvider.callEmbeddingApi (/opt/homebrew/lib/node_modules/promptfoo/dist/src/providers/openai.js:88:19)
I guess my setup is incorrect, but I am not sure where, as I tried to mimic the documentation. Any help would be appreciated.
I used for reference:
https://promptfoo.dev/docs/configuration/expected-outputs/similar/
https://promptfoo.dev/docs/providers/python
https://cdn.discordapp.com/attachments/1247231122010542172/1247231122367316070/pic.png?ex=665f4600&is=665df480&hm=8a0c82ee409b76573a7c553b57346d2296f23c8b491fcc2793f3eef0ef659f33&
https://cdn.discordapp.com/attachments/1247231122010542172/1247231122656591892/pic2.png?ex=665f4600&is=665df480&hm=e8c6d9b5d57caee2f7e07809ad054db42d12924ed671e5593622339981b53990&ian
06/04/2024, 6:09 AMian
06/07/2024, 6:42 AM