https://promptfoo.dev/ logo
Connecting to huggingface
# ❓|questions
z
Hi. I am trying to run promtfoo with huggingface. I have created an API key, and created following section in providers section : providers: - id: huggingface:text-generation:mistralai/Mistral-7B-v0.1 config: temperature: 0.1 max_length: 1024 However when I run my tests i got │ [FAIL] API call error: Internal Server Error I need to know what is going wrong, what is the problem, API? Credits? But I am not able to see any more logs other than above.
i
hmm looks like
Internal Server Error
is the actual error text returned by Huggingface. Two ideas: 1. Try running with
LOG_LEVEL=debug promptfoo eval
to see if there are any more details 2. Generate an API token on Huggingface and set it as the
HF_API_TOKEN
environment variable
z
I've confirmed that the HF_API_TOKEN is set correctly; altering its value immediately triggers the expected error. Unfortunately, setting LOG_LEVEL to debug didn't yield additional useful information. The evaluation progress is stuck at 0% with no further details: yaml Copy code Eval: [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] 0% | ETA: 0s | 0/1 I also observed the following while making API requests: Huggingface API request: https://api-inference.huggingface.co/mo Huggingface API response: {"error":"Internal Server Error"} Interestingly, when I use the same query with the InferenceClient from huggingfacehub, I receive a proper response. This issue seems specific to promptfoo when used with Huggingface as a provider. Has anyone successfully integrated promptfoo with Huggingface, and if so, how did you resolve similar issues?
i
I've used Huggingface Mistral with the following config: Note that this example uses the instruct version not the base version that you're trying, but this eval also runs for me if I use the exact ID you specified. Are you able to share a full config that repros the error? I'll try it on my end. Also,
0.27.0
includes more debug info for the Huggingface provider, so updating may also offer some insight.
8 Views