https://promptfoo.dev/ logo
http provider zod error when using an environment ...
# questions
w
Hi there! I was reading through the documentation, and it mentions that env vars are accessible inside the http provider However, when I try to use an env var as a url, it throws a zod error
Copy code
errors: [
    {
      code: 'invalid_type',
      expected: 'string',
      received: 'object',
      path: [ 'url' ],
      message: 'Expected string, received object'
    }
  ]
Here's my provider:
Copy code
yaml
providers:
  - id: https
    label: Base model
    config:
      url: {{ env.PROVIDER_URL }}
      maxRetries: 3
      method: POST
      headers:
        'Content-Type': 'application/json'
        'Authorization': 'Bearer {{ env.GOOGLE_ID_TOKEN }}'
      body:
        agent:
          query: '{{query}}'
      transformResponse: |
        {
          output: json.finalMessageContent,
          tokenUsage: {
            total: json.tokenUsage?.totalTokens || 0,
            prompt: json.tokenUsage?.inputTokens || 0,
            completion: json.tokenUsage?.outputTokens || 0,
            cached: json.tokenUsage?.cacheReadTokens || 0,
            numRequests: json.tokenUsage?.llmCalls || 0
          },
          cost: json.cost
        }
w
Does
'{{ env.PROVIDER_URL }}'
work?
w
Yep! It fixes that error, thank you!
Though I've got another error with the http provider it seems... Should I open another forum thread for it or continue it here?
I'm getting an "Cost assertion does not support providers that do not return cost", which seems to contradict the docs, as they say the http provider supports costs?
w
You'll need to deserialize the response returned by the HTTP provider
w
I think I'm already doing that in the above example, or am I doing that wrong?
I return a .cost property on the root of the response json, a number
w
Ah, I'm sorry, totally missed that.
Let me take a closer look and I'll get back to you.
w
Oh sure thing thanks! I do appreciate that
Honestly loving promptfoo. Trying to see what I can get set up with github actions and it too. Good fun
Ahh... I fixed it I think It might have been the ? and the || 0 stuff. I removed that, and also fixed a seperate thing that may have been causing network errors