https://promptfoo.dev/ logo
Google Vertex - generationConfig doesn't seem to b...
# ❓|questions
m
Hi! I'm using promptfoo with Vertex. Works great, except I can't seem to get it to work with generationConfig params like temperature? I set it in the .yaml but then when I run promptfoo with
--verbose
, it looks like generationConfig is empty. Here's my promptfooconfig.yaml:
Copy code
description: 'vertex test'
prompts:  
  - Tell me a joke about {{input}}
  
providers:
  - id: vertex:gemini-1.5-pro-preview-0409
    config:
      generationConfig:
        temperature: 2
      safetySettings:
      - category: HARM_CATEGORY_HARASSMENT
        threshold: BLOCK_NONE
      - category: HARM_CATEGORY_DANGEROUS_CONTENT
        threshold: BLOCK_NONE
      - category: HARM_CATEGORY_HATE_SPEECH
        threshold: BLOCK_NONE
      - category: HARM_CATEGORY_SEXUALLY_EXPLICIT
        threshold: BLOCK_NONE

tests:
  - vars:
      input: dogs
  - vars:
      input: cats
I run with this command:
npx promptfoo@latest eval --no-cache --verbose
Then I see from the --verbose logs that this is what's being sent to Vertex. generationConfig is empty:
Copy code
{
  "contents": {
    "role": "user",
    "parts": {
      "text": "Tell me a joke about dogs"
    }
  },
  "generationConfig": {},
  "safetySettings": [
    {
      "category": "HARM_CATEGORY_HARASSMENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_HATE_SPEECH",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
      "threshold": "BLOCK_NONE"
    }
  ]
}
Is this a bug or am I configuring it wrong? I'm admittedly a bit of a yaml dummy.
i
It looks like the vars like
temperature
are taken from the top-level
config
object rather than nested in
generationConfig
https://github.com/promptfoo/promptfoo/blob/main/src/providers/vertex.ts#L196 This does not match the documentation 😱 Sorry about that I'll get this fixed, but basically you can work around this by just doing:
Copy code
yaml
config:
  temperature: 2
m
ahhhhh gotcha, thank you!
i
2 Views