https://promptfoo.dev/ logo
Possible to have a shared system prompt for multi-...
# ❓|questions
f
I'm reading thought the guides but I'm still not sure if this is possible, I have a single system prompt that I'd like to test with multiple scenarios, for example:
Copy code
[
  {
    "role": "system",
    "content": "{{system_prompt}}"
  },
  {
    "role": "user",
    "content": "{{ text }}"
  },
  {
    "role": "function",
    "name": "get_current_weather",
    "content": "102 degrees Fahrenheit"
  }
]
I'd like to have multiple scenarios like this, but the first system prompt will always be the same:
Copy code
{
    "role": "system",
    "content": "{{system_prompt}}"
  },
Is it possible or do I have to define this
system_prompt
variable in every test and the load it from a file? Thanks!
i
Some providers allow system prompts at the model level, but I don't think this is the case for OpenAI. So that means right now you'd have to include it in every test case. You could potentially reduce repetition with a YAML reference:
if there were a
system_prompt
config property at the provider level, would that be useful? Or would the
{{system_prompt}}
vary at all between test cases?
f
For now it will always be the same but we have plans to add some variable data so that might change soon. For now I think this example is what I need: https://github.com/promptfoo/promptfoo/blob/main/examples/openai-chat-history/promptfooconfig.yaml#L4 I think it's fine to repeat the
defaultTest
section in evey test file.
i
ahh I see, sorry I misunderstood
f
no worries!