https://promptfoo.dev/ logo
Pass vars from command line
# ❓|questions
v
Is it possible to pass vars into promptfoo evals from the command line? We have a set of vars in a
test.yaml
file, but it would be nice to supplt an additional vars on the command line that get read into the provider that are not defined in the
test.yaml
file so it is configurable on the fly without editting the test definition. For example, if we have
Copy code
yaml
vars:
  a: "some var"
assert:
  ...
We can pull the vars into the provider (python) similarly to
Copy code
py
kwargs = json.loads(sys.argv[3])["vars"]
var_a = kwargs["a"] # "some var"
But could additionally supply another var such as
Copy code
bash
promptfoo eval --var b="somevalue"
where we could then
Copy code
py
kwargs = json.loads(sys.argv[3])["vars"]
var_a = kwargs["a"] # "some var"
var_b = kwargs.get("a", "default_value")
i
Hmm, not possible right now. The idea is that the var given on the command line would be available as a constant across all test cases?
v
correct!
we could write a scirpt that augments the test cases then calls promptfoo as well, but this would just simplify the workflow
i
v
man you're awesome
i
This change is included in [0.57.0](https://github.com/promptfoo/promptfoo/releases/tag/0.57.0), just published on npm