vitmont
04/30/2024, 9:27 PMtest.yaml file, but it would be nice to supplt an additional vars on the command line that get read into the provider that are not defined in the test.yaml file so it is configurable on the fly without editting the test definition.
For example, if we have
yaml
vars:
a: "some var"
assert:
...
We can pull the vars into the provider (python) similarly to
py
kwargs = json.loads(sys.argv[3])["vars"]
var_a = kwargs["a"] # "some var"
But could additionally supply another var such as
bash
promptfoo eval --var b="somevalue"
where we could then
py
kwargs = json.loads(sys.argv[3])["vars"]
var_a = kwargs["a"] # "some var"
var_b = kwargs.get("a", "default_value")ian
04/30/2024, 9:49 PMvitmont
04/30/2024, 10:40 PMvitmont
04/30/2024, 10:40 PMian
05/01/2024, 6:28 AMian
05/01/2024, 6:47 PM