davidl_96272
05/01/2024, 11:10 PMian
05/01/2024, 11:20 PMdavidl_96272
05/01/2024, 11:26 PMian
05/01/2024, 11:45 PM-j or --max-concurrency param will do that. default is 4 at a time, but if your instance can handle more then go for it!
promptfoo eval -j 4davidl_96272
05/01/2024, 11:49 PMdavidl_96272
05/02/2024, 12:10 AMdescription: romantic_prompts_100concurrency
prompts:
- file://romantic_prompts_1_24/*.txt
providers:
- id: python:inference.py
config:
llm_type: mistralai/Mixtral-8x22B-Instruct-v0.1
tests:
- tests/romantic_prompts_100concurrency/dataset.csv
max_concurrency: 100
Not noticing any latency improvement unfortunately