elsatch
09/02/2024, 12:56 PMyaml
providers:
- id: ollama:chat:llama3.1:8b-instruct-q8_0
label: llama31_8B_Q8
config:
temperature: 0
stream: false
system: "You are an expert in foobar test generation ..."
Sample console output when verbose is enabled:
Ollama generate API response: {"model":"llama3.1:8b-instruct-q8_0","created_at":"2024-09-02T12:49:16.348475547Z","message":{"role":"assistant","content":"No"},"done":false}
{"model":"llama3.1:8b-instruct-q8_0","created_at":"2024-09-02T12:49:16.36468283Z","message":{"role":"assistant","content":" tengo"},"done":false}
{"model":"llama3.1:8b-instruct-q8_0","created_at":"2024-09-02T12:49:16.38080782Z","message":{"role":"assistant","content":" información"},"done":false}
{"model":"llama3.1:8b-instruct-q8_0","created_at":"2024-09-02T12:49:16.396949072Z","message":{"role":"assistant","content":" sobre"},"done":false}
{"model":"llama3.1:8b-instruct-q8_0","created_at":"2024-09-02T12:49:16.413265688Z","message":{"role":"assistant","content":" las"},"done":false}
Note that this behavior is also observed in for the grader LLM.elsatch
09/02/2024, 12:56 PMmichaelmichaelmichael
09/02/2024, 3:43 PMelsatch
09/02/2024, 4:49 PMelsatch
09/02/2024, 4:54 PMian
09/04/2024, 4:05 PMconfig parameter in promptfoo is treated as the ollama options parameter ☠️ That's why stream didn't workelsatch
09/05/2024, 10:40 AM