https://promptfoo.dev/ logo
Evaluating Ollama
# ❓|questions
t
Hello there, i am tring to evaluate ollama prompts but i have some issues: firstly i was getting this https://github.com/promptfoo/promptfoo/issues/160 and i followed the proposed solutioni but then i get the following
i am using ollama on wsl and promtfoo is ran from windows
i
can you run it with the
LOG_LEVEL=debug
environment variable and send the output? kinda hard for me to tell what's going on. For example:
LOG_LEVEL=debug promptfoo eval
t
will do
-firstly i ran ollama on wsl, -then i ran prompt foo outside of wsl, -i get this (image 1) -stoped ollama service in wsl -ran export OLLAMA_HOST=":11434" - started again the ollama service -ran prompt foo eval again -got this (image 2) -in ollama service i get (image 3) my promptfooconfig.yaml file is
Copy code
yaml
description: 'My first eval'

prompts:
  - 'Write a tweet about {{topic}}'

providers:
  - id: ollama:openhermes
    config:
      num_predict: 1024

tests:
  - vars:
      topic: bananas
i
ok, that looks like progress. at least it can talk with ollama, but seems like it is getting an unknown parameter
basePath
and rejecting the request
Here is a fix to the
basePath
issue. It will be in the next release: https://github.com/promptfoo/promptfoo/pull/480
t
thank you very much
has the pr entered?
i
I've just landed it
I will include it in the next published release
you can also use it if you install directly from source 🙂
t
noice, you the best
l
Hi there, I still have this issue with the latest promptfoo (0.43.1). Dose anyone know how to solve that?
and here is my config
Copy code
description: 'My first eval'

prompts:
  # - file://prompt1.json
  - "Answer the question: {{question}}"

providers:
  - ollama:chat:openchat:7b

tests:
  - vars:
      question: What's your name?
    assert:
      - type: equals
        value: Hello, How may I assist you today?
  
  - vars:
      question: Hello
i
in case anyone else comes across this, handled in https://github.com/promptfoo/promptfoo/pull/506
2 Views