https://promptfoo.dev/ logo
HTTP/HTTPS API streaming issue
# ❓|questions
f
Hello ! It seems like streaming is used even when I'm setting stream to false when I'm calling Ollama directly from a URL (I'd like to test a direct call to the localhost, as I don't intend to use Ollama but simply use it for initial testing). Here is my config :
Copy code
providers:
  - id: 'http://127.0.0.1:11434/api/generate'
    config:
      method: 'POST'
      headers:
        'Content-Type': 'application/json'
      body:
        model: 'llama3'
        prompt: '{{prompt}}'
      responseParser: 'json.output'
      stream: false
I also tried this :
Copy code
providers:
  - id: 'http://127.0.0.1:11434/api/generate'
    config:
      method: 'POST'
      headers:
        'Content-Type': 'application/json'
      body:
        model: 'llama3'
        prompt: '{{prompt}}'
        stream: false
      responseParser: 'json.output'
It looks like the problem has already been solved for the call to ollama directly from ollama:llama3 (here : https://discord.com/channels/1153767083381903389/1280149322968596602) but I have the impression that it hasn't been solved for a call directly from the URL (maybe I'm making the call wrong too!). Thanks a lot for your help !
i
the second one should work - i believe ollama accepts a body parameter called
stream
. maybe you can try with
LOG_LEVEL=debug
and see what's being sent directly to ollama (or maybe there is some debugging facility on the ollama side)
f
I have this error when I set the stream parameter in the body : It seems that the responseparser could be the problem but no idea what to change https://cdn.discordapp.com/attachments/1285967658294050827/1286240411555201055/image.png?ex=66ed303b&is=66ebdebb&hm=0de234d83be86b5aecd2dbfb4e72f69b387405d07c729bbcda3966a4d5809560&