https://promptfoo.dev/ logo
Dynamic OpenAI Functions from File
# ❓|questions
x
Hello! My team has our functions stored in a yaml file that matches the specification needed for promptfoo's provider config for OpenAI (name, description, and params). Instead of having to copy over and duplicate my functions (we have multiple versions, regular changes to definitions, etc) I would rather be able to reference the yaml file directly. Is this possible? Or something that might be on the horizon? I can convert the yaml to JSON and use that instead if it would make things simpler. Any help here would be greatly appreciated, thanks!
i
Hi @xpertpupil., the best way to do this right now is load providers from file.
Copy code
yaml
providers:
  - file://./path/to/provider_with_function.yaml
And in the yaml file:
Copy code
yaml
id: openai:chat:gpt-3.5-turbo-0613
config:
  functions:
    - name: get_current_weather
      description: Get the current weather in a given location
      parameters:
        type: object
        properties:
          location:
            type: string
            description: The city and state, e.g. San Francisco, CA
          unit:
            type: string
            enum:
              - celsius
              - fahrenheit
            description: The unit in which to return the temperature
        required:
          - location
This leads to a little repetition with the provider id, but overall mostly what you want I think. See also for a json example
x
Thanks Ian, that makes sense. It would require us to store the
id
separate from the prompt-foo config, and move it into our internal code, which is a bit of a smell, but overall better than the total duplication of the function definitions I think
i
You're not the first person to request this - so I'll create a task for making it so that you can import provider config values using
file://
lmk if you run into any other issues
x
I appreciate it! So far so good (other than some user error on my part hehe). We are integrating into our buildkite pipeline, etc.
If you think it might be a good issue for someone who hasn't contirbuted before, I'd be happy to take a stab at it as well.
d
I had this same issue, I'm using the Instructor library, which dynamically creates the function schema via Pydantic models. After a long and tedious day, I solved it by using a python provider. It's not the best setup, but I did get it to work, my main issue was when using the python provider the api call function is async and i'm guessing prompt foo doesn't support that at the moment?
i
hey @daver987, this PR adds support for async
call_api
functions: It will be in the next release. Hope this helps!
d
Amazing job! This is great. Thank you!
4 Views