https://promptfoo.dev/ logo
How to use PromptFoo with the new Github Models?
# ❓|questions
i
Hi, recently I have gained access to the beta of github models where we can try different set of models (gpt4 , gpt4mini, llama, cohere, llama, etc) I was trying to test it using promptfoo, but i failed in the process. Here you can check about different models: https://github.com/marketplace/models Do anyone here knows how can i use along github models?
m
@isaias I don't have access to these yet. If you can point me to developer docs / sdk etc I can help you set up a provider for them. Feel free to throw time on my calendar and we can look at these together https://cal.com/michael-dangelo/30min
@isaias can you call GH models using curl or the SDKs directly?
@isaias I got this working with
Copy code
providers:
  - id: openai:chat:gpt-4o-mini
    config:
      apiKey: YOUR_GITHUB_PAT
      apiBaseUrl: https://models.inference.ai.azure.com
i
😮