https://promptfoo.dev/ logo
promptfoo + BioGPT
# ❓|questions
j
I am trying to integrate BioGPT (https://huggingface.co/microsoft/BioGPT-Large/tree/main) to run with promptfoo. I am not super familiar with tools that integrate with all of the JSON files in that repo's root. Do you know of a tool (integrated with promptfoo, besides Hugging Face Inference API) that can locally run this model?
For example, is there some way of using Hugging Face's
transformers.AutoModel
locally as a provider?
i
I think you can run it locally using
transformers
- as in this example
j
Yeah that's what I am doing. Does
promptfoo
have some way of invoking locally running
transformers
models? I am unsure which provider to use
i
I think you would have to use a custom script: https://promptfoo.dev/docs/providers/custom-script and invoke a python file that uses
transformers
j
Oh wow, didn't know that. I will give that a whirl next week, thanks! Looks like indeed it will work 👍
2 Views