Powered by
Promptfoo evaluation with custom provider and loca...
# ❓|questions
k
kacperz
03/24/2026, 3:19 PM
I’m evaluating candidate LLMs for my app and need to compare models in a setup where context comes from local files, fetched by my own agent layer (OpenClaw), not by Promptfoo directly. Is it possible to evaluate models using this external context flow as part of Promptfoo evaluation? Promptfoo is currently running tests through a custom provider (defined in a local file), which calls my OpenClaw layer and returns the model output.
https://cdn.discordapp.com/attachments/1486021607053856829/1486021607372754974/message.txt?ex=69c4a5b8&is=69c35438&hm=b348f0ae3d2b8691281ec3449d143202db44f1a6f1a40a23bc16be43fb86d55c&
2
Views
Previous
Next