https://promptfoo.dev/ logo
Automatic Prompt verification
# ❓|questions
a
Does anyone use a prompt verification tool on a daily basis before executing a prompt? The benefits would be minimise cost, iterations, time prompting, error intend communication with AI. The goal is to get the best possible prompt in term of cost execution and quality of outputs. The idea would be to use different verification processes according to the complexity of the task asked so that for simple request a minimal verification is done but for more complex request, a deeper verification is made. The use cases can be regular prompts, custom gpts, skills, agent spec. I tried online tools for prompting check but it is hard to find a good balance between not enough verification and overkill prompt recommendations. I also tried to build a tool for that but the quality of outputs are not that good. Otherwise I asked also chat got to improve it but I end up with an heavy architecture for a prompt OS. I don't want the process to take longer or get more expensive than just executing the prompt and manually verifying the quality of the outputs. I don't want to increase complexity where it is not needed and at same time communicate as efficiently as possible my intention to the AI. What are your thoughts? How are your daily routines? What tools are you using?
i
wdym by prompt verification?
for prompts used in production, evaling them is definitely best practice 🙂
s
Is there any ai or way through which I can make prompt or train chatgpt I mean where I can get question which chat gpt takes 1 correct in 3 different chats I mean it should be very hard to get to it in first response
3 Views