https://promptfoo.dev/ logo
Life after sqlite migration
# ❓|questions
a
Hey everyone. At my last job I selected Promptfoo for our llm testing framework. I just switched jobs and went to implement the latest version and noticed that since [version 0.48](https://github.com/promptfoo/promptfoo/releases/tag/0.48.0) json is replaced as with sqlite. I wonder how other people are managing prompt organization. For example, this is how I organize my prompts in a project
Copy code
src
|_ prompts
   |_ prompt-that-classifies-utterance
   |  |_ prompt.ts
   |  |_ tests
   |     |_ output
   |     |_ testExecutor.ts
   |
   |_ prompt-that-extracts-dates
      |_ prompt.ts
      |_ tests
         |_ output
         |_ testExecutor.ts
I could cd into each /tests directory and run
promptfoo view .
which would only show me evals associated with a specific prompt. This was very neat. Now I don't see how I can distinguish between all of the prompts in my project since they all get saved to a single sqllite database. Is there a legitamate path forward with JSON, or is the sqlite the only way?
i
Hmm, I have not thought about this use case. I wonder if we could add a
promptfoo view --filter <pattern>
option that only shows matching config descriptions. Do you think something like that could help?
a
That would be really helpful. especially if there is a way to assign a name to specific evals -- to denote the prompt!
i
cool, I have a branch with this change and will try to get it in the next release 🙂
a
@ian You're really cool. Promptfoo is an awesome tool. Hopefully some day I carve out some time to contribute.
i
Sorry this took a little while. PR here: https://github.com/promptfoo/promptfoo/pull/780
it will be in the next version!
2 Views