https://www.boundaryml.com logo
How are you guys managing prompts in
# general
m
How are you guys managing prompts in practice? I’m realizing prompt management is not one problem. Some prompts are easy to store/version in tools like PromptLayer or similar registries, and some can be evaluated with straightforward tests. But a lot of prompts seem harder to evaluate because their quality only shows up in a real workflow, with full context, tools, and downstream effects. In those cases, the judgment can be pretty subjective and closer to an end-to-end or integration test than a unit test. How are you guys actually handling this in real systems: • Where do you store/version different kinds of prompts? • How do you decide which prompts can be tested directly vs only evaluated in-context? • For subjective prompts like code review, planning, or UI generation, what does “good enough” evaluation look like for you? • Are you using prompt registries, Git, RAG, human review, E2E evals, or some combination?
v
our bias is git! prompts are effectively code as they control your buisness logic and you dont want to end up with a:
key_prompt_final.md
vs
key_prompt_final_final_v2.md
for evals, we've generally found it to be very use case specific. here's two episodes (+ source code), where we spoke about this at lenght. https://boundaryml.com/podcast/2025-05-13-designing-evals https://boundaryml.com/podcast/2025-12-02-multimodal-evals
2 Views