https://promptfoo.dev/ logo
Is there any example of using promptfoo to evaluat...
# ❓|questions
r
I want to reduce the size of dimensions, if possible, to reduce storage size, and I would like to test different dimensions with my use cases to understand the impact of it.
m
This is a really interesting use case and can definitely make it work in Promptfoo. What metrics are you thinking about using to evaluate your embeddings?
r
It is a good question; I am unsure which metric is the best. My use case is semantic similarity, and I would like to check if reducing the dimensions will still bring the same results using cosine similarity. So my idea is to have some test cases, create embedding for each test case, and in some way check if the cosine similarity is the same when comparing with other embedded text. Does it make sense? I am open to ideas on how to do it.
m
What are you building? Are you doing this for an application or for a class/research project?
r
It is an application of the company I work for.
m
book a time on my calendar https://cal.com/michael-dangelo/15min and we can chat about it