I want to reduce the size of dimensions, if possible, to reduce storage size, and I would like to test different dimensions with my use cases to understand the impact of it.
m
michaelmichaelmichael
11/22/2024, 4:11 PM
This is a really interesting use case and can definitely make it work in Promptfoo. What metrics are you thinking about using to evaluate your embeddings?
r
raphael_03334
11/26/2024, 6:57 PM
It is a good question; I am unsure which metric is the best. My use case is semantic similarity, and I would like to check if reducing the dimensions will still bring the same results using cosine similarity. So my idea is to have some test cases, create embedding for each test case, and in some way check if the cosine similarity is the same when comparing with other embedded text. Does it make sense? I am open to ideas on how to do it.
m
michaelmichaelmichael
11/26/2024, 8:31 PM
What are you building? Are you doing this for an application or for a class/research project?