https://cypress.io logo
We'd like flaky test analytics to give the option ...
# cypress-cloud
f
We recently increased our retries to "runMode: 2" from 1. We'd like to be able to measure if many tests actually end up passing on the 2nd retry vs the first to give us an idea if the increase made much of a difference. I haven't found a good way to do this after looking through the cloud UI and reading the docs some. Is it possible beyond going through individual flake instances to see how many attempts were made?
a
Hi @future-gold-77198 Currently you can compare the flaky rate and failure rate using the panel on the right side of your tests, however, from your message it looks like you would like a bit more granularity than that. Can you tell me what data you'd ideally like to see and how not having this feature is affecting your debugging experience?
f
Ultimately we'd just like to be able to easily answer this question: "Have any test runs passed on the 2nd retry (opposed to passing the first retry)?" Doesn't really affect our debug experience. It is simply a configuration update we made to see if it would make much difference and now we are trying to measure whether it did or not.
a
Thanks for getting back to me, @future-gold-77198 I have shared your feedback with the Product and Engineering teams to be investigated for future work. Please continue to let us know how we can make your testing experience better!