what should happen is like this let's say you have retries = 2 1st round: - fail (retry) - fail (retry) - fail (retry) then the whole CI fails then you manually retry the execution again, this time it will execute the entiire suite that's normal behavior