Paper on evaluating SRE agents: https://ai-discovery-in-the-wild.github.io/papers/4WKu8JP2kI.pdf

Thank you for the link! Speaking of SRE, I was just looking into https://arxiv.org/abs/2603.00468 The benchmark they propose also tries to take into account trajectories and some proxy for causality

There is also this SREGym paper from UIUC: https://www.sregym.com/. Their data curation is unfortunately manual, but Yiming mentioned to me that they are trying to curate a training dataset as well somewhat semi automatically.