Is there a way to resume a red team eval from the progress made once it has begun? If it's interrupted or runs into errors on the way and the eval is canceled, is there a way to pick up where it left off so it doesn't repeat all the same calls?
Running into this issue with large evals that take multiple hours, and it's a problem when you have to kill and restart the eval from scratch. I can't find anything in the docs about this.
i
ian
07/02/2025, 4:16 AM
Unfortunately not at the moment. It's a great idea though
ian
07/02/2025, 4:16 AM
@michael.michael.michael food for thought
b
Bryson
07/02/2025, 6:50 PM
Ah darn, but thanks for the answer! Yeah that would be super valuable, especially with larger evals that contain multiple multi-turn strategies that either take a really long time, or might be prone to failure and you don't want to lose progress