Volkswagen emissions fraud, but for benchmarks: ðŸ«
• bundle solved trajectories in the binary (proprietary/closed of course)
• read from env when it's an eval
• append to the system prompt where to find the solution
• watch agent read and repeat it
--> no 1 on TerminalBench
Oh, but it'd be suspiciously fast, so how about: embed some jitter to force the agent to wait until it "solves" the task 😓