Here is an initial implementation of a format reward addressing the three format issue found by the trajectory analysis script. I'm currently replicating Lintang's training earlier (Qwen3-14B, Batch Size x Rollout: 16x4, and Aditya's copy of Swe-Smith) with f1 reward + format reward and see if it helps.

Awesome! Could you open a PR for this?