Most interestingly, vanilla GRPO and Dr.GRPO variant is now working without errors on H200s. I wonder what changed from our earlier experiments (1.5 months back) where the training would collapse to zero rewards even with H100s (Lintang faced this problem with 14B, and this was the reason we shifted to GSPO). At this point, I have tried so many variants of the training code/algorithm, that I no longer have any idea what we were doing wrong. But the good news is that the current code is working as expected across training algorithms lolsob

same confusion🥹