Some initial results from the ablation experiments: Based on Aditya’s latest PR and a highly reproducible baseline (using GSPO) and the upgraded SkyRL-train package, I tested several algorithm variants: 1. vanilla GRPO 2. SAPO 3. Some Dr.GRPO-style length normalization methods So far, none of them outperform our current baseline on the validation set (I used a subset of SWE-bench Lite for a quick pass). Interestingly, vanilla GRPO achieves higher training rewards but does not translate to better val performance. W&B: https://wandb.ai/leon_at_work/soni_ablation_4b/workspace But one strategy that does seem promising is prefiltering a subset before training — it leads to a noticeably steeper learning curve. The experiment is still running, and I’ll share updates once we have more conclusive results.

I think this is very good analysis that we should include this work in the paper. It would be great if you could push all the final ckpts to HF - I will eval them on Verified/Pro and add a table. Also it would be great if you could give concrete experimental details for each ckpt somewhere - in a google doc or directly edit the overleaf if you like.

Sure, I’ll prepare a PR with the configs and scripts, and also a comparison table for explanation next week. As for the ckpts, the problem is that I have quite a few checkpoints 😅 Here's a short eval summary :

Is the PR you're referring to the codebase cleanup one? I might try to cherry pick it and see if it works for me too

@acceptable-holiday-21046 You mean the one I used for baseline running? It's the latest version of

soni/main

by @busy-branch-28834 https://github.com/OpenHands/codescout/pull/73

You mentioned an upgraded skyrl-train package. Is that Aditya's fork that's pinned in

soni/main

or did you use a different version?

Oh I see. It's the latest skyrl-train with a few convenient patches used in my experiments. But basically using the official version won't change the results of our experiments.

So v0.4.0?

Not very sure... I made a fork here: https://github.com/yucc-leon/SkyRL/tree/merge-sapo-support But I guess using the latest official version for dependency would be ok.

@rich-byte-67407 - basically all I need is the final ckpt for each of the different algorithmic variants you tried before. It would be better if they were trained for the same number of steps (I think all algos were tried for alteast 200 steps which is fine) -- and if all algos are avbl for 200 steps, just push that ckpt for each training run which is just 4-6 ckpts I guess? I can make that table. We want arxiv early next week so I am pushing a bit

I see. I'll try to upload all the step200 ckpts soon.

Thank you for all your hard work 🙏

Thanks! Couldn’t have done it without your cleanup and fixes 🥹

Also since you are trying to see best data subsets for RL, this may help (reference from one tool is enough - reponavigator paper)

@busy-branch-28834 https://huggingface.co/Leon-Leee/codescout-alg-variants

Ok thanks! We also need to mention the training hyper-params somewhere in the paper.

Sure. Please let me know if there’s anything I should prepare.

I am revising the draft if you want to have a look there. sections 3, 4 are done. 5 and 6 will be done by tonight.

Nice, I’ll take a look later today

@rich-byte-67407 sorry if this is already present elsewhere, but could you simply write the detailed experimental configs somewhere describing the low-level details of all these variants?

The README here contains the configs and descriptions for each experiment: https://huggingface.co/Leon-Leee/codescout-alg-variants Could you check if that’s what you had in mind?

@rich-byte-67407 -- I have added a section in appendix with all your models evaluated on SWE-Bench Pro. (check table 9 and appendix G). I am not sure I am fully understanding all the above configs correctly. Could you please refine this section? I have written something meaningful based on what I understood, but it would be great if you could review this to verify the correctness of details and add more if desired. This is in part also because I may not be fully aware about what these training hyper-parameters are actually doing

Thanks a lot for putting this together — really appreciate it! I’ll go through those parts. Might add a bit more detail if needed. Will get back to you so

Thanks! You used 4 steps right in the multi-turn reward setting? I see that you mentioned 10 in the writeup? BTW the writeup is very nice, thanks for improving it!

Oh yes! Thanks for pointing that out. I’ll check the wording to make sure it’s not confusing. The max interaction turns during training is 10, but the multi-turn reward is given when the model finishes the task in exactly 4 turns. (If I remember correctly, that’s the same setup used for the 14B model in the paper.)

the prompt mentions 4 right?

Got it 🫡