Can you also once double check if this logic is used as-is in your codebase: https://github.com/OpenHands/codescout/blob/c03d1fd74a61d8165e6a3bf35c3245f48e27f0de/src/generator/code_search_generator.py#L420-L480 (especially lines 452-456). Also just curious what is the general trend of initial reward curve (for first 50-100 steps) -- if it is a flat-line then there is likely some bug in the codebase.

Will this code fix the reproduction issue? My curves seem incorrect: https://wandb.ai/leon_at_work/soni_baseline_4b/workspace

Yes it should. The older bug in loss mask caused the agent to be trained only on the first turn and all other tirns were masked due to the older bug 💀

Yes, it seems like the code is there (though I believe from before the clean up?) Reward curve also seems to be going up though the reward values are pretty low. Looking at the outputs, it gets the paths wrong (also inconsistent with localization finish tool). Previously I saw the reward reaching 2-3 after around 50-60 steps with 4B thinking.

I can guarantee that the code on soni/main branch works. I trained the 4B instruct and the 1.7B models with it and they have similar reward curves as that before cleanup

Yeah I believe I had pulled from major-update previously and made changes on top of it. Will try re-pulling from Soni/main and see if any bug was introduced from my chnages

Thanks! Now it looks much better:

Nice. Which model is this? And is this stepwise reward by any chance?

Qwen3 4B with stepwise=false; Qwen3 4B 2507 instruct vs Qwen3 4B: 2507 may have encountered instability after ~100 steps when trying to get more F1 reward with more steps https://wandb.ai/leon_at_work/soni_baseline_4b/workspace

What is the batch size for the pink curve? Are you training on the full dataset (like this entire training run is <1 epoch right?). And finally what is the max steps allowed in prompt vs max steps allowed in backend? I never trained beyond 200 steps with batch size of 8, 8 rollouts per sample. \

The run was configured as follows: 1. epoch = 1 on the full swe-smith-py subset/your released version 2. rollout_n = 8 and train_batch_size = 8 (default) 3. 4 turns by default 4. Enabled additional multi-turn reward with max_turns = 4 and min_turns = 4 (mainly out of curiosity to see whether this would improve the run). I left the job running overnight and didn’t actively monitor it. And this morning I noticed the crashed somewhere between step 100~200.

ok thanks! What hardware did you use?

H200*8

Ok this is great (I used H100).

Very helpful! Thanks :D

It should have impact -- our system prompt asks agent to use rg 😶‍🌫️

Yes, it had an impact. Not clearly visible in the logs but the curves changed a lot:

Yes, I need to write READMEs properly. It would be great if you could keep it running till 150 steps to reliably see if it works properly or not

Oh wait, I should say that until more steps have been finished. The curves are smoothed differently. Sorry for the careless conclusion.

Alright, let's see if this helps. The final (unlikely) possibility might be that as rollouts proceed the subsequent steps use RAM intensive operations that may be timing out/causing failures in your runs. We never saw this with our machines though

Got it 🫡 I'll report back soon.

@rich-byte-67407 -- did the training run go fine for you? Your wandb logs seem to suggest it worked but I wanted to double check this

Just opened my dashboard. The curves look smooth all the way to 300 steps; everything seems to be running fine. Thanks again for all the great work on this!

Ok great, so now we have someone who could reproduce our setup (hopefully without much hand-holding from our side). I will write READMEs and it would be great if you could check if you had to do anything extra beyond what was written to setup the infra/train the models

1. Glad to help. 2. Yes, I did it by setting

- fn: multiturn_reward
    args:
      minimal_turns: 4
      maximal_turns: 4

in the 4b's yaml config 3. No problem. I'm also running evals on many checkpoints, but I can upload the 300th step's in a few hours.

Thanks! Just to clarify, are the results comparable to what we have in the paper for the older ckpt?

You mean the CodeScout 4B? Step300's results are actually even competitive with CodeScout-14B on SWE-Bench Lite from my runs. Would be great to get cross-checked on your side as well — especially on Pro/Verified — just in case there are any subtle differences in dataset or eval setup.