Hi everyone! These are the next steps we have in mind based on our discussion today: 1. Use format rewards in training to see if the sudden dip in rewards can be fixed. 2. Try LoRA+RL to see if the step-wise training speeds up. Using a larger model with LoRA seems to be a better choice than using a smaller model without LoRA. 3. Evaluate Qwen3 models with non-thinking mode and their instruct variants. Also try the instruct models without step-wise training. 4. Develop a custom finish tool that the agent must end its trajectory with -- this can possibly help with the sudden dip in rewards or in general be a better scaffold design. 5. Do SFT on the base model before RL: a. Prepare dataset for SFT by performing rollouts using a stronger LLM b. Train a smaller LLM with rejection sampling fine-tuning 6. Prepare SWE-Gym, SWE-Smith, SWE-rebench and SWE-Bench (Verified/Lite) datasets and push to HF. In case we forgot to add something in this list, please let me know. Also if you worked/plan to work on any of these steps and did not mention it here already, please do so in order to avoid repetitive efforts! I am currently looking at tasks 6 and 3 (though it would be nice if multiple variants are tried for the training sub-task in 3).

I'll work on 2. I created a corresponding issue https://github.com/OpenHands/agentic-code-search-oss/issues/53

Thanks for the summary Aditya!

Thanks Aditya. I'm looking into 1 and 4