Hi everyone! Here is the update from my side: 1. Training infra: a. RL runs: I have completed one run with 100 steps and added eval results as Graham sir mentioned (see here). I started another run with batch size=32 and a few more tools (grep, glob and bash) -- I see increasing rewards but it has several issues which we can discuss today. b. Training logic: I studied the code closely and could not find major issues (there were some bug fixes which were merged in this PR) -- the code change is small but has high downstream impact so please pull before large-scale experiments. c. Timing analysis: I analyzed the time it takes for different steps in the workflow and there is a significant time delay involved in the training (fwd/bckwd passes) with step-wise setup and is the primary bottleneck. Rollouts are not the bottleneck for us. Please use 4-4 or 2-6 split for inference-train engines respectively, there is no point in allocating more GPUs to rollouts. 2. Eval scripts: This code is now ready for use (the dataset is pushed locally for now, but I will push it to HF soon) 3. Dataset logic: Please see the update on this issue and here is the updated code for dataset preparation