Hi @plain-jordan-64103! I think I may have figured out a potential reason about why your training runs were dying out. The trick is to not have uv venv in the project's root directory as Ray will copy everything to initialize its workers and the .venv directory is very large O(few GBs). Also, I think I mentioned this to you before, but make sure to set

export RAY_worker_register_timeout_seconds=600

in your

run_async_training.sh

script - without this I observed that it may happen that some vLLM workers will die out without killing the trainer and essentially you will just use 1/4 GPUs to sample rollouts significantly slowing down your trainer. I debugged a similar problem in a somewhat rushed/hasty manner just before the ICML deadline and this detail slipped my mind, due to which I may have shared a stale trainer script with you (really sorry about this mistake). Here is a detailed explanation about why this fix is needed from one of SkyRL's maintainers: https://github.com/NovaSky-AI/SkyRL/issues/224#issuecomment-3246168767. After this fix, the threads/Ray workers dying out errors should become extremely rare (though not completely non-existent).

Thanks! I have the ray worker timeout set to 1200 as per your recommendation, i think this should help with getting more stable launches. However the error that I mentioned in the meeting occurs after training has launched successfully (after 7-10 steps) which is where I'm struggling to debug. I will try to use the revised script you shared and see if that helps. Thanks again!

I am not an expert on Ray/SkyRL infra, but afaik, Ray workers are created not just for vLLM engines but also for the trainers as well and this is mostly not a one-off thing. My guess is that each call to this function will create 1 new Ray worker with 0.01 CPUs and will copy over .venv for each such call which is a very expensive operation. Since you were mentioning that the original source code (w/o the semantic search tool) did not work as well, it should no longer be the case if you incorporate the above changes. This can help us localize and figure out if the source of error is from our code or only specific to semantic search tool-related changes

That makes sense, I’ll try it out and get back to you