Hey guys. Seems like we being talking in dm. Let's talk here so we can help you better

has renamed the channel from "support-runner_process" to "support-vasil"

Hello! So apart from the several model instances that i need to run in parallel on CPU, my other question would be - can i schedule big jobs with lots of images somehow... and not wait for them to complete with an open HTTP connection but rather let them run in the background and check for their status occasionally... something like BackgroundTasks in FastAPI

I see you are looking for running async?

I think what you are looking for is async endpoint. Right now we don't have that at the moment

Is that a blocker for you?

Right now you do can run multiple models parallel

By use the my_runner.predict.async_run

yes, it needs to be async and i need to run multiple instances of the same model

and, having something similar to jobs running in the background, as described in the last section of the answer here - https://stackoverflow.com/questions/63169865/how-to-do-multiprocessing-in-fastapi would be amazing

Adding people who are experts compare to me here