Hi again, I want to use BentoML to build an ML pipeline with multiple components (services). It would process videos, and take multiple times the length of the video to finish processing. For example, it might take 30 min to process a 10 min video. So I was thinking of having one main service that receives the request, starts the pipeline, and returns immediately. Now in order for the client to keep track of the progress I'd have to return some request ID here. Then the client could query with this ID and receive the current status. What would be the best way to go about implementing this in BentoML? I actually found these docs here talking about an inference context and the context.state global attribute that can be read/written across API endpoints and is a per-worker dictionary. Would this be the way to go here? So in this case I wouldn't return any request ID to the client when it does the request but rather just keep track of it internally. I'd then have to implement some status route that basically returns the status of any request running in any of the workers.

Hi @Georg Alem, typically for this type of async inference task/jobs, we’d recommend using an external queue system (e.g. Redis) for updating and fetching job status. BentoML’s inference context already provide a UUID you could use. You can also create a custom UUID for each request programmatically (e.g.

uuid.*uuid4()*

)

Hi @Chaoyu I see. Ok, thanks!