Hello, can anyone verify if BentoML v1.1.11 uses asyncio as the asynchronous engine for the runners? I am trying to create a custom runner that runs a background task using asyncio but I am getting an error that claims there is

no current event loop in thread

.

Hi can you use

anyio

from_thread

or

to_thread

to manage async job? there is some intricacies within BentoML with async/sync conversion which we use

anyio

to manage

My main objective is to be able to monitor how much GPU one of my BentoML runners is using by creating a custom runner from it that, in addition to completing the runner's original task, executes a background task for gathering the metrics. Do you know what the best approach to achieving that would be? I have found that using Python's

threading

package works, but I would like to ideally do it without the use of threads. I have tried

asyncio

and

anyio

but both resulted in issues.

from anyio import from_thread
from_thread.run(xxxx)

should do the job and is the recommended way. Could you maybe share your code and maybe we can walk through together

Does

from_thread.run(xxxx)

block the current process until the function has concluded? My background task is meant to run indefinitely until the service ends so this would not be a feasible solution for my use case.