hello we are running into an odd error when our code is runn BentoML #ask-for-help

hello we are running into an odd error when our co...

Frank Cash

05/07/2024, 10:24 PM

hello we are running into an odd error when our code is running on kubernetes

Copy code

2024-05-07T22:12:37+0000 [ERROR] [api_server:1] Exception on /1.0/route/route [POST] 
  File "/usr/local/lib/python3.10/site-packages/anyio/from_thread.py", line 45, in run
    asynclib = threadlocals.current_async_module
AttributeError: '_thread._local' object has no attribute 'current_async_module'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/server/service_app.py", line 316, in api_func
    output = await api.func(input_data, ctx)
  File "/home/bentoml/bento/src/service.py", line 442, in calculate_score
    result = await bento_runner.compute_func.run(hexbin_data, demo_data, guid)
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 44, in run
    return self.runner._runner_handle.run_method(  # type: ignore
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/runner/runner_handle/remote.py", line 211, in run_method
    return anyio.from_thread.run(  # type: ignore (pyright cannot infer the return type)
  File "/usr/local/lib/python3.10/site-packages/anyio/from_thread.py", line 47, in run
    raise RuntimeError("This function can only be run from an AnyIO worker thread")
RuntimeError: This function can only be run from an AnyIO worker thread

more details in 🧵

👀 1

Frank Cash

05/07/2024, 10:27 PM

where

bento_runner = bentoml.Runner(Runnable)

and

Copy code

class Runnable(bentoml.Runnable):
    SUPPORTED_RESOURCES = ("cpu",)
    SUPPORTS_CPU_MULTI_THREADING = True

    @bentoml.Runnable.method(batchable=False)
    async def compute_func(self, hexbin_data, demo_data, guid) -> pd.DataFrame:
        ...

amd we call this when a route is hit as follows

Copy code

@svc.api(input=JSON(), output=PandasDataFrame(), route="/1.0/route/route")
async def calculate_score(input, context: bentoml.Context) -> pd.DataFrame:
    start = time.time()
    # Testing to see what the context object will give us in a Lambda
    kwargs_dict = input["kwargs"]
    LOG.info(f"Correlation ID: {input['correlation_id']}")
    guid = input["correlation_id"]
    data = kwargs_dict["data"]
    other_data = kwargs_dict["demo_data"]
    result = await rsci_impact_runner.compute_impact.run(data, other_data, guid)

we are running on EKS

Copy code

python --version #Python 3.10.14

and

Copy code

bentoml 1.0.4
aiohttp 3.8.11
anyio 3.6.2

Open in Slack

Previous Next