This message was deleted.

Hi @Dajana Muho - sorry about the confusion, the

BENTOML__APISERVER__DEFAULT_TIMEOUT

env var is an legacy option only available in BentoML < 1.0. For 1.0 and above, you can use a config file to configure the timeout options

Hey @Chaoyu Thank you very much for your time to check this issue. We are still facing the same issue even though we applied the change you suggest. It crushes on the deployment env with the error,

textPayload: "Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/server/http_app.py", line 336, in api_func
    output = await run_in_threadpool(api.func, input_data)
  File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/home/bentoml/bento/src/src/services/bento_service.py", line 90, in forecast_target_stats
    return time_series_runner.predict.run('target_stats', input)
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 52, in run
    return self.runner._runner_handle.run_method(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/runner/runner_handle/remote.py", line 291, in run_method
    anyio.from_thread.run(
  File "/usr/local/lib/python3.10/site-packages/anyio/from_thread.py", line 49, in run
    return asynclib.run_async_from_thread(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 970, in run_async_from_thread
    return f.result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/runner/runner_handle/remote.py", line 220, in async_run_method
    async with <http://self._client.post|self._client.post>(
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 560, in _request
    await resp.start(conn)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 894, in start
    with self._timer:
  File "/usr/local/lib/python3.10/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None"

Im wondering if there is anything I can do on the code side to improve it. The runner is declared like

class timeseriesforecasting(bentoml.Runnable):
    SUPPORTED_RESOURCES = ("cpu",)
    SUPPORTS_CPU_MULTI_THREADING = True

    @bentoml.Runnable.method(batchable=False)

Does it look good to you, should i add any other config here ? Is a bit strange why on my local machine works, but on the Google Cloud Run fails 😕

Hi @Dajana Muho, from the error message, looks like you have a timeout on the Runner call, could you try set a timeout for the runner as well? e.g.:

BENTOML_CONFIG_OPTIONS="api_server.timeout=1800 runners.timeout=1800"

Thanks for your help @Chaoyu that change fixed the issue.