Dajana Muho
05/24/2023, 11:21 PMspec:
containerConcurrency: 80
timeoutSeconds: 1800
serviceAccountName: xx
containers:
- name: x-anomaly-detection-1
image: xxx
ports:
- name: http1
containerPort: 3000
env:
- name: BENTOML_PORT
value: '3000'
- name: BENTOML__APISERVER__DEFAULT_TIMEOUT
value: '1800'
resources:
limits:
cpu: 4000m
memory: 16Gi
startupProbe:
timeoutSeconds: 240
periodSeconds: 240
failureThreshold: 1
tcpSocket:
port: 3000
Chaoyu
05/26/2023, 2:19 AMBENTOML__APISERVER__DEFAULT_TIMEOUT
env var is an legacy option only available in BentoML < 1.0. For 1.0 and above, you can use a config file to configure the timeout optionsBENTOML_CONFIG_OPTIONS='api_server.timeout=300'
Dajana Muho
05/26/2023, 4:55 PMtextPayload: "Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/server/http_app.py", line 336, in api_func
output = await run_in_threadpool(api.func, input_data)
File "/usr/local/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/home/bentoml/bento/src/src/services/bento_service.py", line 90, in forecast_target_stats
return time_series_runner.predict.run('target_stats', input)
File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 52, in run
return self.runner._runner_handle.run_method(self, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/runner/runner_handle/remote.py", line 291, in run_method
anyio.from_thread.run(
File "/usr/local/lib/python3.10/site-packages/anyio/from_thread.py", line 49, in run
return asynclib.run_async_from_thread(func, *args)
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 970, in run_async_from_thread
return f.result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/runner/runner_handle/remote.py", line 220, in async_run_method
async with <http://self._client.post|self._client.post>(
File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 1141, in __aenter__
self._resp = await self._coro
File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 560, in _request
await resp.start(conn)
File "/usr/local/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 894, in start
with self._timer:
File "/usr/local/lib/python3.10/site-packages/aiohttp/helpers.py", line 721, in __exit__
raise asyncio.TimeoutError from None"
Im wondering if there is anything I can do on the code side to improve it.
The runner is declared like
class timeseriesforecasting(bentoml.Runnable):
SUPPORTED_RESOURCES = ("cpu",)
SUPPORTS_CPU_MULTI_THREADING = True
@bentoml.Runnable.method(batchable=False)
Does it look good to you, should i add any other config here ?
Is a bit strange why on my local machine works, but on the Google Cloud Run fails 😕Chaoyu
05/30/2023, 4:55 PMBENTOML_CONFIG_OPTIONS="api_server.timeout=1800 runners.timeout=1800"