This message was deleted.

What were the errors that caused these failures?

@Sean please find below the error message:

2022-11-04T14:29:58.314084791Z 2022-11-04T14:29:58+0000 [INFO] [api_server:1] 109.205.64.66:37643 (scheme=http,method=POST,path=/mrrr_similarities_async,type=application/json,length=9321) (status=500,type=application/json,length=110) 1189.316ms (trace=a3f4eab83bd5eec6c077ee49a9c3748f,span=bae39225181c5df0,sampled=0)
2022-11-04T14:29:58.314614512Z 2022-11-04T14:29:58+0000 [INFO] [api_server:4] 109.205.64.66:48905 (scheme=http,method=POST,path=/mrrr_similarities_async,type=application/json,length=9321) (status=500,type=application/json,length=110) 721.918ms (trace=1cb12d99a2000730adc09d3ffbbe4496,span=ea61914c8ffe6192,sampled=0)
2022-11-04T14:29:58.315006090Z 2022-11-04T14:29:58+0000 [ERROR] [api_server:1] Exception on /mrrr_similarities_async [POST] (trace=d8731f9c64f71d12715bf85cef71bb78,span=ed86b0ddab16c208,sampled=0)
2022-11-04T14:29:58.315018316Z Traceback (most recent call last):
2022-11-04T14:29:58.315023559Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/server/http_app.py", line 311, in api_func
2022-11-04T14:29:58.315028482Z     output = await api.func(input_data)
2022-11-04T14:29:58.315041058Z   File "/home/bentoml/bento/src/bentoml_gpu_onnx_ct2_service.py", line 178, in mrrr_similarities_async
2022-11-04T14:29:58.315077043Z     return await onnx_mrrr_runner.run.async_run(encoded_input_onnx.get('input_ids'), encoded_input_onnx.get('attention_mask'))
2022-11-04T14:29:58.315105483Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner.py", line 51, in async_run
2022-11-04T14:29:58.315181129Z     return await self.runner._runner_handle.async_run_method(  # type: ignore
2022-11-04T14:29:58.315187821Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 149, in async_run_method
2022-11-04T14:29:58.315190917Z     async with <http://self._client.post|self._client.post>(
2022-11-04T14:29:58.315193632Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 1141, in __aenter__
2022-11-04T14:29:58.315196475Z     self._resp = await self._coro
2022-11-04T14:29:58.315199106Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 560, in _request
2022-11-04T14:29:58.315201895Z     await resp.start(conn)
2022-11-04T14:29:58.315204604Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client_reqrep.py", line 899, in start
2022-11-04T14:29:58.315207386Z     message, payload = await protocol.read()  # type: ignore[union-attr]
2022-11-04T14:29:58.315210183Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/streams.py", line 616, in read
2022-11-04T14:29:58.315212965Z     await self._waiter
2022-11-04T14:29:58.315215685Z aiohttp.client_exceptions.ClientOSError: [Errno 104] Connection reset by peer
2022-11-04T14:29:58.315606465Z 2022-11-04T14:29:58+0000 [ERROR] [api_server:4] Exception on /mrrr_similarities_async [POST] (trace=149fec75ce62d70750ed1d02a3b8351f,span=513de76c0b0ddd80,sampled=0)
2022-11-04T14:29:58.315618192Z Traceback (most recent call last):
2022-11-04T14:29:58.315622246Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/server/http_app.py", line 311, in api_func
2022-11-04T14:29:58.315626417Z     output = await api.func(input_data)
2022-11-04T14:29:58.315630204Z   File "/home/bentoml/bento/src/bentoml_gpu_onnx_ct2_service.py", line 178, in mrrr_similarities_async
2022-11-04T14:29:58.315634372Z     return await onnx_mrrr_runner.run.async_run(encoded_input_onnx.get('input_ids'), encoded_input_onnx.get('attention_mask'))
2022-11-04T14:29:58.315638377Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner.py", line 51, in async_run
2022-11-04T14:29:58.315642500Z     return await self.runner._runner_handle.async_run_method(  # type: ignore
2022-11-04T14:29:58.315646628Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 149, in async_run_method
2022-11-04T14:29:58.315667932Z     async with <http://self._client.post|self._client.post>(
2022-11-04T14:29:58.315672319Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 1141, in __aenter__
2022-11-04T14:29:58.315683415Z     self._resp = await self._coro
2022-11-04T14:29:58.315687595Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 560, in _request
2022-11-04T14:29:58.315691731Z     await resp.start(conn)
2022-11-04T14:29:58.315695711Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client_reqrep.py", line 899, in start
2022-11-04T14:29:58.315699779Z     message, payload = await protocol.read()  # type: ignore[union-attr]
2022-11-04T14:29:58.315727791Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/streams.py", line 616, in read
2022-11-04T14:29:58.315745449Z     await self._waiter
2022-11-04T14:29:58.315749618Z aiohttp.client_exceptions.ClientOSError: [Errno 104] Connection reset by peer
2022-11-04T14:29:58.316132642Z 2022-11-04T14:29:58+0000 [ERROR] [api_server:3] Exception on /mrrr_similarities_async [POST] (trace=d7f378776c0cbb60647d2be8d5ac3a6b,span=cd6ffb06ef8280cf,sampled=0)
2022-11-04T14:29:58.316140303Z Traceback (most recent call last):
2022-11-04T14:29:58.316144466Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/server/http_app.py", line 311, in api_func
2022-11-04T14:29:58.316148298Z     output = await api.func(input_data)
2022-11-04T14:29:58.316150947Z   File "/home/bentoml/bento/src/bentoml_gpu_onnx_ct2_service.py", line 178, in mrrr_similarities_async
2022-11-04T14:29:58.316153853Z     return await onnx_mrrr_runner.run.async_run(encoded_input_onnx.get('input_ids'), encoded_input_onnx.get('attention_mask'))
2022-11-04T14:29:58.316173745Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner.py", line 51, in async_run
2022-11-04T14:29:58.316180760Z     return await self.runner._runner_handle.async_run_method(  # type: ignore
2022-11-04T14:29:58.316184898Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 149, in async_run_method
2022-11-04T14:29:58.316189135Z     async with <http://self._client.post|self._client.post>(
2022-11-04T14:29:58.316222466Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 1141, in __aenter__
2022-11-04T14:29:58.316229323Z     self._resp = await self._coro
2022-11-04T14:29:58.316232583Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 560, in _request
2022-11-04T14:29:58.316235450Z     await resp.start(conn)
2022-11-04T14:29:58.316278870Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client_reqrep.py", line 899, in start
2022-11-04T14:29:58.316285269Z     message, payload = await protocol.read()  # type: ignore[union-attr]
2022-11-04T14:29:58.316289099Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/streams.py", line 616, in read
2022-11-04T14:29:58.316293093Z     await self._waiter
2022-11-04T14:29:58.316297221Z aiohttp.client_exceptions.ClientOSError: [Errno 104] Connection reset by peer