https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
x
No need to have the same number of replicas
b
hmm ya that's what I thought
Do u know what could cause this?
Here's a snippet:
Copy code
detected_card_image = await card_detector_runnable.detect.async_run(
        image=image
    )
    ktp_1: KTP = await easyocr_runnable.detect.async_run(
        image=detected_card_image,
        reason="",
        overall_start=overall_start,
        hparams=hparams,
        debug=False,
        enable_metrics=False,
        nik_threshold=0.75,
    )

    ktp_2: KTP = await doctr_runnable.detect.async_run(
        image=detected_card_image,
        reason="",
        overall_start=overall_start,
        hparams=hparams,
        debug=False,
        enable_metrics=False,
        nik_threshold=0.75,
    )
Seems like the problem stems at
Copy code
ktp_1: KTP = await easyocr_runnable.detect.async_run(
But what I don't understand is why it works locally and occasionally in prod
Am I doing this right?
x
what’s the error log?
b
Copy code
023-01-13T17:34:14.597553548Z   File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 160, in async_run_method
2023-01-13T17:34:14.597556400Z     async with <http://self._client.post|self._client.post>(
2023-01-13T17:34:14.597559798Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 1141, in __aenter__
2023-01-13T17:34:14.597562458Z     self._resp = await self._coro
2023-01-13T17:34:14.597565232Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 560, in _request
2023-01-13T17:34:14.597568598Z     await resp.start(conn)
2023-01-13T17:34:14.597575825Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/client_reqrep.py", line 899, in start
2023-01-13T17:34:14.597578559Z     message, payload = await protocol.read()  # type: ignore[union-attr]
2023-01-13T17:34:14.597581010Z   File "/usr/local/lib/python3.8/dist-packages/aiohttp/streams.py", line 616, in read
2023-01-13T17:34:14.597583578Z     await self._waiter
2023-01-13T17:34:14.597586286Z aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
2023-01-13T17:34:14.598250567Z 2023-01-13T17:34:14+0000 [INFO] [api_server:1] 10.65.140.137:43048 (scheme=http,method=POST,path=/api/v0/predict,type=image/bmp,length=4517517) (status=500,type=application/json,length=110) 24113.531ms (trace=16377efb9ecc4315489d39cc590bec4b,span=0bb31926addd62c5,sampled=0)
2023-01-13T17:34:29.679542241Z Image shape:
Inference doesn't take more than 6 seconds so I doubt it's a timeout problem
Could
Copy code
[2023-01-14 01:28:12] [Unhealthy] Readiness probe failed: Get "<http://10.65.137.93:3000/readyz>": dial tcp 10.65.137.93:3000: connect: connection refused
[2023-01-14 01:29:22] [Unhealthy] Liveness probe failed: Get "<http://10.65.137.93:3000/livez>": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
[2023-01-14 01:29:32] [Unhealthy] Readiness probe failed: Get "<http://10.65.137.93:3000/readyz>": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
cause the 
ServerDisconnect
?
Open in Slack
PreviousNext
Powered by