https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
m
Copy code
2022-11-11T00:13:31+0500 [ERROR] [runner:traffic_violation_classification:1] Exception in ASGI application
Traceback (most recent call last):
  File "/home/mona/.local/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/mona/.local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await <http://self.app|self.app>(scope, receive, send)
  File "/home/mona/.local/lib/python3.10/site-packages/uvicorn/middleware/message_logger.py", line 86, in __call__
    raise exc from None
  File "/home/mona/.local/lib/python3.10/site-packages/uvicorn/middleware/message_logger.py", line 82, in __call__
    await <http://self.app|self.app>(scope, inner_receive, inner_send)
  File "/home/mona/.local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/mona/.local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/mona/.local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await <http://self.app|self.app>(scope, receive, _send)
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/opentelemetry/instrumentation/asgi/__init__.py", line 482, in __call__
    await <http://self.app|self.app>(scope, otel_receive, otel_send)
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/server/http/instruments.py", line 293, in __call__
    await <http://self.app|self.app>(scope, receive, wrapped_send)
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/server/http/access.py", line 126, in __call__
    await <http://self.app|self.app>(scope, receive, wrapped_send)
  File "/home/mona/.local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/home/mona/.local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await <http://self.app|self.app>(scope, receive, sender)
  File "/home/mona/.local/lib/python3.10/site-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/home/mona/.local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await <http://self.app|self.app>(scope, receive, send)
  File "/home/mona/.local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/server/runner_app.py", line 271, in _request_handler
    payload = await infer(params)
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/marshal/dispatcher.py", line 166, in _func
    raise r
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/marshal/dispatcher.py", line 232, in outbound_call
    outputs = await self.callback(tuple(d for _, d, _ in inputs_info))
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/server/runner_app.py", line 235, in infer_batch
    batched_params, indices = payload_paramss_to_batch_params(
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/utils.py", line 128, in payload_paramss_to_batch_params
    _converted_params = Params.agg(
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/utils.py", line 97, in agg
    args = tuple(
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/utils.py", line 98, in <genexpr>
    agg_func(tuple(params.args[i] for params in params_list))
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/utils.py", line 130, in <lambda>
    agg_func=lambda i: AutoContainer.from_batch_payloads(
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/container.py", line 546, in from_batch_payloads
    return container_cls.from_batch_payloads(payloads, batch_dim)
  File "/home/mona/.local/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/container.py", line 418, in from_batch_payloads
    return cls.batches_to_batch(batches, batch_dim)
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/container.py", line 367, in batches_to_batch
    indices = list(itertools.accumulate(len(subbatch) for subbatch in batches))
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/container.py", line 367, in <genexpr>
    indices = list(itertools.accumulate(len(subbatch) for subbatch in batches))
  File "/home/mona/.local/lib/python3.10/site-packages/scipy/sparse/_base.py", line 345, in __len__
    raise TypeError("sparse matrix length is ambiguous; use getnnz()"
TypeError: sparse matrix length is ambiguous; use getnnz() or shape[0]
2022-11-11T00:13:31+0500 [ERROR] [api_server:traffic_violation_classifier:2] Exception on /classify [POST] (trace=f3245b58dc5180b788d783eab8037232,span=ccb4bf4c270316cc,sampled=0)
Traceback (most recent call last):
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/server/http_app.py", line 311, in api_func
    output = await api.func(input_data)
  File "/home/mona/bentoml/bentos/traffic_violation_classifier/ppmb6idbfok5qaav/src/service.py", line 77, in classify
    prediction = await model_runner.predict.async_run(vector)
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 51, in async_run
    return await self.runner._runner_handle.async_run_method(  # type: ignore
  File "/home/mona/miniconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/runner/runner_handle/remote.py", line 163, in async_run_method
    raise RemoteException(
bentoml.exceptions.RemoteException: An exception occurred in remote runner traffic_violation_classification: [500] Internal Server Error
2022-11-11T00:13:31+0500 [INFO] [api_server:traffic_violation_classifier:2] 127.0.0.1:43388 (scheme=http,method=POST,path=/classify,type=application/json,length=718) (status=500,type=application/json,length=2) 1448.182ms (trace=f3245b58dc5180b788d783eab8037232,span=ccb4bf4c270316cc,sampled=0)
g
Same here! The problem goes away if I set batch able to false
👍 1
Debugging it looks like the model gets a malformed input when using --production, but it doesn't without that
To be honest, a bit of documentation of what exactly --production does would be helpful
m
Exactly, I finally understood the problem was from batchable, and the reason in my case was, I was using a One Hot Encoder that was creating a sparse vector, so when each input resulted in a different length vector, it failed
I didn't try yet, but after transforming the input in service.py with the One Hot Ecoder, I think I will try setting it to dense again, because batchable is what increases performance so setting it to false is kind of besides the point. I saw one of the examples in bentoML documentation about IRIS dataset (which is similar to my problem), but they've also set batchable to False. Not sure why. I couldn't find anything else.
Open in Slack
PreviousNext
Powered by