This message was deleted.

@Aaron Pham or @larme (shenyang) do you know anything about this^?

Hi Sangeon, currently our PyTorch runner only support tensor/ndarray inputs. You can create a custom runner and move the tensors inside dictionary manually. An custom runner example is at https://github.com/bentoml/BentoML/blob/main/examples/custom_model_runner/service.py . You should add codes to put tensors inside dictionary to GPU before line 38 (

output = super().__call__(input_data)

).

Problem solved! Thanks a lot!!