This message was deleted.
# ask-for-help
s
This message was deleted.
a
Can you provide your service definition?
s
I have added device in the pipeline example
Copy code
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
pipeline(task="text2text-generation", model=model, tokenizer=tokenizer,device=device)
Further i call runner for inference
Copy code
runner = bentoml.transformers.get("model:latest").to_runner()
service = bentoml.Service("model", runners=[runner])
@service.api():
runer.run("its an example")
a
that is not a valid service definition no?
s
Yes, this is not a valid, i changed due to privacy. But it has essential elements