Hi! I am new to bento and yatai, my aim is to create versatile container that will load model on demand and have it as a runner. My current problem is how to pack models into service. I wanted to download it on demand with pull, but that does not work as a pull method will give me back `Error: [serve]

serve

failed: BentoCloud API token for default cloud context is required.` Thought of subprocess to wait for authentication, but it has timed out (probably unreachable).

import subprocess
import xml.etree.ElementTree as ETree

from bentoml.lightgbm import load_model
from pydantic import BaseModel

import bentoml


YOUR_TOKEN = "token"
subprocess.call(f"bentoml cloud login --api-token {YOUR_TOKEN} --endpoint <http://10.244.0.31:7777>".split(" "))

bentoml.models.pull("model-part0:latest")
bentoml.models.pull("model-part1:latest")
bentoml.models.pull("model-part2:latest")


class MessageModel(BaseModel):
    message: dict


@bentoml.service(
    resources={"cpu": "1"},
    traffic={"timeout": 10}
)
class dynamicmodelservice:
    def __init__(self):
        self.model: list = [
            load_model(f'model-part0:latest'),
            load_model(f'model-part1:latest'),
            load_model(f'model-part2:latest'),
        ]

    @bentoml.api(input_spec=MessageModel)
    def predict(self, message: dict) -> dict:
        ...

Can you please help me? Have already spent so much time on trying to load the model in container. Thank you very much!

Are you trying to use BentoCloud as a remote model repository and download model on demand?

Im trying to connect to Yatai repository and download those models. Don't know if BentoCloud is the same thing (I assume it is).

the

bentoml.cloud

sdk does not support yatai unfortunately

On my version when I tried

bentoml yatai login --api-token {YOUR_TOKEN} --endpoint <http://127.0.0.1:8000>

According to github Yatai ( https://github.com/bentoml/Yatai/blob/main/README.md ) during installation I tried bentoml yatai login and that did not worked. Then I tried bentoml cloud login and it worked just fine