This message was deleted.

Hi there, can you send your

bentofile.yaml

that you used to containerize the image?

Hi @Aaron Pham please find below the `bentofile.yaml`:

service: "bentoml_gpu_onnx_ctranslate2_service:mrr"  # Same as the argument passed to `bentoml serve`
labels:
    owner: tinycoaching-ml-team
    stage: dev
include:
- "*.py"  # A pattern for matching which files to include in the bento
exclude:
- "__pycache__/"
- "tokenizer/"
- "model_default/"
- "model_float16/"
- "model_int8/"
- "model_int8_float16/"
- "onnx_save_models_bentoml.py"
- "serve_bentoml_API.py"
- "service.py"
- "transformers_save_model.py"
- "yaml_files/"
python:
    packages:  # Additional pip packages required by the service
    - "transformers==4.23.1"
    - "sentencepiece==0.1.97"
    - "onnx==1.10.2"
    - "onnxruntime-gpu==1.9.0"
    - "pydantic==1.10.2"
    - "six>=1.4.0"
    - "websocket-client>=0.32.0"
    - "python-dateutil==2.8.2"
    - "tzlocal==4.2"
    - "attrs>=17.4.0"
    - "protobuf==3.19.6"
docker:
    distro: debian
    python_version: "3.8.13"
    cuda_version: "11.6.2"