ze jing
04/11/2024, 12:18 PMFile "/usr/local/bin/bentoml", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/bentoml_cli/utils.py", line 377, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/bentoml_cli/utils.py", line 348, in wrapper
return_value = func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/bentoml_cli/start.py", line 479, in start_runner_server
start_runner_server_impl(
File "/usr/local/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
File "/usr/local/lib/python3.10/site-packages/bentoml/start.py", line 64, in start_runner_server
for runner in svc.runners:
AttributeError: 'Service' object has no attribute 'runners'
My deployment file is:
apiVersion: <http://resources.yatai.ai/v1alpha1|resources.yatai.ai/v1alpha1>
kind: Bento
metadata:
name: bento-blip-demo
namespace: kubeflow
spec:
tag: blip_image_captioning:bufsixhxw2iybyit
image: easzlab.io.local:5000/blip_image_captioning:bufsixhxw2iybyit
runners:
- name: bento-blip-demo-0
runnableType: XGBoost
---
apiVersion: <http://serving.yatai.ai/v2alpha1|serving.yatai.ai/v2alpha1>
kind: BentoDeployment
metadata:
name: bento-blip-demo
namespace: kubeflow
spec:
bento: bento-blip-demo
ingress:
enabled: false
resources:
limits:
cpu: 20000m
memory: "40Gi"
requests:
cpu: 10000m
memory: "20Gi"
autoscaling:
maxReplicas: 5
minReplicas: 1
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
runners:
- name: bento-blip-demo-0
resources:
limits:
cpu: 20000m
memory: "40Gi"
requests:
cpu: 10000m
memory: "20Gi"
autoscaling:
maxReplicas: 2
minReplicas: 1
In the same situation, I deployed the official BentoML example and it ran normally.
I have been troubleshooting this problem all day and have no clue. I hope you can help me, thank you!Alexei Fokin
04/11/2024, 1:41 PMAlexei Fokin
04/11/2024, 1:43 PMColin
04/11/2024, 2:55 PMSoungRyoul Kim
04/12/2024, 4:38 AMHugo Guilbot
04/12/2024, 8:51 AMbentoml serve bentoml_app_pandas.py:service --reload
, I encounter the following error: "`[ERROR] [api_server:11] The "bentoml.sklearn.load_runner" method is deprecated. "load_runner" arguments will be ignored. Use "bentoml.sklearn.get("pca:latest").to_runner()`" instead."
So, I've updated my code:
import bentoml
import pandas as pd
from <http://bentoml.io|bentoml.io> import NumpyNdarray, PandasDataFrame
import numpy as np
# Load transformers and model
# scaler = bentoml.sklearn.load_runner("scaler:latest", function_name="transform")
scaler = bentoml.sklearn.get("scaler:latest").to_runner()
# pca = bentoml.sklearn.load_runner("pca:latest", function_name="transform")
pca = bentoml.sklearn.get("pca:latest").to_runner()
# model = bentoml.sklearn.load_runner("customer_segmentation_kmeans:latest")
model = bentoml.sklearn.get("customer_segmentation_kmeans:latest").to_runner()
service = bentoml.Service("customer_segmentation_kmeans", runners=[scaler, pca, model])
@service.api(input=PandasDataFrame(), output=NumpyNdarray())
def predict(df: pd.DataFrame) -> np.array:
# Process data
scaled_df = pd.DataFrame([scaler.run(df)], columns=df.columns)
processed_df = pd.DataFrame([pca.run(scaled_df)], columns=['col1', 'col2', 'col3'])
# Predict data
result = model.run(processed_df)
return np.array(result)
But I'm getting the error:
(I don't use 'predict' in my code)
AttributeError: 'StandardScaler' object has no attribute 'predict'
2024-04-11T14:38:02+0200 [ERROR] [api_server:customer_segmentation_kmeans:2] Exception on /predict [POST] (trace=b038efd3e586ab0e884ec2f092b2bab3,span=1e547feb6f675e0b,sampled=0,service.name=customer_segmentation_kmeans)
Traceback (most recent call last):
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\bentoml\_internal\server\http_app.py", line 344, in api_func
output = await run_in_threadpool(api.func, *args)
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\starlette\concurrency.py", line 42, in run_in_threadpool
return await <http://anyio.to|anyio.to>_thread.run_sync(func, *args)
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\anyio\to_thread.py", line 49, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\anyio\_backends\_asyncio.py", line 2103, in run_sync_in_worker_thread
return await future
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\anyio\_backends\_asyncio.py", line 823, in run
result = context.run(func, *args)
File "C:\Users\hguilbot\Documents\BentoML_Project\src\bentoml_app_pandas.py", line 18, in predict
scaled_df = pd.DataFrame([scaler.run(df)], columns=df.columns)
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\bentoml\_internal\runner\runner.py", line 53, in run
return self.runner._runner_handle.run_method(self, *args, **kwargs)
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\bentoml\_internal\runner\runner_handle\remote.py", line 346, in run_method
anyio.from_thread.run(
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\anyio\from_thread.py", line 48, in run
return async_backend.run_async_from_thread(func, args, token=token)
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\anyio\_backends\_asyncio.py", line 2143, in run_async_from_thread
return f.result()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\concurrent\futures\_base.py", line 446, in result
return self.__get_result()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\concurrent\futures\_base.py", line 391, in __get_result
raise self._exception
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\anyio\_backends\_asyncio.py", line 2130, in task_wrapper
return await func(*args)
File "C:\Users\hguilbot\Documents\BentoML_Project\bentomlenv\lib\site-packages\bentoml\_internal\runner\runner_handle\remote.py", line 242, in async_run_method
raise RemoteException(
bentoml.exceptions.RemoteException: An unexpected exception occurred in remote runner scaler: [500] Internal Server Error
2024-04-11T14:38:02+0200 [INFO] [api_server:customer_segmentation_kmeans:2] 127.0.0.1:63611 (scheme=http,method=POST,path=/predict,type=application/json,length=185) (status=500,type=application/json,length=2) 132.446ms (trace=b038efd3e586ab0e884ec2f092b2bab3,span=1e547feb6f675e0b,sampled=0,service.name=customer_segmentation_kmeans)
I'm not sure what to do next. Do you have any ideas? (
I use "bentoml, version 1.2.10",so I think the reason is that I'm using a syntax that works in version 1.1 but I'm using 1.2, so if that's the case, do you have any idea how to help me?
Thanks youXabier Cadenas Yoldi
04/12/2024, 12:31 PMFile "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner.py", line 52, in run
return self.runner._runner_handle.run_method(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 346, in run_method
anyio.from_thread.run(
File "/usr/local/lib/python3.8/dist-packages/anyio/from_thread.py", line 48, in run
return async_backend.run_async_from_thread(func, args, token=token)
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 2143, in run_async_from_thread
return f.result()
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result
return self.__get_result()
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 2130, in task_wrapper
return await func(*args)
File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 242, in async_run_method
raise RemoteException(
bentoml.exceptions.RemoteException: An unexpected exception occurred in remote runner textbox_ocr_recognition: [404] Not Found
"POST http%3A//127.0.0.1%3A8000/model_response HTTP/1.1" 404
I can't replicate the issue locally. Is there some configuration I'm doing wrong? This is my configuration.yml:
api_server:
workers: 1
traffic:
timeout: 100
http:
port: 5000
And this is how I define the runners:
class RunnableClass(bentoml.Runnable):
SUPPORTED_RESOURCES = ("cpu",)
SUPPORTS_CPU_MULTI_THREADING = True
def __init__(self, model_artifact):
self.artifact = model_artifact
self.artifact.load(self.artifact.models_path)
@bentoml.Runnable.method(batchable=False)
def model_response(self, process_info):
"""
"""
logger = get_logger(__name__)
logger.info("%s detection request started",
process_info.get("process_name"))
image_str = process_info.get("image")
image = image_str2numpy(image_str)
result = self.artifact.recognize(image)
return result
model_runner = bentoml.Runner(
RunnableClass,
name="image_recognition",
runnable_init_params={"model_name": "image_recognition",})
svc = bentoml.Service("service", runners=[model_runner])
@svc.api(input=JSON(), output=JSON())
def image_recognition(annotations: JSON, ctx: bentoml.Context):
"""
"""
try:
annotations["process_name"] = "image_recognition"
with StopWatch(annotations["process_name"]):
response = model_runner.\
model_response.run(process_info=annotations)
except Exception as exception:
response = error_handler(exception)
return response
Do you have any idea where the issue might be coming from?Eric Riddoch
04/13/2024, 4:03 PMjiewpeng
04/14/2024, 8:47 AM@bentoml.service
decorator? Since we do not directly create an instance of the Service
, I don't think we can directly use the add_asgi_middleware
method, neither does adding the attribute middlewares
to our own class seem to work
e.g. I tried the following and it did not work
@bentoml.service
class MyService:
middlewares = [(MyMiddleware, {})]
Lucas
04/15/2024, 3:28 PMbentoml 1.1.7
.
Save model:
import bentoml
from transformers import AutoImageProcessor, AutoModelForImageClassification
preprocessor = AutoImageProcessor.from_pretrained("rizvandwiki/gender-classification")
model = AutoModelForImageClassification.from_pretrained("rizvandwiki/gender-classification")
bentoml.transformers.save_model('gender-classifier-preprocessor', preprocessor)
bentoml.transformers.save_model('gender-classifier-model', model)
bentofile:
service: 'service:svc'
include:
- '*'
python:
requirements_txt: requirements.txt
models:
- gender-classifier-preprocessor:latest
- gender-classifier-model:latest
Service:
import bentoml
from bentoml.io import Image, NumpyNdarray
preprocessor_runner = bentoml.transformers.get("gender-classifier-preprocessor").to_runner()
model_runner = bentoml.transformers.get("gender-classifier-model").to_runner()
svc = bentoml.Service("gender-classifier", runners=[preprocessor_runner, model_runner])
@svc.api(input=Image(), output=NumpyNdarray())
def gender_classifier(img: Image):
inputs = preprocessor_runner.run(img, return_tensors="pt")
predictions = model_runner.run(inputs["pixel_values"])
return predictions.numpy()
And I'm getting an error: bentoml.exceptions.MissingDependencyException: None (reason: No module named 'tensorflow')
But there no mention of tensorflow anywhere in my codeHamarios
04/16/2024, 7:39 PMPanslothda
04/16/2024, 8:55 PMdocker run --rm -it -p 3000:3000 -e TRUST_REMOTE_CODE=True -e HUGGING_FACE_HUB_TOKEN=XXXXXXXXXXXXXXXXXXXX ghcr.io/bentoml/openllm start databricks/dbrx-instruct --backend pt
Which I assume is because in github readme it says
Note: Dbrx requires to install with:
pip install "openllm[dbrx]"
But how do I add the install to the container? I didn't really find a documentation for the container besides the run commands in readme unfortunately 😕
EDIT: Actually seems i have a general problem as well.
Tried to use a model that does not require install.
But then I just get this.... O.o
File "/openllm-python/src/openllm/serialisation/transformers/_helpers.py", line 33, in infer_autoclass_from_llm
raise ValueError(
ValueError: Invalid configuration for google/flan-t5-large. ``trust_remote_code=True`` requires `transformers.PretrainedConfig` to contain a `auto_map` mapping
2024-04-16T21:06:49+0000 [ERROR] [runner:llm-flan-t5-runner:1] Application startup failed. Exiting.
This is now simple "start google/flan-t5-large --backend pt" for the container.
Doing this on an up2date debian 12Isu Yu
04/17/2024, 6:03 AMErwin de Haan
04/17/2024, 12:02 PMErwin de Haan
04/17/2024, 12:02 PMErwin de Haan
04/17/2024, 1:39 PMErwin de Haan
04/17/2024, 1:40 PMTom Matthews
04/17/2024, 2:17 PMapi_v2-whisper-1 | 2024-04-17 14:02:55.996 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=1.8219. model_execution_time_s=1.3762, runner_queue_time_s=0.4457, runner_id=139997053363856, server_id='oWJg'
api_v2-whisper-1 | 2024-04-17 14:02:57.308 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=5.6077. model_execution_time_s=2.3055, runner_queue_time_s=3.3022, runner_id=139820294678096, server_id='oWJg'
api_v2-whisper-1 | 2024-04-17 14:02:58.174 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=2.4474. model_execution_time_s=0.5741, runner_queue_time_s=1.8734, runner_id=139820294678096, server_id='oWJg'
api_v2-whisper-1 | 2024-04-17 14:02:58.310 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=2.4084. model_execution_time_s=1.7065, runner_queue_time_s=0.7019, runner_id=140034448911888, server_id='oWJg'
api_v2-whisper-1 | 2024-04-17 14:02:58.738 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=1.6986. model_execution_time_s=0.7371, runner_queue_time_s=0.9615, runner_id=139820294678096, server_id='zSgj'
api_v2-whisper-1 | 2024-04-17 14:03:00.076 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=5.4660. model_execution_time_s=1.3389, runner_queue_time_s=4.1271, runner_id=139820294678096, server_id='zSgj'
api_v2-whisper-1 | 2024-04-17 14:03:00.550 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=1.4659. model_execution_time_s=0.6177, runner_queue_time_s=0.8482, runner_id=140034448911888, server_id='oWJg'
api_v2-whisper-1 | 2024-04-17 14:03:00.691 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=11.1913. model_execution_time_s=0.4736, runner_queue_time_s=10.7177, runner_id=139820294678096, server_id='oWJg'
api_v2-whisper-1 | 2024-04-17 14:03:00.923 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=3.0950. model_execution_time_s=0.7635, runner_queue_time_s=2.3315, runner_id=139820294678096, server_id='oWJg'
api_v2-whisper-1 | 2024-04-17 14:03:03.693 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=3.5236. model_execution_time_s=2.1864, runner_queue_time_s=1.3373, runner_id=139997053363856, server_id='oWJg'
api_v2-whisper-1 | 2024-04-17 14:03:05.903 | INFO | service:transcribe:134 - Generated transcription in api_execution_time_s=5.8310. model_execution_time_s=5.7058, runner_queue_time_s=0.1251, runner_id=140034448911888, server_id='zSgj'
Here's my configuration.yml
:
api_server:
workers: 3
traffic:
timeout: 90
max_concurrency:
# max_runner_connections: 1
runners:
resources:
<http://nvidia.com/gpu|nvidia.com/gpu>: 1
workers_per_resource: 3
traffic:
timeout: 60
max_concurrency:
Does anyone have any thoughts on the cause of this issue, and any recommendations for solutions? I'm thinking about experimenting with the max_runner_concurrency or reducing the number of runners (taking a hit to throughput, in exchange for smoother latency across requests)Gary Chen
04/17/2024, 10:47 PMbentoml serve service:Summarization
command, I get the following error
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/mambaforge/base/envs/venv/bin/bentoml", line 5, in <module>
from bentoml_cli.cli import cli
File "/opt/homebrew/Caskroom/mambaforge/base/envs/venv/lib/python3.12/site-packages/bentoml_cli/cli.py", line 57, in <module>
cli = create_bentoml_cli()
^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/mambaforge/base/envs/venv/lib/python3.12/site-packages/bentoml_cli/cli.py", line 8, in create_bentoml_cli
from bentoml._internal.configuration import BENTOML_VERSION
File "/opt/homebrew/Caskroom/mambaforge/base/envs/venv/lib/python3.12/site-packages/bentoml/__init__.py", line 26, in <module>
load_config()
File "/opt/homebrew/Caskroom/mambaforge/base/envs/venv/lib/python3.12/site-packages/bentoml/_internal/configuration/__init__.py", line 184, in load_config
BentoMLConfiguration(
File "/opt/homebrew/Caskroom/mambaforge/base/envs/venv/lib/python3.12/site-packages/bentoml/_internal/configuration/containers.py", line 67, in __init__
self.config = get_default_config(version=use_version)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/mambaforge/base/envs/venv/lib/python3.12/site-packages/bentoml/_internal/configuration/helpers.py", line 111, in get_default_config
assert hasattr(mod, "SCHEMA"), (
^^^^^^^^^^^^^^^^^^^^^^
AssertionError: version 1 does not have a validation schema
I'm running off the latest release bentoml==1.2.11
Felix Ivander G
04/18/2024, 2:43 AMLucas
04/18/2024, 12:50 PMservice: 'models.serving.service:MyService
include:
- 'commons/**/*.py'
python:
requirements_txt: requirements-commons.txt
requirements_txt: /requirements.txt
Tom RIVERO
04/18/2024, 1:59 PMbentoml serve
I encounter the following issue :
Traceback (most recent call last):
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/bin/bentoml", line 8, in <module>
sys.exit(cli())
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/utils.py", line 362, in wrapper
return func(*args, **kwargs)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/utils.py", line 333, in wrapper
return_value = func(*args, **kwargs)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/utils.py", line 290, in wrapper
return func(*args, **kwargs)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/env_manager.py", line 122, in wrapper
return func(*args, **kwargs)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml_cli/serve.py", line 260, in serve
serve_http_production(
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/serve.py", line 327, in serve_http_production
json.dumps(runner.scheduled_worker_env_map),
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 356, in scheduled_worker_env_map
for worker_id in range(self.scheduled_worker_count)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 341, in scheduled_worker_count
return self.scheduling_strategy.get_worker_count(
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/runner/strategy.py", line 68, in get_worker_count
resource_request = system_resources()
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/resource.py", line 46, in system_resources
res[resource_kind] = resource.from_system()
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/bentoml/_internal/resource.py", line 248, in from_system
pynvml.nvmlInit()
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/pynvml/nvml.py", line 1770, in nvmlInit
nvmlInitWithFlags(0)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/pynvml/nvml.py", line 1760, in nvmlInitWithFlags
_nvmlCheckReturn(ret)
File "/home/tom/Desktop/Stage/ml-reconciliation/venv/lib/python3.10/site-packages/pynvml/nvml.py", line 833, in _nvmlCheckReturn
raise NVMLError(ret)
pynvml.nvml.NVMLError_DriverNotLoaded: Driver Not Loaded
It seems to be a nvidia driver related issue, but I couldn't find a working solution by myself so I'm asking for your help 🙏
About my working environment :
BentoML: 1.1.11
Python: 3.10
torch: 2.2.1
Ubuntu: 22.04
no Nvidia GPUEric Riddoch
04/18/2024, 11:16 PMSuhas
04/19/2024, 6:08 AMToke Emil Heldbo Reines
04/19/2024, 8:10 AMhj d
04/22/2024, 7:38 AMhj d
04/22/2024, 7:39 AMPanslothda
04/22/2024, 11:11 AMAdam Liter
04/22/2024, 8:08 PMbentoml build --containerize .
no longer work, especially with bentoml>=1.2.x
? Details in thebentoml.models.get
not from the __init__
method but instead assign the result of that to a class variable. This wasn’t that clear to me from the documentation on the Services doc page.Sebastián Cepeda
04/24/2024, 8:09 PM