https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
b
can you share the error logs?
m
Sure
C:\Users\IBS6 Software\anaconda3\lib\site-packages\paramiko\transport.py219 CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, 2022-12-21T234407+0100 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "quali-type-classifier-service.py:svc" can be accessed at http://localhost:3000/metrics. 2022-12-21T234409+0100 [INFO] [cli] Starting development HTTP BentoServer from "quali-type-classifier-service.py:svc" listening on http://0.0.0.0:3000 (Press CTRL+C to quit) 2022-12-21T234410+0100 [INFO] [dev_api_server:qualification_type_service] Jax version 0.4.1, Flax version 0.3.3 available.
This line keeps on printing :INFO] [dev_api_server:qualification_type_service] Jax version 0.4.1, Flax version 0.3.3 available.
b
can you run with 
--debug
flag on and see what's the erro??
what command are you running?
m
I am doing bentoml serve <service_python_file.py>:svc
C:\Users\IBS6 Software\anaconda3\lib\site-packages\paramiko\transport.py219 CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, 2022-12-21T235610+0100 [DEBUG] [cli] Importing service "quali-type-classifier-service.py:svc" from working dir: "C:\Users\IBS6 Software\Documents\IBS6_Project\Analyzer" 2022-12-21T235611+0100 [DEBUG] [cli] Default runner method set to '__call__', it can be accessed both via 'runner.run' and 'runner.call.async_run'. 2022-12-21T235611+0100 [DEBUG] [cli] 'qualification_type_service' imported from source: bentoml.Service(name="qualification_type_service", import_str="quali-type-classifier-service:svc", working_dir="C:\Users\IBS6 Software\Documents\IBS6_Project\Analyzer") 2022-12-21T235611+0100 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "quali-type-classifier-service.py:svc" can be accessed at http://localhost:3000/metrics. 2022-12-21T235612+0100 [INFO] [cli] Installing handle_callback_exception to loop 2022-12-21T235612+0100 [INFO] [cli] Registering signals... 2022-12-21T235612+0100 [INFO] [cli] Starting master on pid 22512 2022-12-21T235612+0100 [DEBUG] [cli] Socket bound at 0.0.0.0:3000 - fd: 396 2022-12-21T235612+0100 [INFO] [cli] sockets started 2022-12-21T235612+0100 [DEBUG] [cli] Initializing watchers 2022-12-21T235612+0100 [DEBUG] [cli] cmd: C:\Users\IBS6 Software\anaconda3\python.exe 2022-12-21T235612+0100 [DEBUG] [cli] args: ['-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py:svc', '--fd', '$(circus.sockets._bento_api_server)', '--working-dir', 'C:\\Users\\IBS6 Software\\Documents\\IBS6_Project\\Analyzer', '--prometheus-dir', 'C:\\Users\\IBS6 Software\\bentoml\\prometheus_multiproc_dir'] 2022-12-21T235612+0100 [DEBUG] [cli] process args: ['C:\\Users\\IBS6', 'Software\\anaconda3\\python.exe', '-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py:svc', '--fd', '396', '--working-dir', 'C:\\Users\\IBS6 Software\\Documents\\IBS6_Project\\Analyzer', '--prometheus-dir', 'C:\\Users\\IBS6 Software\\bentoml\\prometheus_multiproc_dir'] 2022-12-21T235612+0100 [DEBUG] [cli] running dev_api_server process [pid 20648] 2022-12-21T235612+0100 [INFO] [cli] Arbiter now waiting for commands 2022-12-21T235612+0100 [INFO] [cli] dev_api_server started 2022-12-21T235612+0100 [INFO] [cli] Starting development HTTP BentoServer from "quali-type-classifier-service.py:svc" listening on http://0.0.0.0:3000 (Press CTRL+C to quit) 2022-12-21T235613+0100 [DEBUG] [dev_api_server] Importing service "quali-type-classifier-service.py:svc" from working dir: "C:\Users\IBS6 Software\Documents\IBS6_Project\Analyzer" 2022-12-21T235614+0100 [DEBUG] [dev_api_server] Default runner method set to '__call__', it can be accessed both via 'runner.run' and 'runner.call.async_run'. 2022-12-21T235614+0100 [DEBUG] [dev_api_server] 'qualification_type_service' imported from source: bentoml.Service(name="qualification_type_service", import_str="quali-type-classifier-service:svc", working_dir="C:\Users\IBS6 Software\Documents\IBS6_Project\Analyzer") 2022-12-21T235614+0100 [DEBUG] [dev_api_server] 'tracing.sample_rate' is set to zero. No traces will be collected. Please refer to https://docs.bentoml.org/en/latest/guides/tracing.html for more details. 2022-12-21T235614+0100 [INFO] [dev_api_server] Jax version 0.4.1, Flax version 0.3.3 available. 2022-12-21T235616+0100 [DEBUG] [cli] cmd: C:\Users\IBS6 Software\anaconda3\python.exe 2022-12-21T235616+0100 [DEBUG] [cli] args: ['-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py:svc', '--fd', '$(circus.sockets._bento_api_server)', '--working-dir', 'C:\\Users\\IBS6 Software\\Documents\\IBS6_Project\\Analyzer', '--prometheus-dir', 'C:\\Users\\IBS6 Software\\bentoml\\prometheus_multiproc_dir'] 2022-12-21T235616+0100 [DEBUG] [cli] process args: ['C:\\Users\\IBS6', 'Software\\anaconda3\\python.exe', '-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py:svc', '--fd', '396', '--working-dir', 'C:\\Users\\IBS6 Software\\Documents\\IBS6_Project\\Analyzer', '--prometheus-dir', 'C:\\Users\\IBS6 Software\\bentoml\\prometheus_multiproc_dir'] 2022-12-21T235616+0100 [DEBUG] [cli] running dev_api_server process [pid 11548] 2022-12-21T235617+0100 [DEBUG] [dev_api_server] Importing service "quali-type-classifier-service.py:svc" from working dir: "C:\Users\IBS6 Software\Documents\IBS6_Project\Analyzer" 2022-12-21T235618+0100 [DEBUG] [dev_api_server] Default runner method set to '__call__', it can be accessed both via 'runner.run' and 'runner.call.async_run'. 2022-12-21T235618+0100 [DEBUG] [dev_api_server] 'qualification_type_service' imported from source: bentoml.Service(name="qualification_type_service", import_str="quali-type-classifier-service:svc", working_dir="C:\Users\IBS6 Software\Documents\IBS6_Project\Analyzer") 2022-12-21T235618+0100 [DEBUG] [dev_api_server] 'tracing.sample_rate' is set to zero. No traces will be collected. Please refer to https://docs.bentoml.org/en/latest/guides/tracing.html for more details. 2022-12-21T235618+0100 [INFO] [dev_api_server] Jax version 0.4.1, Flax version 0.3.3 available. 2022-12-21T235620+0100 [DEBUG] [cli] cmd: C:\Users\IBS6 Software\anaconda3\python.exe 2022-12-21T235620+0100 [DEBUG] [cli] args: ['-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py:svc', '--fd', '$(circus.sockets._bento_api_server)', '--working-dir', 'C:\\Users\\IBS6 Software\\Documents\\IBS6_Project\\Analyzer', '--prometheus-dir', 'C:\\Users\\IBS6 Software\\bentoml\\prometheus_multiproc_dir'] 2022-12-21T235620+0100 [DEBUG] [cli] process args: ['C:\\Users\\IBS6', 'Software\\anaconda3\\python.exe', '-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py:svc', '--fd', '396', '--working-dir', 'C:\\Users\\IBS6 Software\\Documents\\IBS6_Project\\Analyzer', '--prometheus-dir', 'C:\\Users\\IBS6 Software\\bentoml\\prometheus_multiproc_dir'] 2022-12-21T235620+0100 [DEBUG] [cli] running dev_api_server process [pid 17708] 2022-12-21T235621+0100 [DEBUG] [dev_api_server] Importing service "quali-type-classifier-service.py:svc" from working dir: "C:\Users\IBS6 Software\Documents\IBS6_Project\Analyzer" 2022-12-21T235622+0100 [DEBUG] [dev_api_server] Default runner method set to '__call__', it can be accessed both via 'runner.run' and 'runner.call.async_run'. 2022-12-21T235622+0100 [DEBUG] [dev_api_server] 'qualification_type_service' imported from source: bentoml.Service(name="qualification_type_service", import_str="quali-type-classifier-service:svc", working_dir="C:\Users\IBS6 Software\Documents\IBS6_Project\Analyzer") 2022-12-21T235622+0100 [DEBUG] [dev_api_server] 'tracing.sample_rate' is set to zero. No traces will be collected. Please refer to https://docs.bentoml.org/en/latest/guides/tracing.html for more details. 2022-12-21T235622+0100 [INFO] [dev_api_server] Jax version 0.4.1, Flax version 0.3.3 available. 2022-12-21T235623+0100 [INFO] [cli] Got signal SIG_INT 2022-12-21T235623+0100 [INFO] [cli] Arbiter exiting 2022-12-21T235623+0100 [DEBUG] [cli] stopping the dev_api_server watcher 2022-12-21T235623+0100 [DEBUG] [cli] gracefully stopping processes [dev_api_server] for 30.0s 2022-12-21T235623+0100 [DEBUG] [cli] dev_api_server: kill process 17708 2022-12-21T235623+0100 [DEBUG] [cli] sending signal Signals.SIGTERM to 17708 2022-12-21T235624+0100 [DEBUG] [cli] reaping already dead process 17708 [dev_api_server] 2022-12-21T235624+0100 [INFO] [cli] dev_api_server stopped
Here are the logs after adding --debug
Shall I show you the python service file as well?
s
Sure! It looks to me like your service is erroring for some reason; I think circus sometimes swallows logs.
m
import bentoml from bentoml.io import Text, JSON runner = bentoml.transformers.get("qualification-type-classifier:cpwptbubnoqwqagy").to_runner() svc = bentoml.Service("qualification_type_service", runners=[runner]) @svc.api(input=Text(), output=JSON()) async def classify(input_series: str) -> list: res = await runner.predict.async_run(input_series) print(res) return res
Here it is
s
What happens if you just do 
python service.py
?
m
Nothing it runs and simply exits
s
Hm, odd. What version of BentoML are you running? Could you build / export the bento and send it to us?
m
I am using version 1.0.12. And yeah I already have it built where shall I send it?
s
Oh! I think I figured it out; it looks like we aren't handling spaces in filenames properly. Would you be able to run this in a folder with no space in the name? (IBS6 Software is the culprit, here, I think)
We should definitely fix this.
m
Ok i'll try that, thanks
s
Sorry, I think the actual problem is that your python executable is in a folder with a space.
m
Yeah cause it hanged again
I'll check if I can change the python executable and run from a different environment
s
That would be great! Sorry about this.
m
Yeah same thing happens
C\Users\Arebi\anaconda3\lib\site packages\paramiko\transport.py219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, 2022-12-22T003726+0100 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "quali-type-classifier-service.py" can be accessed at http://localhost:3000/metrics. 2022-12-22T003728+0100 [INFO] [cli] Starting development HTTP BentoServer from "quali-type-classifier-service.py" running on http://0.0.0.0:3000 (Press CTRL+C to quit) 2022-12-22T003729+0100 [WARNING] [dev_api_server:qualification_type_service] No versions of Flax or Jax are found under the current machine. In order to use Flax with transformers 4.x and above, refers to https://github.com/google/flax#quick-install
I changed the envronment to my user Arebi (it doesn't have fkax or jax but I guess it's just a warning)
So i've been playing around with and got this error "Runner is not initialized" when using the runner seperatly. Why is that?
s
Ah, the runner is meant to be run specifically with 
bentoml serve
and not really separately.
Wait, is it still hanging? Can you send the debug logs?
m
Yeah
C\Users\Arebi\anaconda3\lib\site packages\paramiko\transport.py219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, 2022-12-22T011914+0100 [DEBUG] [cli] Importing service "quali-type-classifier-service.py" from working dir: "C:\Users\Arebi\Desktop\Analyzer" 2022-12-22T011914+0100 [DEBUG] [cli] Default runner method set to 
__call__
, it can be accessed both via 
runner.run
and 
runner.__call__.async_run
2022-12-22T011914+0100 [DEBUG] [cli] 'qualification_type_service' imported from source: bentoml.Service(name="qualification_type_service", import_str="quali-type-classifier-service:svc", working_dir="C:\Users\Arebi\Desktop\Analyzer") 2022-12-22T011914+0100 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "quali-type-classifier-service.py" can be accessed at http://localhost:3000/metrics. 2022-12-22T011915+0100 [INFO] [cli] Registering signals... 2022-12-22T011916+0100 [INFO] [cli] Starting master on pid 22584 2022-12-22T011916+0100 [DEBUG] [cli] Socket bound at 0.0.0.0:3000 - fd: 776 2022-12-22T011916+0100 [INFO] [cli] sockets started 2022-12-22T011916+0100 [DEBUG] [cli] Initializing watchers 2022-12-22T011916+0100 [DEBUG] [cli] cmd: C:\Users\Arebi\anaconda3\python.exe 2022-12-22T011916+0100 [DEBUG] [cli] args: ['-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py', '--fd', '$(circus.sockets._bento_api_server)', '--working-dir', 'C:\\Users\\Arebi\\Desktop\\Analyzer', '--prometheus-dir', 'C:\\Users\\Arebi\\bentoml\\prometheus_multiproc_dir'] 2022-12-22T011916+0100 [DEBUG] [cli] process args: ['C:\\Users\\Arebi\\anaconda3\\python.exe', '-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py', '--fd', '776', '--working-dir', 'C:\\Users\\Arebi\\Desktop\\Analyzer', '--prometheus-dir', 'C:\\Users\\Arebi\\bentoml\\prometheus_multiproc_dir'] 2022-12-22T011916+0100 [DEBUG] [cli] running dev_api_server process [pid 25216] 2022-12-22T011916+0100 [INFO] [cli] Arbiter now waiting for commands 2022-12-22T011916+0100 [INFO] [cli] dev_api_server started 2022-12-22T011916+0100 [INFO] [cli] Starting development HTTP BentoServer from "quali-type-classifier-service.py" running on http://0.0.0.0:3000 (Press CTRL+C to quit) 2022-12-22T011917+0100 [DEBUG] [dev_api_server] Importing service "quali-type-classifier-service.py" from working dir: "C:\Users\Arebi\Desktop\Analyzer" 2022-12-22T011917+0100 [DEBUG] [dev_api_server] Default runner method set to 
__call__
, it can be accessed both via 
runner.run
and 
runner.__call__.async_run
2022-12-22T011917+0100 [DEBUG] [dev_api_server] 'qualification_type_service' imported from source: bentoml.Service(name="qualification_type_service", import_str="quali-type-classifier-service:svc", working_dir="C:\Users\Arebi\Desktop\Analyzer") 2022-12-22T011917+0100 [WARNING] [dev_api_server] No versions of Flax or Jax are found under the current machine. In order to use Flax with transformers 4.x and above, refers to https://github.com/google/flax#quick-install 2022-12-22T011919+0100 [DEBUG] [dev_api_server] Falling back to TensorFlow client; we recommended you install the Cloud TPU client directly with pip install cloud-tpu-client. 2022-12-22T011919+0100 [DEBUG] [dev_api_server] Creating converter from 7 to 5 2022-12-22T011919+0100 [DEBUG] [dev_api_server] Creating converter from 5 to 7 2022-12-22T011919+0100 [DEBUG] [dev_api_server] Creating converter from 7 to 5 2022-12-22T011919+0100 [DEBUG] [dev_api_server] Creating converter from 5 to 7 2022-12-22T011920+0100 [INFO] [dev_api_server] Note: NumExpr detected 16 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. 2022-12-22T011920+0100 [INFO] [dev_api_server] NumExpr defaulting to 8 threads. 2022-12-22T011922+0100 [DEBUG] [cli] cmd: C:\Users\Arebi\anaconda3\python.exe 2022-12-22T011922+0100 [DEBUG] [cli] args: ['-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py', '--fd', '$(circus.sockets._bento_api_server)', '--working-dir', 'C:\\Users\\Arebi\\Desktop\\Analyzer', '--prometheus-dir', 'C:\\Users\\Arebi\\bentoml\\prometheus_multiproc_dir'] 2022-12-22T011922+0100 [DEBUG] [cli] process args: ['C:\\Users\\Arebi\\anaconda3\\python.exe', '-m', 'bentoml_cli.worker.http_dev_api_server', 'quali-type-classifier-service.py', '--fd', '776', '--working-dir', 'C:\\Users\\Arebi\\Desktop\\Analyzer', '--prometheus-dir', 'C:\\Users\\Arebi\\bentoml\\prometheus_multiproc_dir'] 2022-12-22T011922+0100 [DEBUG] [cli] running dev_api_server process [pid 21112] 2022-12-22T011923+0100 [DEBUG] [dev_api_server] Importing service "quali-type-classifier-service.py" from working dir: "C:\Users\Arebi\Desktop\Analyzer" 2022-12-22T011923+0100 [DEBUG] [dev_api_server] Default runner method set to 
__call__
, it can be accessed both via 
runner.run
and 
runner.__call__.async_run
2022-12-22T011923+0100 [DEBUG] [dev_api_server] 'qualification_type_service' imported from source: bentoml.Service(name="qualification_type_service", import_str="quali-type-classifier-service:svc", working_dir="C:\Users\Arebi\Desktop\Analyzer")
s
Hm, can you let it run for a little longer? Maybe use pastebin or something to paste longer logs.
Does it keep looping and printing things now?
m
Yeah exactly it prints the same logs each time
My transformers pipeline is the TextClassificationPipeline does BentoML only support only the normal huggingface Pipeline
s
I think that shouldn't be a problem, did you get a longer run?
m
Yeah it's still running and keeps on looping the same logs
s
Can you copy it all and send it in a pastebin?
m
Sure
Here you go
s
Thanks!
One more thing: can you try doing 
bentoml serve --production --debug
instead and see if it prints anything?
m
Ok
Now it works but a new error happens
2022-12-22T014413+0100 [ERROR] [runnerqualification type classifier2] Traceback (most recent call last): File "C:\Users\Arebi\anaconda3\lib\site-packages\starlette\routing.py", line 671, in lifespan async with self.lifespan_context(app): File "C:\Users\Arebi\anaconda3\lib\site-packages\starlette\routing.py", line 566, in aenter await self._router.startup() File "C:\Users\Arebi\anaconda3\lib\site-packages\starlette\routing.py", line 650, in startup handler() File "C:\Users\Arebi\anaconda3\lib\site-packages\bentoml\_internal\runner\runner.py", line 230, in init_local self._init_local() File "C:\Users\Arebi\anaconda3\lib\site-packages\bentoml\_internal\runner\runner.py", line 221, in _init_local self._init(LocalRunnerRef) File "C:\Users\Arebi\anaconda3\lib\site-packages\bentoml\_internal\runner\runner.py", line 215, in _init runner_handle = handle_class(self) File "C:\Users\Arebi\anaconda3\lib\site-packages\bentoml\_internal\runner\runner_handle\local.py", line 25, in init self._runnable = runner.runnable_class(**runner.runnable_init_params) # type: ignore File "C:\Users\Arebi\anaconda3\lib\site-packages\bentoml\_internal\frameworks\transformers.py", line 473, in init self.pipeline = load_model(bento_model, **kwargs) File "C:\Users\Arebi\anaconda3\lib\site-packages\bentoml\_internal\frameworks\transformers.py", line 216, in load_model with open(bento_model.path_of(PIPELINE_PICKLE_NAME), "rb") as f: FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\IBS6 Software\\bentoml\\models\\qualification-type-classifier\\cpwptbubnoqwqagy\\pipeline.v1.pkl'
Maybe this caused the first problem
s
Ah, yeah. That sounds like it's more likely. Not sure why that's not surfacing in development mode...
m
There is no file pipeline.v1.pkl' I saved the pipeline with bentoml.transformers.save_model(name="Qualification-Type-Classifier", pipeline=pipe)
The pipeline: pipe = TextClassificationPipeline(model=model.to('cpu'), tokenizer=tokenizer, top_k=1)
s
What's the result of 
dir C:\\Users\\IBS6 Software\\bentoml\\models\\qualification-type-classifier\\cpwptbubnoqwqagy\\
?
m
21/12/2022 21:07 <DIR> . 21/12/2022 21:07 <DIR> .. 21/12/2022 21:07 1,016 config.json 21/12/2022 21:07 442 model.yaml 21/12/2022 21:07 438,014,325 pytorch_model.bin 21/12/2022 21:07 132 special_tokens_map.json 21/12/2022 21:07 416 tokenizer_config.json 21/12/2022 21:07 262,030 vocab.txt 6 File(s) 438,278,361 bytes 2 Dir(s) 40,364,027,904 bytes free
s
Ok, I think this is a bug with our transformers support 😅
m
Thanks a lot for the help, hopefully this problem detected an issue you can solve. I'll try something different
Maybe the normal pipeline or something else I'll see
s
Can you quickly paste the model.yaml here too?
m
name: qualification-type-classifier version: cpwptbubnoqwqagy module: bentoml.transformers labels: {} options: task: '' tf: [] pt: [] default: {} type: null kwargs: {} metadata: {} context: framework_name: transformers framework_versions: transformers: 4.20.1 bentoml_version: 1.0.12 python_version: 3.9.13 signatures: __call__: batchable: false api_version: v1 creation_time: '2022-12-21T200721.414466+00:00'
s
I think you might be able to work around this by providing 
task_name='text-classification'
to 
save_model
, but that might break other things.
m
Yeah same error it doesn't save the pipeline pickle somehow
I guess this is where the pickle of the pipeline should be saved, but somehow it doesn't do it. I worked around it by doing it manually but maybe you can look into it
s
Great that you resolved the issue; we've figured out the problem and hopefully it should be resolved soon.
m
Thank you for your help
3 Views
Open in Slack
PreviousNext
Powered by