Serve Bento and reload with new bentotag Hello I try to ser BentoML #ask-for-help

Serve Bento and reload with new bentotag Hello! I ...

Chris

02/26/2024, 8:45 AM

Serve Bento and reload with new bentotag Hello! I try to serve a endpoint locally on a pc. I tried to start the server with the following command:

Copy code

serve_http_production(
    bentotag,
    working_dir=working_dir,
    port=8088,
    host="0.0.0.0",
    api_workers=1,
    reload=False,
    development_mode=False,
)

After some time (e.g. 20 minutes) i want to end the serving and restart the server with a possible refreshed bento tag (e.g. new service is available for serving). Any idea how i could solve this in python (e.g. processing,...)?

Jian Shen Yap

02/26/2024, 2:19 PM

you can either use

threading

processing

to schedule your task. may I ask what are the usecase for doing so?

Chris

02/26/2024, 2:54 PM

@Jian Shen Yap thanks for the answer, i will try this tomorrow. I do not not yet what is the best way to end the http server, beside terminating the process I have a quite old Windows machine and am not able/allowed to use docker :D

Chris

02/27/2024, 8:51 AM

@Jian Shen Yap I tried to start the endpoint the following way:

Copy code

working_dir = "."
if os.path.isdir(os.path.expanduser(self._active_deployment)):
    working_dir = os.path.expanduser(self._active_deployment)
keywords = {
'bento_identifier': self._active_deployment,
'working_dir': working_dir,
'port': 8088,
'host': "0.0.0.0",
'api_workers': 1,
'reload': False,
'development_mode': False,
}
self._inference_endpoint_process=Process(target=serve_http_production, kwargs=keywords)
self._inference_endpoint_process.start()

And End it the following way:

Copy code

self._inference_endpoint_process.terminate()
self._inference_endpoint_process.join()
self._inference_endpoint_process.close()

However, the server does not stop. Any ideas?

Open in Slack

Previous Next