This message was deleted BentoML #ask-for-help

Join Slack

This message was deleted.

# ask-for-help

Slackbot

11/04/2022, 1:20 PM

This message was deleted.

Susana Bouchardet

11/04/2022, 1:23 PM

I jus get the BENTOML_CONFIG content from an production pod

Copy code

bentoml@rec-model-sknn-b6b9fdf9b-hkq9p:~/bento$ cat $BENTOML_CONFIG 
api_server:
  workers: 1
  timeout: 3
  logging:
    access:
      enabled: False
runners:
  logging:
    access:
      enabled: False

Susana Bouchardet

11/04/2022, 2:55 PM

since my app doesn't receive too many requests, I want to decrease the amount of API workers, and decrease the resources of it. My model is tiny, so, my idea was to parallelize it between multiple tiny pods. those poses don't need to have more than one worker.

Tim Liu

11/04/2022, 2:58 PM

Are you setting the BENTOML_CONFIG environment parameters? https://docs.bentoml.org/en/latest/guides/configuration.html

Tim Liu

11/04/2022, 2:58 PM

is this in the container or serving from the command line?

Susana Bouchardet

11/04/2022, 2:59 PM

The container is from the containerize command.

Tim Liu

11/04/2022, 3:00 PM

Are you running the container locally?

Susana Bouchardet

11/04/2022, 3:00 PM

no, in a gke pod.

Tim Liu

11/04/2022, 3:00 PM

maybe try it real quick like this to confirm if it's a bug? https://docs.bentoml.org/en/latest/guides/configuration.html#docker-deployment

Tim Liu

11/04/2022, 3:01 PM

Copy code

docker run -v /local/path/configuration.yml:/home/bentoml/configuration.yml -e BENTOML_CONFIG=/home/bentoml/configuration.yml

Susana Bouchardet

11/04/2022, 3:01 PM

I will try it. I'll come back with the results asap

Tim Liu

11/04/2022, 3:02 PM

if that doesn't work then it would seem like it's a bug

Tim Liu

11/04/2022, 3:02 PM

if it does work, then we might need to see how you're passing the configuration in production

Susana Bouchardet

11/04/2022, 3:05 PM

I'm passing it as a configmap. Like this

Copy code

12:02 $ kubectlr get configmap bento-server-config -o yaml
apiVersion: v1
data:
  configuration.yaml: |
    api_server:
      workers: 1
      timeout: 1
      logging:
        access:
          enabled: False
    runners:
      logging:
        access:
          enabled: False

And using it in the pod like this (here some parts of the pod yaml). The env var definition:

Copy code

spec:
      containers:
      - name: rec-model-sknn
        image: $IMAGE_TAG
        env:
          - name: BENTOML_CONFIG
            value: /var/config/configuration.yaml

The config map usage:

Copy code

volumeMounts:
        - name: bento-config
          mountPath: /var/config
          readOnly: true
      volumes:
      - name: bento-config
        configMap:
          name: bento-server-config

Susana Bouchardet

11/04/2022, 3:06 PM

but i will run the test here

Susana Bouchardet

11/04/2022, 3:06 PM

btw, I'm using

bentoml = "1.0.3"

👍 1

Susana Bouchardet

11/04/2022, 3:33 PM

@Tim Liu, i just run the command here, and i still get more workers than i requested

Copy code

✔ ~/Workspace/rec-sknn-pipeline [main|…4] 
12:31 $ cat $PWD/config.yaml
api_server:
  workers: 1
  timeout: 3
  logging:
    access:
      enabled: False
runners:
  logging:
    access:
      enabled: False

✔ ~/Workspace/rec-sknn-pipeline [main|…4] 
12:32 $ docker run -v $PWD/config.yaml:/home/bentoml/configuration.yml -e BENTOML_CONFIG=/home/bentoml/configuration.yml -p 3000:3000 local-test
2022-11-04T15:32:13+0000 [INFO] [cli] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:13+0000 [INFO] [cli] Starting production BentoServer from "/home/bentoml/bento" running on <http://0.0.0.0:3000> (Press CTRL+C to quit)
2022-11-04T15:32:15+0000 [INFO] [runner:sknn:1] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:15+0000 [INFO] [api_server:4] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:15+0000 [INFO] [api_server:1] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:15+0000 [INFO] [runner:sknn:1] Setting up worker: set CPU thread count to 8
2022-11-04T15:32:15+0000 [INFO] [api_server:3] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:15+0000 [INFO] [api_server:8] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:15+0000 [INFO] [api_server:6] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:15+0000 [INFO] [api_server:5] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:15+0000 [INFO] [api_server:2] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")
2022-11-04T15:32:15+0000 [INFO] [api_server:7] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:hzrexns4ko44ykkb", path="/home/bentoml/bento/")

Tim Liu

11/04/2022, 3:41 PM

That's weird… I just tested it on a raw bentoml serve command and it worked… wonder if it's a containerization issue.

Tim Liu

11/04/2022, 3:43 PM

Could you try to serve the bento locally with the configuration file? like:

Copy code

export BENTOML_CONFIG=config.yaml
bentoml serve --production --verbose

Susana Bouchardet

11/04/2022, 3:48 PM

Same issue (but with a huge log 😂 )

log.txt

Tim Liu

11/04/2022, 3:50 PM

Oh… good actually…. I'm not 100% because I don't remember us fixing any configuration issues, but can you upgrade to the latest version on your local to rule out differences? Also could you try to add a —api-workers 1 To the serve command and try that way? Make sure something really weird isn't going on

Susana Bouchardet

11/04/2022, 4:07 PM

i will have a break right now. But i will test it as soon as i be back

Susana Bouchardet

11/04/2022, 4:45 PM

looks like update the bentoml version fixed the problem. 🙏 Look the new log

Copy code

✔ ~/Workspace/rec-sknn-pipeline [main|✚ 2…5] 
13:43 $ docker run -v $PWD/config.yaml:/home/bentoml/configuration.yml -e BENTOML_CONFIG=/home/bentoml/configuration.yml -p 3000:3000 local-test
2022-11-04T16:43:41+0000 [INFO] [cli] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:266p73s4l244ykkb", path="/home/bentoml/bento/")
2022-11-04T16:43:41+0000 [INFO] [cli] Environ for worker 0: set CPU thread count to 8
2022-11-04T16:43:41+0000 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "/home/bentoml/bento" can be accessed at <http://localhost:3000/metrics>.
2022-11-04T16:43:42+0000 [INFO] [cli] Starting production HTTP BentoServer from "/home/bentoml/bento" listening on <http://0.0.0.0:3000> (Press CTRL+C to quit)
2022-11-04T16:43:43+0000 [INFO] [runner:sknn:1] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:266p73s4l244ykkb", path="/home/bentoml/bento/")
2022-11-04T16:43:43+0000 [INFO] [api_server:1] Service loaded from Bento directory: bentoml.Service(tag="sknn_svc:266p73s4l244ykkb", path="/home/bentoml/bento/")

It isn't spawning other workers. only the

api_server:1

Susana Bouchardet

11/04/2022, 4:46 PM

i will get the verbose version too

Susana Bouchardet

11/04/2022, 4:48 PM

I will update the bentoml version in the production environment to see if it works too.

👍 1

Susana Bouchardet

11/04/2022, 5:09 PM

Now it's working 🙌 Thanks @Tim Liu for the help. Now I've learned to keep the newest version if it's possible 😂

😅 1

Open in Slack

Previous Next