Hi all, We have a BentoML service with an endpoint that submits a long-running Bento task, along with an async function that periodically updates a database. The issue we're running into is that when Bento scales down, the task continues running (as seen in the logs), but we get the below repeating error and the database stops receiving updates — leading to inconsistent behavior. We found that increasing the

scale_down_stabilization_window

to longer than the task’s max duration (~1920s) seems to prevent the issue. However, setting this parameter under the scaling policy in a YAML or JSON file appears to be ignored when deploying via the CLI — it always defaults to 600s. We can change it manually through the UI, but we need this setting to be configured automatically as part of our CI/CD pipeline. Here's the error that occurs when bento scales down in the middle of a task:

2025-05-22T21:06:52Z [Service: Algorithm][Replica: 92cdv]
[ERROR] [cli] Exception in callback <bound method Arbiter.manage_watchers of <bentoml._internal.utils.circus.Arbiter object at 0x7f0cbdfbcbd0>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/tornado/ioloop.py", line 945, in _run
    val = self.callback()
  File "/usr/local/lib/python3.11/site-packages/circus/util.py", line 1038, in wrapper
    raise ConflictError("arbiter is already running arbiter_stop command")
circus.exc.ConflictError: arbiter is already running arbiter_stop command

We’re on BentoML version 1.3.14. Is this a known issue? Is there a workaround to ensure

scale_down_stabilization_window

is applied automatically via the CLI? Thanks!

Hello! i replied you in your email! this has been fixed in the latest BentoML version