Hi, hoping someone can help with this - deploying to ComfyService and getting an exception on /generate [POST] Works when served locally, comfy-pack worked as expected, actually deploying the bento works - just hits this wall when we call it via the playground.

2025-02-25T02:44:28Z
[Service: ComfyService][Replica: tztj7]
Starting server
2025-02-25T02:44:28Z
[Service: ComfyService][Replica: tztj7]
To see the GUI go to: <http://127.0.0.1:8188>
2025-02-25T02:44:30Z
[Service: ComfyService][Replica: tztj7]
2025-02-25T02:44:30+0000 [INFO] [entry_service:ComfyService:1] Successfully started ComfyUI in the background
2025-02-25T02:44:30Z
[Service: ComfyService][Replica: tztj7]
2025-02-25T02:44:30+0000 [INFO] [entry_service:ComfyService:1] ComfyUI Server started at localhost:8188
2025-02-25T02:44:30Z
[Service: ComfyService][Replica: tztj7]
2025-02-25T02:44:30+0000 [INFO] [entry_service:ComfyService:1] Watch thread started
2025-02-25T02:44:30Z
[Service: ComfyService][Replica: tztj7]
2025-02-25T02:44:30+0000 [INFO] [entry_service:ComfyService:1] Service ComfyService initialized
2025-02-25T02:44:30Z
[Service: ComfyService][Replica: tztj7]
2025-02-25T02:44:30+0000 [ERROR] [entry_service:ComfyService:1] Exception on /generate [POST] (trace=88c136b002e44a9d153d0763303135be,span=d9daac0255ced1ee,sampled=0,service.name=ComfyService)
2025-02-25T02:44:30Z
[Service: ComfyService][Replica: tztj7]
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/_bentoml_impl/server/app.py", line 599, in api_endpoint_wrapper
    resp = await self.api_endpoint(name, request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/_bentoml_impl/server/app.py", line 695, in api_endpoint
    output = await self._to_thread(func, *input_args, **input_params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/_bentoml_impl/server/app.py", line 545, in _to_thread
    output = await anyio.to_thread.run_sync(func, limiter=self._limiter)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2461, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 962, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/_bentoml_sdk/method.py", line 169, in wrapped
    return self.func(instance, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-02-25T02:44:30Z
[Service: ComfyService][Replica: tztj7]
TypeError: ComfyService.generate() missing 1 required keyword-only argument: 'ctx'

Hi @Joseph Scully! This has to do with a bug introduced in BentoML 1.4, a fix is coming out soon and downgrading to 1.3 version also works: pip install bentoml==1.3.22

Thankyou - downgrading worked to resolve this issue. Sadly the workflow is now producing an error which suggests some of the custom-nodes are missing? Is this even possible if it's working locally, and comfy-pack has added all of the custom nodes? Or are there some custom-nodes that are known not to package correctly? I could maybe refactor the workflow if so.

2025-02-25T07:45:07Z
[Service: ComfyService][Replica: 9qwzq]
invalid prompt: {'type': 'invalid_prompt', 'message': 'Cannot execute because node Evaluate Integers does not exist.', 'details': "Node ID '#209'", 'extra_info': {}}

Hi, BentoML v1.4.1 has been released to address this issue. It should work without having to pin the version

Awesome, thankyou - the support on here is excellent. Does this new versions also resolve the aforementioned missing custom_node issue? Thanks.

for the missing node issue, can you send me the deployment url?

no, i mean the link to the deployment page, not the endpoint url

apologies https://revere.cloud.bentoml.com/deployments/oil-1-4l71/playground

The custom node “efficiency-nodes-comfyui”, which provides the missing node is not collected by the comfy-pack. Does it exist in your local workspace? how did you install it

yeah - all the custom nodes I have are installed using comfy-manager. I tried removing the offending ones and reinstalling a them to no effect. I checked each of them for a .git directory and some were missing them (even those I had just reinstalled via comfy-manager). Installing each custom_node directly from git has allowed me to run the workflow to a point - I can see the logs loading the custom_nodes. Unfortunately I am now getting 504 Gateway Timeout. I can see the workflow seems to be running partially, see the progress of Ksampler and various other stages, but it stops on this 504 error. Is there a maximum runtime for a request on bentocloud?

how long does it take to return 504 and how long should the workflow run approximately did you enable the request queue? if not try enabling it and try again

seems like the in-process request lives for around 3.5-4.0 mins based on the monitoring output. adding the request queue doesn't seem to have made any difference.