This message was deleted.

If it helps, bringing max_batch_size to 2 and rolling back to a single image per call seems to work. On a separate note, seems like there's some sort of signature mismatch between local and EC2 for the inpaint method of the runner. Locally runs just fine but on EC2 throws a missing parameter exception:

2022-10-23T23:12:09+0000 [INFO] [api_server:4] 181.105.138.37:61867 (scheme=http,method=POST,path=/inpaint4,type=multipart/form-data; boundary=----WebKitFormBoundaryidXgqeypKBA2XUB2,length=112314) (status=500,type=application/json,length=2) 500.698ms (trace=f3e988573ec52e14868426354802efa7,span=7a9dbe969889f80d,sampled=0)
2022-10-23T23:15:17+0000 [ERROR] [runner:stable_diffusion_runner:1] Exception on runner 'stable_diffusion_runner' method 'inpaint' (trace=0e19c31e6191dff82d8a34b825ae2b51,span=cd382b4e328ab896,sampled=0)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/server/runner_app.py", line 309, in _run
    ret = await runner_method.async_run(*params.args, **params.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner.py", line 51, in async_run
    return await self.runner._runner_handle.async_run_method(  # type: ignore
  File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/local.py", line 57, in async_run_method
    return await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runnable.py", line 139, in method
    return self.func(obj, *args, **kwargs)
  File "/home/bentoml/bento/src/service.py", line 134, in inpaint
    images = self.inpaint_pipe(
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
TypeError: __call__() missing 1 required positional argument: 'image'
2022-10-23T23:15:17+0000 [INFO] [runner:stable_diffusion_runner:1] _ (scheme=http,method=POST,path=/inpaint,type=application/octet-stream,length=1562724) (status=500,type=text/plain,length=0) 6.018ms (trace=0e19c31e6191dff82d8a34b825ae2b51,span=cd382b4e328ab896,sampled=0)
2022-10-23T23:15:17+0000 [ERROR] [api_server:2] Exception on /inpaint [POST] (trace=0e19c31e6191dff82d8a34b825ae2b51,span=9b1c06a271e970ca,sampled=0)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/server/http_app.py", line 317, in api_func
    output: t.Any = await run_in_threadpool(api.func, **input_data)
  File "/usr/local/lib/python3.8/dist-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/home/bentoml/bento/src/service.py", line 235, in inpaint
    images = stable_diffusion_runner.inpaint.run(img, mask, data)
  File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner.py", line 44, in run
    return self.runner._runner_handle.run_method(  # type: ignore
  File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 211, in run_method
    anyio.from_thread.run(
  File "/usr/local/lib/python3.8/dist-packages/anyio/from_thread.py", line 49, in run
    return asynclib.run_async_from_thread(func, *args)
  File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 970, in run_async_from_thread
    return f.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 163, in async_run_method
    raise RemoteException(
bentoml.exceptions.RemoteException: An exception occurred in remote runner stable_diffusion_runner: [500] 
2022-10-23T23:15:17+0000 [INFO] [api_server:2] 181.105.138.37:62623 (scheme=http,method=POST,path=/inpaint,type=multipart/form-data; boundary=----WebKitFormBoundary4eWLLlcWFySBgpuT,length=112314) (status=500,type=application/json,length=2) 614.259ms (trace=0e19c31e6191dff82d8a34b825ae2b51,span=9b1c06a271e970ca,sampled=0)

@Ezequiel Wajs hmm interesting on the signature mismatch. Let me try again with the inpaint endpoint and get back to you

Of course, thanks a lot. For reference, it works locally so I'm guessing the docker build for the Bento is likely using different versions of some libraries. I found it a bit odd to have to specify the python requirements for the Bento in the YAML file instead of a requirements.txt file. I understand needing to separate dev dependencies from prod dependencies but maybe having a requirements.txt and requirements.dev.txt and feeding the first to the yaml with pinned package versions might help.