This message was deleted.
# ask-for-help
s
This message was deleted.
n
I confirmed that the bytes received are the same between versions, so it must be something to do with how PIL is opening the image?
One more observation - this happens when the image is passed onto the runner, the service receives the same data across both versions
c
What are the differences you are observing?
n
The file size is vastly different (922KB as received by service, 386KB as received by runner)
c
I think @Frost Ming implemented some optimization around the image data serialization between runner IPCs
So the file size change is to be expected
It shouldn’t affect the performance or behavior
n
Well it will given that the image is not the same as the input? In our use case it is critical to match inference with what was obtained during validation and testing (healthcare domain). If the image is modified before reaching our model, we will never be able to do that?
I understand why optimization might be desirable but there should be a way to disable them?
I reverted to 1.0.22 (before the change above) and I can confirm it now works as expected again
I am not sure this should be on by default? As a user I would expect my input not to be modified before being fed to my model?
c
the input should be identical
n
It's not, at least something between
1.0.22
and the current version is modifying my input between the service and runner
c
@Frost Ming any insights?
n
(I don't mean this to sound too negative, I love BentoML and just want to help make it better 🙂 )
c
No worries Nicolas, we love feedbacks!
f
can you see how it is modified? random bytes changed, truncated or it can rebuild a valid image object?
n
@Frost Ming It is a valid image - basically I have tested at various places to see where the content gets modified. From the initial reception of the bytes in the internal BentoML code to my service and runner. The service gets the expected input, the runner gets a vastly different, smaller input. The input received by the runner is a valid JPEG object, I can save it (which is how I debugged this). It is the same image, just vastly smaller in size. And because it is a JPEG, this means that the content is also different. I initially detected this because our integration tests, which test whether the inference results are the same than those obtained during validation of the model, showed significant discrepancies with the new version whereas
1.0.5
produced the exact same outputs.
And I just confirmed that
1.0.22
also produces the correct results.
I can test with other versions as well if useful?
Also happy to test if there is a way to toggle optimizations off
f
Tested locally, the image size doesn’t change between server and runner, just a slight difference on the binary size(only observed on JPEG) which doesn’t matter much. Can you provide the image file that doesn’t work properly on your side?
Here is the test code:
Copy code
from io import BytesIO

import bentoml
from <http://bentoml.io|bentoml.io> import JSON, Image
from PIL.Image import Image as PILImage


class MyRunnable(bentoml.Runnable):
    SUPPORTED_RESOURCES = ("cpu",)
    SUPPORTS_CPU_MULTI_THREADING = False

    @bentoml.Runnable.method()
    def predict(self, image: PILImage):
        buffer = BytesIO()
        image.save(buffer, format=image.format)
        return {
            "size": image.size,
            "format": image.format,
            "length": len(buffer.getvalue()),
        }


runner = bentoml.Runner(MyRunnable, name="runner")


svc = bentoml.Service("test_runner", runners=[runner])


@svc.api(input=Image(), output=JSON())
async def classify(input: PILImage) -> dict:
    buffer = BytesIO()
    input.save(buffer, format=input.format)
    runner_result = await runner.predict.async_run(input)
    return {
        "input_size": input.size,
        "input_format": input.format,
        "input_length": len(buffer.getvalue()),
        "runner_result": runner_result,
    }
n
@Frost Ming I will try and provide this information today. A couple notes: • You mention that a change in binary size doesn't matter but it absolutely does - given the lossy nature of many formats, including JPEGs, changes in binary size would likely be linked to changes in data, correct? I wouldn't expect the framework to alter my input data at all - small imperceptible changes can lead to vastly different results with modern ML architectures • I don't think the size of the array would differ - can you compare the actual content? If data isn't altered then data arrays at the server and runner levels should be identical, correct?
f
@Nicolas Jaccard hi, not hearing from you since a while. Do you get the image file to reproduce?
n
@Frost Ming I haven't been able to create a sample project just yet - we are still experiencing the issue with newer versions of BentoML though