김도민
03/22/2024, 8:41 AM@bentoml.service()
class A:
@bentoml.api
def stream(self) -> typing.Generator(bytes, None, None):
for i in range(10):
arr = np.empty((10, 10, 3), dtype=np.uint8).tobytes()
yield arr
@bentoml.service()
class B:
a: A = bentoml.depends(A)
@bentoml.api
def stream(self) -> typing.Generator(bytes, None, None):
response = self.a.stream()
for res in response:
yield res
When I request to bentoml API using python requests.
response = <http://requests.post|requests.post>(f'<http://localhost:3000/stream>', data=data, stream=True)
for r in response.iter_content(chunk_size=None):
print(f"recieved: {len(r)}")
I just get a error
("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))
So when I debugged the code.
response' s type is generator. the value is <generator object SyncHTTPClient._get_stream at 0x7f4a923b8a50>
The above results were obtained.
When I searched more, I found that class A is a remote proxy object, and it is calling the api inside.
There is a boolean value called stream_output in the ClientEndPoint used at this time, but it seems that the default is False, so it is not handling the stream request well.
I would appreciate it if you could let me know if I'm missing something or if I'm mistaken.