Is possible to use the Adaptive batching with websockets?
c
Chaoyu
05/28/2024, 1:52 AM
Yes it is possible- you just need to call a depend service that does batching, from a websocket callback function
a
Anderson Karl
05/28/2024, 11:58 AM
Thank you for the answer!
I think that would be nice to have more examples using websockets with bentoml.
Just clarifying my use case for websockets, i have a custom STT model and the audio data was send as a stream so we can return interim results.
Anderson Karl
05/28/2024, 12:00 PM
By now I'm thinking about using http endpoints because they are simpler but I'm planning on changing to websockets in the near future.