<@U0491B8FXAP> well with this increase you are all...
# troubleshooting
d
@Jeremy DeGroot well with this increase you are allowing each fetch request from consumers to grab a larger chunk. This can positively improve performance in several ways: reduced network overhead and latency. fewer connections to negotiate basically. better utilization of buffers which can be underutilized with smaller chunks. There is also potentially reduced back pressure propagation as well. Your effectively creating more breathing room in your system.
j
What's considered a "small chunk" from buffer standpoints?
d
iin practical applications it can be from a few kilobtyes to a few megabytes. A starting point for tuning might be 5-10mb
its going to be closer to the default minimum fetch size in kafka. So its sort of proportional to Kafka sizes.
You also need to consider if you are aiming to maximize throughput or latency
gratitude thank you 1