Hello team
I’m currently aiming to keep all segments generated during batch ingestion close to 256MB in size. To achieve this, I’ve implemented a logic that sets a maximum document count per segment, which I adjust dynamically based on the characteristics of the data, so that the segment size stays approximately within the target.
I’m wondering if there’s a more efficient or standardized approach to achieve this?