Hi, can we customise segment creation on the basis of incoming data? So right now I have different partitions for different brand Ids but the number of partitions have gone to 1000 in a topic and will keep increasing and I think there is an upper cap in kafka for partitions per broker. Instead of having a partition and segment one to one mapping, if I can configure somehow to send data to different segments based on brand id then it would be great

@Sonit Rathi 1000 partitions already ? That is too large. You can reduce the number of partitions upstream before ingesting ( partition function modulo K) Seek to 2:15 here: https://developer.confluent.io/learn-kafka/apache-kafka/partitions/