I have a question about the kafka-setup container....
# getting-started
b
I have a question about the kafka-setup container. It looks like it is used to precreate topics and the default value is set to false in the docker-compose for the quickstart. In what circumstances would we want to set it to true and have it precreate topics?
r
In production environments you may want to provision topics with partitions which are greater or lower then the default values given. Additionally for the max message size you might want to specify different values then the kafka cluster's defaults. In these cases you want to precreate the topics matching the desired settings/configuration rather than rely on default values.
b
Ok, thanks for this info! I also ran across these comments in one of the docker-compose files saying that kafka-setup was just for backwards compatibility purposes. We were going to remove it, but maybe we'll keep it around in case there's a need for it later.
r
The comment is with respect to the docker compose files used with quickstart. It is unlikely a locally running instance would need the more advanced configuration options for kafka. The intent is to remove kafka-setup for quickstart but for production deployments it is still a very important part of running datahub at scale.
👍 1