Good morning. I have question regarding Kafka stuff for Datahub. Currently I don't have opportunity to use Kafka on AWS (msk) or GCP, I can use only remote Kafka on Aiven and there I have difficulties with authentication. Aiven represent authentication thru certs but I didn't find any ways for use this type authentication for Datahub deployment. ( https://github.com/acryldata/datahub-helm ) Question, how I can avoid this issue, is it possible to pass authentication creds to Datahub ???

Depending on your kafka environment, you would need to create and add various cert files to the containers using extraVolumes extraVolumeMounts and configuration the various Spring related environment variables to point to the file locations mounted in the container. An alternative to adding the files to k8 and mounting them would be to create custom docker images which include the required files and using the helm chart to point to your custom images.

Thank you. Will test it

The SSL protocol is not PEM, take a look at the spring kafka documentation. The values are documented here, the corresponding environment variables are essentially uppercase and replacing the

.

with

_

, then for kafka documentation this has details. Notice that for example the SSL_PROTOCOL is the network protocol, so SSL or if you happen to be using SASL. Since the schema registry is provided by the service you are using for kafka (or not used), you do not have to run the schema registry component in k8. The latter configuration is if you are running it yourself in k8. Also you will need to create the java key store files, you will need to create the jks files. The second link as information for configuring kafka and links to information on creating those jks files.