Hi. I'm new to Apache pinot.
I have a use case whe...
# troubleshootingm
Mohamed Sultan
03/25/2021, 7:27 AMHi. I'm new to Apache pinot.
I have a use case where I need to ingest data through kafka (externally with SSL certs configured) to pinot which is installed in kubernetes environment. How can i connect exernal kafka with SSL certs configured in pinot on kubernetes environment?
It would be great if you forward me to the respective documentation.
k
Kishore G
03/25/2021, 7:35 AM
https://docs.pinot.apache.org/basics/data-import/pinot-stream-ingestion/import-from-apache-kafka#add-sample-data-to-the-kafka-topic
Use Kafka Partition(Low) Level Consumer with SSL
m
Mohamed Sultan
03/25/2021, 7:45 AMHi @Kishore G Thanks for sharing.
can you please point me where should i supposed to add the JSON script in which the config will reflect in pinot.
t
Tanmay Movva
03/25/2021, 8:22 AMHow can i connect exernal kafka with SSL certs configured in pinot on kubernetes environment?You can reference Environment variables in table config. That should be enough to connect to SSL.
Tanmay Movva
03/25/2021, 8:23 AMUntitled
Tanmay Movva
03/25/2021, 8:24 AM^This is how our streamConfigs for kafka look like.
The secrets and the truststore, keystore location generation are handled using the InitScript of the sts.
Tanmay Movva
03/25/2021, 8:25 AMMore on how to use Env vars - https://docs.pinot.apache.org/configuration-reference/table#environment-variables-override
m
Mohamed Sultan
03/25/2021, 8:38 AM@Tanmay Movva so we can add streamconfigs JSON scripts in OFFLINE table, right?
Mohamed Sultan
03/25/2021, 8:54 AMThere are some tables already created, I have checked the table configs, there i see streamConfigs, if I change that, the ingestion from the external kafka? on the particular table. am i right?
Mohamed Sultan
03/25/2021, 1:22 PM@Tanmay Movva trust store location and key store location should be the mounted path in pinot-zookeeper pods right? since i'm using kubernetes environment.
Please confirm this.
t
Tanmay Movva
03/25/2021, 1:24 PMIn the pinot servers, controllers and brokers(might not be required).
m
Mohamed Sultan
03/25/2021, 1:25 PMI just want to ask where to mount the keystore and truststore
Mohamed Sultan
03/25/2021, 1:26 PMsince we need to mention in table config
2 Views