Hello everyone!
I'm writing an article for medium of a real case implemantation of Apache Pinot as a way of studying. For that, I'm using a Strimzi cluster and the Twitter Api Kafka Connector (that you can find at https://www.confluent.io/hub/jcustenborder/kafka-connect-twitter) running on Minikube, to get data from twitter's api and ingest it into Pinot.
I followed the steps explained in this video
https://www.youtube.com/watch?v=Jc03u8rXc2w▾
making some adaptations to run on kubernetes. That way I was able to infer the schema of the "twitter-sample.json" file attached to this message by generating the schema file "twitter-old-schema.json", after that I had to remove some fields:
"schema.type", "schema.fields", "schema.optional", "schema.name",
and remove the prefix "payload." of every column to generate the file "twitter-schema.json". Then with this schema file and with the table config file "twitter-config.json" I created the REALTIME table "twitter-status-events" (using the column "CreatedAt" as datetime column) using pinot-admin.sh inside pinot controller's pod. But for some reason that I don't know I'm not getting any record in this table, do you have any idea what I'm doing wrong ?
(more information replied to this comment)