Hi When trying to `Upload the schema and Table Config`

Question

Hi When trying to `Upload the schema and Table Config` I am getting the following error ```Sending request to controller ea8d7bfc16ea version Unknown code 500 error org apache kafka common errors TimeoutException Timeout expired while fetching topic metadata ``` However I am able to login to my kafka docker and describe and that works fine ```bash 4 4 bin kafka topics sh bootstrap server kafka 9092 topic transcript topic describe Topic transcript topic PartitionCount 1 ReplicationFactor 1 Configs segment bytes=1073741824 Topic transcript topic Partition 0 Leader 0 Replicas 0 Isr 0``` Any idea where i am missing

Daniel Lavoie · Answer

`Tumeout expiered while fetching topic metadata` is typically a sign that pinot was able to connect to your kafka instance but failing to load the topic Double check the topic name

Daniel Lavoie · Answer

potentially the kafka hostname might also be the problem

Arun Vasudevan · Answer

hmm When i try to describe the topic specifying the hostname it works ```incubator pinot master docker exec t kafka opt kafka bin kafka topics sh bootstrap server kafka 9092 topic transcript topic describe Topic transcript topic PartitionCount 1 ReplicationFactor 1 Configs segment bytes=1073741824 Topic transcript topic Partition 0 Leader 0 Replicas 0 Isr 0``` Doesn t this mean its already able to recognize the host

Arun Vasudevan · Answer

Pinot UI is also able to list out the topics

Arun Vasudevan · Answer

Apologies for some newbie questions here In the Streaming Example <https docs pinot apache org basics getting started running pinot in docker streaming example> I see that it says it starts the Pinot deployment by starting `Zookeeper` `Kafka` But i don t see any other docker containers started other than `pinot quickstart`

Kishore G · Answer

quickstart starts everything in one process

Xiang Fu · Answer

you can follow this doc for running all components in different docker container <https docs pinot apache org basics getting started advanced pinot setup>

Arun Vasudevan · Answer

Awesome this is what i was looking for Thanks point up

Arun Vasudevan · Answer

Where should the `schemaFile` and the `tableconfigFile` be placed Tried it with placing it in local as well as in controller didn t work ```docker run network=pinot demo name pinot streaming table creation 0e536a319df3 AddTable schemaFile tmp pinot quick start transcript schema json tableConfigFile tmp pinot quick start transcript table realtime json controllerHost pinot controller controllerPort 9000 exec```

Arun Vasudevan · Answer

Steps until here point up As of now running all the components in different docker containers Created a kafka topic Facing `FileNotFoundException` while creating table and schema

Arun Vasudevan · Answer

Tried it with placing it in local as well as in controller didn t work

Xiang Fu · Answer

You need to use v to mount your local disk to docker container

Xiang Fu · Answer

Then you can use those files in the docker container

Arun Vasudevan · Answer

Thanks now its able to pickup the schemaFile but I am back to the old issue of `Timeout expired while fetching topic metadata` when trying with the docker command So i tried to execute `pinot admin sh` within the Controller Container But I am getting an exception `Schema is incompatible with tableConfig` ```2021 05 07 12 36 02 787 ERROR PinotAdministrator main Exception caught org apache pinot common exception HttpErrorStatusException Got error status code 400 Bad Request with reason Cannot add invalid schema transcript Reason Schema is incompatible with tableConfig with name transcript REALTIME and type REALTIME while sending request to controller 31edf0f41322 version Unknown``` I am using the `Schema` and `Table Config` from this page Kindly advice

Arun Vasudevan · Answer

Is there a documentation of the list of Validations done in the schema <https github com apache incubator pinot blob master pinot segment local src main java org apache pinot segment local utils TableConfigUtils java L94|https github com apache incubator pinot blob master pinot segment local src main java org apache pinot segment local utils TableConfigUtils java>

Arun Vasudevan · Answer

I do see the schema is created thru the UI but the table is not created

Arun Vasudevan · Answer

Found out the issue By just running the command ```bin pinot admin sh AddTable tableConfigFile tmp pinot quick start transcript table json exec``` as the schema was created earlier it pointed out the issue in the `timestamp` field So Fixed it in the table creation to have the same name as schema ``` segmentsConfig timeColumnName timestampInEpoch ```

Arun Vasudevan · Answer

Pushed a documentation PR see if this makes sense <https github com pinot contrib pinot docs pull 29>

Mayank · Answer

Thanks a lot < Arun Vasudevan> for debugging the issue almost on your own and contributing back to the community We thrive because of members like you much appreciated