https://pinot.apache.org/ logo
Join Slack
Powered by
Hi…When trying to `Upload the schema and Table Con...
# troubleshooting
a
Hi…When trying to 
Upload the schema and Table Config
- https://docs.pinot.apache.org/basics/getting-started/pushing-your-streaming-data-to-pinot#uploading-your-schema-and-table-config I am getting the following error…
Copy code
Sending request: <http://pinot-quickstart:9000/schemas> to controller: ea8d7bfc16ea, version: Unknown
{"code":500,"error":"org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata"}
However I am able to login to my kafka docker and describe and that works fine…
Copy code
bash-4.4# bin/kafka-topics.sh --bootstrap-server kafka:9092 --topic transcript-topic --describe
Topic: transcript-topic	PartitionCount: 1	ReplicationFactor: 1	Configs: segment.bytes=1073741824
	Topic: transcript-topic	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
Any idea where i am missing?
d
Tumeout expiered while fetching topic metadata
is typically a sign that pinot was able to connect to your kafka instance but failing to load the topic. Double check the topic name
potentially the kafka hostname might also be the problem
a
hmm….When i try to describe the topic specifying the hostname it works…
Copy code
incubator-pinot (master) ✗ docker exec \
  -t kafka \
  /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server kafka:9092 \
  --topic transcript-topic --describe
Topic: transcript-topic	PartitionCount: 1	ReplicationFactor: 1	Configs: segment.bytes=1073741824
	Topic: transcript-topic	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
Doesn’t this mean its already able to recognize the host.
Pinot UI is also able to list out the topics….
Apologies for some newbie questions here….. In the Streaming Example - https://docs.pinot.apache.org/basics/getting-started/running-pinot-in-docker#streaming-example I see that it says it starts the Pinot deployment by starting 
Zookeeper
, `Kafka`…. But, i don’t see any other docker containers started other than 
pinot-quickstart
….
k
quickstart starts everything in one process
x
you can follow this doc for running all components in different docker container: https://docs.pinot.apache.org/basics/getting-started/advanced-pinot-setup
a
Awesome this is what i was looking for. Thanks! ☝️
Where should the 
schemaFile
and the 
tableconfigFile
be placed? Tried it with placing it in local as well as in controller..didn’t work..
Copy code
docker run \
    --network=pinot-demo \
    --name pinot-streaming-table-creation \
    0e536a319df3 AddTable \
    -schemaFile /tmp/pinot-quick-start/transcript-schema.json \
    -tableConfigFile /tmp/pinot-quick-start/transcript-table-realtime.json \
    -controllerHost pinot-controller \
    -controllerPort 9000 \
    -exec
Steps until here ☝️: • As of now running all the components in different docker containers…. • Created a kafka topic • Facing 
FileNotFoundException
while creating table and schema
Tried it with placing it in local as well as in controller..didn’t work..
x
You need to use -v to mount your local disk to docker container
Then you can use those files in the docker container
a
Thanks now its able to pickup the schemaFile, but I am back to the old issue of 
Timeout expired while fetching topic metadata
when trying with the docker command. So, i tried to execute 
pinot-admin.sh
within the Controller Container… But, I am getting an exception 
Schema is incompatible with tableConfig
Copy code
2021/05/07 12:36:02.787 ERROR [PinotAdministrator] [main] Exception caught:
org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 400 (Bad Request) with reason: "Cannot add invalid schema: transcript. Reason: Schema is incompatible with tableConfig with name: transcript_REALTIME and type: REALTIME" while sending request: <http://172.18.0.3:9000/schemas> to controller: 31edf0f41322, version: Unknown
I am using the 
Schema
and 
Table Config
from this page - https://docs.pinot.apache.org/basics/data-import/pinot-stream-ingestion Kindly advice!
I do see the schema is created thru the UI but the table is not created…
Found out the issue…. By just running the command
Copy code
bin/pinot-admin.sh AddTable     -tableConfigFile /tmp/pinot-quick-start/transcript-table.json     -exec
as the schema was created earlier it pointed out the issue in the 
timestamp
field. So Fixed it in the table creation to have the same name as schema
Copy code
"segmentsConfig": {
    "timeColumnName": "timestampInEpoch",
Pushed a documentation PR see if this makes sense - https://github.com/pinot-contrib/pinot-docs/pull/29
m
Thanks a lot @Arun Vasudevan for debugging the issue almost on your own and contributing back to the community. We thrive because of members like you, much appreciated.
🎉 1
Open in Slack
PreviousNext
Powered by