< Radhika> < Sadim Nadeem> Please check the controller and s Apache Pinot #troubleshooting

<@U0278E1KLVA> <@U0224CF6LMT> Please check the con...

Jackie

07/05/2021, 4:40 PM

@Radhika @Sadim Nadeem Please check the controller and server log and see if there is any error logged? Even if the partitioning is wrong, you should still be able to see data consumed, so that should not be the cause

Sadim Nadeem

07/06/2021, 12:40 PM

no errors in logs..only warnings which @Radhika shared before.. let me reshare it

upsertlogs.txt

Jackie

07/06/2021, 6:13 PM

This should not be related. I didn't see the

stream.kafka.zk.broker.url

config in your table config, which might be the cause. By default it should be

localhost:2191/kafka

Radhika

07/07/2021, 9:30 AM

These are logs and 2 table schema where we added the url with "\kafka" and without "\kafka".

table1 table upsertlogs.txt

Sadim Nadeem

07/07/2021, 9:36 AM

still it didnt worked @Jackie . need some help .. please have a look .. cc: @Mohamed Hussain @Mayank @Xiang Fu @Kishore G

Sadim Nadeem

07/07/2021, 9:39 AM

for few seconds .. the doc count showed on controller UI as 1 but data was not showing and after few second ..it came back to zero .. even though if I run the command for kafka consumer with --from-beginning .. I can see the data on kafka topic written by samza streaming app with partition by on the topic from where pinot upsert enabled table consumes from .this topic has 64 partitions as well.

Sadim Nadeem

07/07/2021, 9:40 AM

data is there on topic but table is not consuming .. upsert enabled table creation script and schema is available above as shared by @Radhika

Radhika

07/07/2021, 9:44 AM

This is the schema for above table

schema

Sadim Nadeem

07/07/2021, 9:50 AM

Capture.PNG

Jackie

07/07/2021, 6:17 PM

Are you using the latest pinot version? I feel this might not be related to upsert feature but the table setup. Can you please try without upsert and see if the records are consumed?

Sadim Nadeem

07/08/2021, 5:41 AM

yes its consumed for other tables without upsert feature

Sadim Nadeem

07/08/2021, 5:44 AM

we are using "latest-jdk11" tag image version of pinot provided in helm .. cc: @Mohamed Sultan @Xiang Fu. .we faced some java version related issue when used just "latest" as the tag in helm .. "https://apache-pinot.slack.com/archives/C011C9JHN7R/p1625123376339500"

Xiang Fu

07/08/2021, 5:56 AM

you need to use the latest helm chart

Sadim Nadeem

07/08/2021, 6:09 AM

you mean issue in upsert may be because of not using the "latest" tag helm chart

Sadim Nadeem

07/08/2021, 6:18 AM

cc: @Mohamed Kashifuddin @Arun Kumar

Open in Slack

Previous Next