https://pinot.apache.org/ logo
Channels
Powered by
j

Jackie

07/05/2021, 4:40 PM
@Radhika @Sadim Nadeem Please check the controller and server log and see if there is any error logged? Even if the partitioning is wrong, you should still be able to see data consumed, so that should not be the cause
s

Sadim Nadeem

07/06/2021, 12:40 PM
no errors in logs..only warnings which @Radhika shared before.. let me reshare it
upsertlogs.txt
j

Jackie

07/06/2021, 6:13 PM
This should not be related. I didn't see the 
stream.kafka.zk.broker.url
config in your table config, which might be the cause. By default it should be 
localhost:2191/kafka
r

Radhika

07/07/2021, 9:30 AM
These are logs and 2 table schema where we added the url with "\kafka" and without "\kafka".
table1tableupsertlogs.txt
s

Sadim Nadeem

07/07/2021, 9:36 AM
still it didnt worked @Jackie . need some help .. please have a look .. cc: @Mohamed Hussain @Mayank @Xiang Fu @Kishore G
for few seconds .. the doc count showed on controller UI as 1 but data was not showing and after few second ..it came back to zero .. even though if I run the command for kafka consumer with --from-beginning .. I can see the data on kafka topic written by samza streaming app with partition by on the topic from where pinot upsert enabled table consumes from .this topic has 64 partitions as well.
data is there on topic but table is not consuming .. upsert enabled table creation script and schema is available above as shared by @Radhika
r

Radhika

07/07/2021, 9:44 AM
This is the schema for above table
schema
s

Sadim Nadeem

07/07/2021, 9:50 AM
message has been deleted
j

Jackie

07/07/2021, 6:17 PM
Are you using the latest pinot version? I feel this might not be related to upsert feature but the table setup. Can you please try without upsert and see if the records are consumed?
s

Sadim Nadeem

07/08/2021, 5:41 AM
yes its consumed for other tables without upsert feature
we are using "latest-jdk11" tag image version of pinot provided in helm .. cc: @Mohamed Sultan @Xiang Fu. .we faced some java version related issue when used just "latest" as the tag in helm .. "https://apache-pinot.slack.com/archives/C011C9JHN7R/p1625123376339500"
x

Xiang Fu

07/08/2021, 5:56 AM
you need to use the latest helm chart
s

Sadim Nadeem

07/08/2021, 6:09 AM
you mean issue in upsert may be because of not using the "latest" tag helm chart
cc: @Mohamed Kashifuddin @Arun Kumar
Powered by