I'm able to see other column data only for newly a...
# troubleshooting
r
I'm able to see other column data only for newly added column it's not showing any data.this is my configuration file. Kindly suggest.
x
old data are not refilled as pinot won’t consume old data again.
r
@Xiang Fu @Jackie Followed below steps still newl added column is populating with default value which I have added in updated schema file. Kindly suggest. added pinot.server.instance.reload.consumingSegment=true in server.conf file # create a kafka topic bin/kafka-topics.sh --create --topic test_schema_hybrid --bootstrap-server localhost:9092 # Push some Events (original data) bin/kafka-console-producer.sh --topic test_schema_hybrid --bootstrap-server localhost:9092 {"id":1, "name":"A1","mtime":1610841000000} {"id":2, "name":"A2", "mtime":1610841000000} {"id":3, "name":"A3", "mtime":1610841000000} # create hybrid table with original schema bin/pinot-admin.sh AddTable \ -schemaFile /Users/dev/apps/apache-pinot-incubating-0.7.1-bin/original_Schema.json \ -tableConfigFile /Users/dev/apps/apache-pinot-incubating-0.7.1-bin/relatime_config.json \ -exec bin/pinot-admin.sh AddTable \ -schemaFile /Users/rahul1.patidar/apps/apache-pinot-incubating-0.7.1-bin/original_Schema.json \ -tableConfigFile /Users/rahul1.patidar/apps/apache-pinot-incubating-0.7.1-bin/offline_config.json \ -exec # Update Schema in Pinot bin/pinot-admin.sh AddSchema -schemaFile Updated_schema.json -exec # Reload the SEGMENT able to see the new Column value with default value(defaultNullValue in updated_schema.json) # now pushed some more events in same kafka topic with new fields. {"id":4, "name":"A4", "city":"noida", "mtime":1610841000000} {"id":5, "name":"A5", "city":"mumbai", "mtime":1610841000000} #Reload the SEGMENT But still in all the rows values are showing (defaultNullValue='Bangalore') for city column(newly added). Kindly suggest what i am missing here.how can i Update reltime values in pinot hybrid table.?
This is now I added new field in updated schema file.
IMG_20210625_111620.jpg
x
cool
what’s the issue?
r
Whenever I am pushing some new messages in Kafka topic so for the new column i.e. city instead of taking the value from Kafka topic it's taking city column value as defaultvalue from the schema file. I.e. "Bangalore". @Xiang Fu
x
Has your previous segment already persisted ?
You can try to restart the server or refresh the current consuming segment then try
r
I have only one segment and for that status is showing as consuming
Ohh okay...means it requires some downtime whenever any schema changes happens
Thanks @Xiang Fu worked after server restart
j
@RK The schema change won't directly apply to the current consuming segment (new added columns will be filled with default value). After the current consuming segment is persisted, the next one will pick up the change and consume the actual value
r
Okay Thanks @Jackie