In a realtime table .. I went to cluster manager > tables > mytable > edit schema > added one string column *=> newly added column in schema was not reflecting on sql query editor(query is "_select * from mytable"_) on pinot controller UI* again went to cluster manager > tables > mytable > reload all segments *=> now column reflecting on sql query editor(query is "_select * from mytable"_) on pinot controller UI* with all existing rows for this particular column having null values which is ok .. the issue is newly added records also getting updated with null while json posted on kafka topic for the table have non null value of the new column *also when i fire select query with newly added column like (query is "_select * from mytable where newColumn = 'null'"_) then its returning 10 rows with newly addedcolumn only as null but when i fire query like (query is "_select * from mytable where newColumn != 'null'"_) .,. then no rows are returned ..* also when i do "_select newColumn from mytable where newColumn != 'null'"_ .. then no rows returned but when I do "_select newColumn from mytable where newColumn = 'null' "_.. then 10 rows returned with all newColumn value as null

Are you saying that after you modified the schema, every row is null?

no

@Jackie Do we support schema evolution for consuming segments?

Does below apply @Sadim Nadeem

Real-Time Pinot table: In case of real-time tables, make sure the "pinot.server.instance.reload.consumingSegment" config is set to true inside Server config. Without this, the current consuming segment(s) will not reflect the default null value for newly added columns.

Thanks @Kulbir Nijjer.

Looks like default for

pinot.server.instance.reload.consumingSegment

is false but we have a PR to make it true in next release : https://github.com/apache/incubator-pinot/pull/7078

thanks @Mayank and @Kulbir Nijjer .. very helpful

You will need to restart the server

will kubectl delete po pinot-broker-0 pinot-controller-0 pinot-controller-1 pinot-controller-2 pinot-controller-3 pinot-server-0 pinot-server-1 pinot-server-2 pinot-server-3 -n myNamespace

You will only need to restart Pinot servers, controllers/brokers shouldn't require restart since this change doesn't apply to them. Prior to restart make sure property is in the conf and set to true.

in a general table creation script.. i can see stream config and segment config but not server config

It's NOT a table config, but server side config https://docs.pinot.apache.org/configuration-reference/server

ok got it

After adding the pinot.server.instance.reload.consumingSegment to true in /var/pinot/server/config/server.conf, we restarted the pods, all other pods got restarted properly except broker pods shows the below issue. @Mayank @Xiang Fu @Jackie

Can u try to delete broker pods and let them recreate?

this screenshot is the final line of logs, and also after removing that pinot.server.instance.reload.consumingSegment, it still shows the same error in the broker and broker is restarting again and again

I think the issue is that the previous Pinot broker session wasn’t timeout yet

sure @Xiang Fu .. we are able to add new column successfully in existing table .. thanks @Mayank @Kulbir Nijjer