Hi Team, There's a doubt I had, there was some cha...
# troubleshooting
r
Hi Team, There's a doubt I had, there was some change in the transformation function in the Table config of my Realtime Table, But the changes doesn't reflect on the table. this transform function is an UDF and it is working when I recreated the table, but I want the older table to get refresh with new settings and configs, how can I do it? @Xiang Fu @Jackie I cannot point queries to new table how can I get it refresh with older table
x
if this is a new column, then try to reload all segments with download from deep store
if the column is already populated with old value or default null value then there you need to regenerate the segment
r
no not a new column, but earlier this field was populated with wrong transform function we were using,
{
"columnName": "s_total_double",
"transformFunction": "bytesToDouble(o_net_total, 10, 2)"
}
now I changed it to,
{
"columnName": "s_total_double",
"transformFunction": "bytesToDouble(s_total, 10, 2)"
}
that is the only change I gave
but the value is not showing for new data as well after segment reloading
x
that’s what I said, the column is already populated with wrong data. So the only way to fix it is to re-create segment from the data source
r
recreate segment means to push into the the source kafka topic from earliest?
or something else you mean?
x
or you can try minion task or spark job to recreate the segment
I would say the simplest way is to create a new column
r
new column then I will have to re run the table? isn't there any way to do that without any remaking the table?
m
Changing transform function is backward incompatible, and hence not allowed. You don’t need to create the table again, just add the new column with new transform and reload segments
r
ah okay, sure will check this one too
so @Mayank in general if I want to add any new transform function, I can add and with new column it will not impact the older records, right?
and would I get any values for older records for that column?
after reloading segments?
x
after reload the new column values will be populated and kept
r
for older segments as well?
m
Yes, reload is for older segments
r
okay thanks. Will try