Can the partial upsert be conditional? For example...
# general
h
Can the partial upsert be conditional? For example, set the column’s partialUpsertStrategies as OVERWRITE, but only overwrite the existing value when the event current being ingested has a non null value for the column.
j
Does a filter config, https://docs.pinot.apache.org/developers/advanced/ingestion-level-transformations#filtering, work if you remove that event without processing? I don’t use this feature so not 100% sure they work together, but it’s not limited as a limitation.
h
The filterConfig drops a whole event of all fields/columns. I’m thinking about the effect of dropping some fields/columns of an event. For MySQL it’s achieved with the IF
Copy code
update my_table set my_col =if(the_field_value_from_the_event!=null, the_field_value_from_the_event, my_col);
k
You can contribute that merge logic.. it’s a simple interface that takes current-value and incoming record.. I think it’s a valid strategy.. please file an issue with example
h