vishal
01/12/2023, 10:14 AMAPPEND and UNION strategies. how to configure it?vishal
01/12/2023, 10:15 AMsaurabh dubey
01/12/2023, 11:14 AMAPPEND, the value from the new record being ingested will be appended to the last column's value. UNION will merge the incoming value with existing and only retain the unique set amongst them.
Both strategies are only applicable for multi-value columnssaurabh dubey
01/12/2023, 11:14 AMvishal
01/12/2023, 11:15 AMvishal
01/12/2023, 11:15 AMvishal
01/12/2023, 11:15 AM"upsertConfig": {
"mode": "PARTIAL",
"defaultPartialUpsertStrategy": "OVERWRITE",
"partialUpsertStrategies":{
"uuid":"APPEND"
}
},vishal
01/12/2023, 11:15 AMvishal
01/12/2023, 11:15 AM{
"code": 400,
"error": "APPEND merger cannot be applied to single-value column: uuid"
}'saurabh dubey
01/12/2023, 11:23 AMsaurabh dubey
01/12/2023, 11:24 AMvishal
01/12/2023, 11:25 AMvishal
01/12/2023, 11:25 AM{
"schemaName": "tab",
"dimensionFieldSpecs": [
{
"name": "key",
"dataType": "INT"
},
{
"name": "uuid",
"dataType": "STRING"
}
],
"metricFieldSpecs": [
{
"name": "count",
"dataType": "INT"
}
],
"dateTimeFieldSpecs": [
{
"name": "ts",
"dataType": "TIMESTAMP",
"format": "1:MILLISECONDS:EPOCH",
"granularity": "1:MILLISECONDS"
}
],
"primaryKeyColumns": [
"key"
]
}saurabh dubey
01/12/2023, 11:27 AMvishal
01/12/2023, 11:28 AMsaurabh dubey
01/12/2023, 11:40 AMvishal
01/12/2023, 11:41 AMrobert zych
01/12/2023, 3:21 PM"ingestionConfig": {
"transformConfigs": [{
"columnName": "fullName",
"transformFunction": "Groovy({firstName+' '+lastName}, firstName, lastName)"
}]
}
https://docs.pinot.apache.org/developers/advanced/ingestion-level-transformations