Hi Team, What is the recommended way of updating t...
# troubleshootingd
Devashish Gupta
02/10/2021, 11:06 AMHi Team, What is the recommended way of updating table schema. I used the following job to create my table
Copy code
apiVersion: batch/v1
kind: Job
metadata:
name: request-realtime-table-creation
namespace: data2
spec:
template:
spec:
containers:
- name: request-realtime-table-json
image: apachepinot/pinot:latest
args: [ "AddTable", "-schemaFile", "/var/pinot/examples/request_schema.json", "-tableConfigFile", "/var/pinot/examples/request_realtime_table_config.json", "-controllerHost", "pinot2-controller", "-controllerPort", "9000", "-exec" ]
env:
- name: JAVA_OPTS
value: "-Xms4G -Xmx4G -Dpinot.admin.system.exit=true"
volumeMounts:
- name: examples
mountPath: /var/pinot/examples
restartPolicy: OnFailure
volumes:
- name: examples
configMap:
name: pinot-table
backoffLimit: 100Devashish Gupta
02/10/2021, 2:22 PMCurrently I have updated the schema using the controller portal. I wanted to know if there is a way of doing this k8s natively. Also the schema change didnt reflect in the console until I did the complete segment reload. Is this a mandatory step or there is some work around to have simple schema updates like column addition as a O(1) operation.
x
Xiang Fu
02/10/2021, 5:29 PM
you can also use controller swagger api to update schema
n
Neha Pawar
02/10/2021, 5:38 PM
@Devashish Gupta adding columns to a schema necessarily needs a reload : https://docs.pinot.apache.org/operators/tutorials/schema-evolution#update-the-schema
Neha Pawar
02/10/2021, 5:38 PM
that’s how the changes trickle to the segments
d
Devashish Gupta
02/10/2021, 7:11 PMAlright. On a different topic, I have created a table with
Copy code
"dateTimeFieldSpecs": [
{
"name": "TimeStamp",
"dataType": "LONG",
"format": "1:SECONDS:EPOCH",
"granularity": "1:HOURS"
}
]n
Neha Pawar
02/10/2021, 7:12 PM
granularity is not used anywhere atm. It will be used in the future for segment merge/rollups
d
Devashish Gupta
02/11/2021, 2:38 PMalright. Thanks.