https://pinot.apache.org/ logo
Join Slack
Powered by
Hi Team, Does anyone know any bash or python scrip...
# general
m
Hi Team, Does anyone know any bash or python script to create schema and table in pinot ?
@User @User
j
You may use 
pinot-admin.sh
within the distribution to achieve that
m
Can you please point me to the reference pinot-admin.sh file ?
@User
j
You should be able to find it under 
pinot-tools/target/pinot-tools-pkg
If you are using the distribution, this page should provide some guidance: https://docs.pinot.apache.org/basics/getting-started/running-pinot-locally#getting-pinot
m
The pinot is running in K8s schema and table are being created via web-ui, so if there is any bash or python script which I can run and let the script create it @User?
j
pinot-admin.sh
should be able to serve the purpose
m
Okay let me try but I see it's generally used as bin/pinot-admin.sh to start the pinot in local but I don't see any eg sample table and schema inside pinot-admin.sh file which I can use it as reference @User
j
You can check all the sub-command using 
pinot-admin.sh -h
m
Yes I can understand using this we can create schema and tables inside the container eg (inside controller pod) but how do I run outside the container ? because In my case its not ideal to copy schema and table json inside a container then run it. it does makes more complex then creating it from WebUI. @User @User
Copy code
bin/pinot-admin.sh AddTable \
    -schemaFile examples/stream/airlineStats/airlineStats_schema.json \
    -tableConfigFile examples/stream/airlineStats/airlineStats_realtime_table_config.json \
    -exec
d
In YouGov we ended up using the Swagger UI to help understanding what endpoints in the Controller we could use to create the schemas and tables, via a Python script.
m
I didn't understand about what your saying @User
Can you elaborate it more ?
d
When you get Pinot running, you can reach a Swagger UI which is a UI that documents the API endpoints available in the controller, where you can also do some experimentation with endpoints and see which ones are available. So it's just a matter of sending requests to some of those endpoints, if you want to create schemas and tables.
j
As @User suggested, if you find using WebUI is easier, you may just send the same HTTP/HTTPS request using the script to achieve the same effect. The pinot-admin is essentially a helper to create the request and send it to the controller.
x
You can run
Copy code
bin/pinot-admin.sh AddTable -controllerProtocol http -controllerHost 127.0.0.1 -controllerPort 9000 ....
specify with your local reachable controller url
m
Yes agree its kind of need to do some experimentation with it @User
Is it possible to point a folder instead of a pointing a single json file eg .
Copy code
bin/pinot-admin.sh AddTable \
    -schemaFile examples/stream/airlineStats \
    -tableConfigFile examples/stream/airlineStats
In my case even I can copy the tables and schema folder then at once then if I can point each folder of table and schema folder using 
bin/pinot-admin.sh AddTable \
like above then too its fine . but the question is whether 
bin/pinot-admin.sh AddTable
can take the whole folder and apply each json file inside the folder ?
@User @User ??
j
No, but you may loop over each json file and invoke pinot-admin in your script
m
yaa that is also a way but it would have been simple if we can simple point a folder and rest is done any reason why 
bin/pinot-admin.sh AddTable
can't take the whole folder ?
x
No, you can only create a table per command
Just write a wrapper script to loop the folder and create the table
m
okay
Open in Slack
PreviousNext
Powered by