https://pinot.apache.org/ logo
Join Slack
Powered by
Hello Everyone, Very quick question I have lots of...
# general
p
Hello Everyone, Very quick question I have lots of data in s3 bucket (parquet format). Can I use Pinot to retrieve or query single record base on query or condition? Is Pinot right software to do such thing?
m
Hello, is your requirement just to access the data, or do you want to run some analytical queries?
p
Hello Mayank, both
m
Ok, for analytics, Pinot is certainly the right software to do it
The data on S3 can be batch ingested into Pinot, and then can be queried
p
what about query to specific data? is it possible fastest way possible?
m
what do you mean by query to specific data?
You want to run a query directly on the data sitting on S3?
p
yes (don't want to use Athena because it is slow)
m
Hmm, ok, to use Pinot, the data has to be at least indexed and ingested into pinot
p
Yes I want to ingest the data to Pinot and then retireve via API or query..
m
You can explore Presto if you just need a SQL engine on top of data at rest
p
Yes idea is what you said: 1. S3 can be batch ingested into Pinot, and then can be queried 2. Query via API or using Pinot Dashboard for business analytics queries
m
The idea sounds good
p
Thank you very @Mayank I will start work it now 🙂
m
👍
p
Sorry one more question.. do you aware of any example or article to achieve the above?
m
p
m
That is for configuring S3 as the deep-store for Pinot
You can also look at the docs above for setting up batch ingestion
p
ok sure thank you..
Sorry again for asking so many question... is there any tool to create Pinot table & schema configuration json file from athena table or from glue?
or does this need to be done manually?
m
I am not aware if there is one
p
ok
k
There is no tool to do that
p
ok thanks @Kishore G
ok I will write small tool from converting DDL to Pinot table schema json file using node for future use 🙂
👍 3
m
Awesome
Open in Slack
PreviousNext
Powered by