Join Slack
Powered by
I am looking for a spark connector for writing the...
# getting-started
f
Filip D
05/16/2022, 9:45 AM
I am looking for a spark connector for writing the data to pinot. I saw on github, that write support will be availible in the future. Do you have any news about it or tips how to deal it? Thank you in advance !
k
Kartik Khare
05/16/2022, 9:57 AM
we currently don't support writing spark dataframes/RDDs directly to Pinot. However, you can use our spark plugin to read your data and dump into pinot's storage. You can find the documentation here -
https://docs.pinot.apache.org/basics/data-import/batch-ingestion/spark
Kartik Khare
05/16/2022, 9:57 AM
Example recipe -
https://docs.pinot.apache.org/users/tutorials/ingest-parquet-files-from-s3-using-spark
2
Views
Open in Slack
Previous
Next