Hello everyone , I am shubham . I am new to Apache...
# general
s
Hello everyone , I am shubham . I am new to Apache pinot , so any one could plz tell me is that possible to ingest data (offline table) in pinot using java spark program . SO my use case is that i want to load and processed data using java spark and need to sink the data in pinot offline table and i have installed pinot locally in my machine ?
s
but i want to write a java program for that its scala
one more thing how can i add transformation or filtration for data ingestion ?
k
you can simply import pinot-spark-ingestion modules into your java program and then trigger LaunchDataIngestionJob there’s no tutorial for it as such though
for transform and filters you can simply add this to table configs https://docs.pinot.apache.org/developers/advanced/ingestion-level-transformations side note: don’t use the groovy functions
🌟 1
s
thanks alot man
e
@Kartik Khare Curious , any specific reason to avoid groovy functions in transform configs ?
k
security, we also keep them disabled by default
e
Yes , was aware of that . Other than that is there any other reason to avoid groovy ?