Hi folks, I dont see a channel for spark-pinot connector. Was wondering if its in plan to support the connector with Spark 3 in upcoming releases?

Hi, If you are talking about Spark ingestion plugin, the support for Spark 3.x has been released with pinot 0.11

https://github.com/apache/pinot/blob/release-0.11.0/pinot-connectors/pinot-spark-connector/pom.xml#L35 I was looking at this. Am i mistaken?

For ingestion, there is a separate plugin for Spark 3.x have a look at this https://github.com/apache/pinot/tree/master/pinot-plugins/pinot-batch-ingestion

Apologies, I wanted to ask for reading data via spark from pinot ... that connector is still 2.5.4? Any plan to upgrade that to 3.x

DatasourceV2 API

is no longer available in Spark 3.x

Ohh. For that I am not sure. Let me check if that can be a small fix or not.

Thank you @Kartik Khare I think usually metric consumer are always gonna be two types, folks with ML background (spark, flink) & Analytics folks (rest API, presto etc). Will be really helpful in that regard. Let me know how can I help