hello everyone, does any have any idea on this ? . i' m using latest release version of pinot (0.7.1). while doing the spark batchIngestion, its is throwing this. thanks

Trying to create instance for class org.apache.pinot.plugin.ingestion.batch.spark.SparkSegmentGenerationJobRunner
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/api/java/function/VoidFunction
	at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137)
	at java.base/java.lang.Class.getConstructor0(Class.java:3342)
	at java.base/java.lang.Class.getConstructor(Class.java:2151)
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:295)
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:264)
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:245)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:135)

I think pinot-spark requires spark version plugins

true. i ran this command

bin/pinot-admin.sh LaunchDataIngestionJob  -jobSpecFile /data_ssd/spark-ingestion/spark-ingestion.yml

installed spark. added the SPARK_HOME and spark sbin to path

It should work with Standalone spark installed locally too, I assume Spark was installed after you saw the initial error ? If yes please try it again, basically required jar

spark-core_*.jar

is included in <SPARK_HOME>/jars and running spark-submit should set required classpath variables. EMR can also be used if you have a running Spark cluster and subsequently Spark-Submit command you run in Pinot will require changes to

master

and

deploy-mode

settings accordingly.

im using this image

apachepinot/pinot:latest

. i tried it again and trying to run the examples one.

You can download the jar and use it

i installed spark in the container

So you need to mount all your spark classpath to it and update the classpath