https://pinot.apache.org/ logo
Join Slack
Powered by
Hey, I am getting the following error. ```Caused b...
# troubleshooting
y
Hey, I am getting the following error.
Copy code
Caused by: java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.api.java.JavaRDDLike$$anonfun$foreach$1.f$14 of type org.apache.spark.api.java.function.VoidFunction in instance of org.apache.spark.api.java.JavaRDDLike$$anonfun$foreach$1
I am using
Copy code
PINOT_VERSION=0.4.0
With the following overridden versions configs to match the environment
Copy code
<scala.version>2.11.8</scala.version>
<spark.version>2.3.1.tgt.17</spark.version> (which is specific to target)
Env:
Copy code
Spark version 2.3.1.tgt.17
Using Scala version 2.11.8, Java HotSpot(TM) 64-Bit Server VM, 1.8.0_73
Run Command:
Copy code
spark-submit \
  --class org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand \
  --master yarn \
  --deploy-mode client \
  --conf "spark.driver.extraJavaOptions=-Dplugins.dir=${PINOT_DISTRIBUTION_DIR}/plugins -Dlog4j2.configurationFile=${PINOT_DISTRIBUTION_DIR}/conf/pinot-ingestion-job-log4j2.xml" \
  --conf "spark.driver.extraClassPath=${PINOT_DISTRIBUTION_DIR}/lib/pinot-all-${PINOT_VERSION}-jar-with-dependencies.jar" \
  local://${PINOT_DISTRIBUTION_DIR}/lib/pinot-all-${PINOT_VERSION}-jar-with-dependencies.jar \
  -jobSpecFile /home_dir/z00290g/guestSdrGstDataSgl_sparkIngestionJobSpec.yaml
The complete stack trace
Copy code
Caused by: java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.api.java.JavaRDDLike$$anonfun$foreach$1.f$14 of type org.apache.spark.api.java.function.VoidFunction in instance of org.apache.spark.api.java.JavaRDDLike$$anonfun$foreach$1
	at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2133) ~[?:1.8.0_73]
	at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1305) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2006) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) ~[?:1.8.0_73]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) ~[?:1.8.0_73]
	at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75) ~[spark-core_2.11-2.3.1.tgt.17.jar:2.3.1.tgt.17]
	at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114) ~[spark-core_2.11-2.3.1.tgt.17.jar:2.3.1.tgt.17]
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:80) ~[spark-core_2.11-2.3.1.tgt.17.jar:2.3.1.tgt.17]
	at org.apache.spark.scheduler.Task.run(Task.scala:109) ~[spark-core_2.11-2.3.1.tgt.17.jar:2.3.1.tgt.17]
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345) ~[spark-core_2.11-2.3.1.tgt.17.jar:2.3.1.tgt.17]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_73]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_73]
	at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_73]
5 Views
Open in Slack
PreviousNext
Powered by