22/03/09 163903 ERROR McpEmitter: Failed to emit metadata to DataHub java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: 10,000 milliseconds timeout on connection http-outgoing-3 [ACTIVE] at datahub.spark2.shaded.http.concurrent.BasicFuture.getResult(BasicFuture.java:71) at datahub.spark2.shaded.http.concurrent.BasicFuture.get(BasicFuture.java:84) at datahub.spark2.shaded.http.impl.nio.client.FutureWrapper.get(FutureWrapper.java:70) at datahub.client.MetadataResponseFuture.get(MetadataResponseFuture.java:52) at datahub.client.MetadataResponseFuture.get(MetadataResponseFuture.java:13) at datahub.spark.consumer.impl.McpEmitter.lambda$emit$1(McpEmitter.java:39) at java.util.ArrayList.forEach(ArrayList.java:1257) at datahub.spark.consumer.impl.McpEmitter.emit(McpEmitter.java:37) at datahub.spark.consumer.impl.McpEmitter.accept(McpEmitter.java:71) at datahub.spark.DatahubSparkListener$3.apply(DatahubSparkListener.java:300) at datahub.spark.DatahubSparkListener$3.apply(DatahubSparkListener.java:285) at scala.Option.foreach(Option.scala:407) at datahub.spark.DatahubSparkListener.processExecutionEnd(DatahubSparkListener.java:285) at datahub.spark.DatahubSparkListener.onOtherEvent(DatahubSparkListener.java:272) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:82) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37) at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:115) at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:99) at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:105) at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:105) at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:100) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:96) at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1319) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:96) Caused by: java.net.SocketTimeoutException: 10,000 milliseconds timeout on connection http-outgoing-3 [ACTIVE] at datahub.spark2.shaded.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at datahub.spark2.shaded.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:98) at datahub.spark2.shaded.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:40) at datahub.spark2.shaded.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at datahub.spark2.shaded.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at datahub.spark2.shaded.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReact

we’ll take a look Jing

@stocky-midnight-78204 this error is typical when datahub GM's server is not accessible from spark cluster. Can you please check connectivity? You can execute bellow command to confirm connection. Curl '<gmsserver>/config'

I got some tasks lineage from spark and lost some task's lineage

this is pointing to intermittent connectivity drop. As stack trace shows that listener did tried to send the lineage.