Hi All, We are on Datahub v0.10.1 with opensearch ...
# troubleshoot
w
Hi All, We are on Datahub v0.10.1 with opensearch (
OpenSearch 1.3
) and we are facing 500s while downloading a lineage data for a Swnoflake table. Error stack trace is in thread.
✅ 1
l
Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: 1️⃣ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? Yes button 2️⃣ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Yes button
w
Copy code
2023-05-03 12:38:15,253 [ForkJoinPool.commonPool-worker-9] ERROR c.l.d.g.r.s.ScrollAcrossEntitiesResolver:93 - Failed to execute search for multiple entities: entity types [], query , filters: [com.linkedin.datahub.graphql.generated.AndFilterInput@254460ff], searchAfter: null, count: 500
2023-05-03 12:38:15,253 [ForkJoinPool.commonPool-worker-9] ERROR c.l.d.g.e.DataHubDataFetcherExceptionHandler:21 - Failed to execute DataFetcher
java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to execute search: entity types [], query , filters: [com.linkedin.datahub.graphql.generated.AndFilterInput@254460ff], start: null, count: 500
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1702)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.RuntimeException: Failed to execute search: entity types [], query , filters: [com.linkedin.datahub.graphql.generated.AndFilterInput@254460ff], start: null, count: 500
	at com.linkedin.datahub.graphql.resolvers.search.ScrollAcrossEntitiesResolver.lambda$get$0(ScrollAcrossEntitiesResolver.java:97)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
	... 6 common frames omitted
Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: Failed to serialize 'org.javatuples.Sextet'
	at com.hazelcast.internal.serialization.impl.SerializationUtil.handleSerializeException(SerializationUtil.java:129)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:241)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:217)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toData(AbstractSerializationService.java:202)
	at com.hazelcast.map.impl.proxy.MapProxySupport.toDataWithStrategy(MapProxySupport.java:1338)
	at com.hazelcast.map.impl.proxy.MapProxySupport.getInternal(MapProxySupport.java:371)
	at com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:123)
	at com.hazelcast.spring.cache.HazelcastCache.lookup(HazelcastCache.java:162)
	at com.hazelcast.spring.cache.HazelcastCache.get(HazelcastCache.java:73)
	at com.linkedin.metadata.search.client.CachingEntitySearchService.getCachedScrollResults(CachingEntitySearchService.java:248)
	at com.linkedin.metadata.search.client.CachingEntitySearchService.scroll(CachingEntitySearchService.java:131)
	at com.linkedin.metadata.search.SearchService.scrollAcrossEntities(SearchService.java:113)
	at com.linkedin.metadata.client.JavaEntityClient.scrollAcrossEntities(JavaEntityClient.java:341)
	at com.linkedin.datahub.graphql.resolvers.search.ScrollAcrossEntitiesResolver.lambda$get$0(ScrollAcrossEntitiesResolver.java:79)
	... 7 common frames omitted
Caused by: java.io.NotSerializableException: com.linkedin.metadata.query.filter.Filter
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
	at java.base/java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1369)
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1165)
	at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1543)
	at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1500)
	at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1423)
	at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1169)
	at java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:345)
	at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.write(JavaDefaultSerializers.java:124)
	at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.write(JavaDefaultSerializers.java:117)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:39)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:238)
	... 19 common frames omitted
2023-05-03 12:38:15,254 [ForkJoinPool.commonPool-worker-111] ERROR c.datahub.graphql.GraphQLController:101 - Errors while executing graphQL query: ...
The GraphQL query in the error logs is too long to paste it over a slack. But last few line of the error are
Copy code
result: {errors=[{message=An unknown error occurred., locations=[{line=2, column=3}], path=[scrollAcrossEntities], extensions={code=500, type=SERVER_ERROR, classification=DataFetchingException}}], data={scrollAcrossEntities=null}, extensions={tracing={version=1, startTime=2023-05-03T12:38:15.244434Z, endTime=2023-05-03T12:38:15.254164Z, duration=9732078, parsing={startOffset=1935993, duration=1916505}, validation={startOffset=6525765, duration=4576027}, execution={resolvers=[{path=[scrollAcrossEntities], parentType=Query, returnType=ScrollResults, fieldName=scrollAcrossEntities, startOffset=6606056, duration=2736292}]}}}}, errors: [DataHubGraphQLError{path=[scrollAcrossEntities], code=SERVER_ERROR, locations=[SourceLocation{line=2, column=3}]}]
a
Hi! I think @gray-shoe-75895 might be able to help here
a
I believe this was fixed recently - either in 0.10.2 or in the upcoming 0.10.3 release (which should be next week)