https://datahubproject.io logo
Join Slack
Powered by
Hi, I am manually ingesting data with SchemaMetada...
# troubleshoot
b
Hi, I am manually ingesting data with SchemaMetadata and when I try to fetch it I get 
Exception while fetching data (/browse/entities) : java.lang.RuntimeException: Failed to retrieve entities of type Dataset
on the frontend (GMS Stacktrace in Thread) I create the data similar to glue source connector but unsure which field is NULL and how to debug it. If I push the record without SchemaMetadata it works so something gets wrong with that… Maybe someone can help me to move forward identifying the issue? Thanks in advance 🙂
Copy code
java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to retrieve entities of type Dataset
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Failed to retrieve entities of type Dataset
	at com.linkedin.datahub.graphql.GmsGraphQLEngine.lambda$null$102(GmsGraphQLEngine.java:719)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	... 1 common frames omitted
Caused by: java.lang.RuntimeException: Failed to batch load Datasets
	at com.linkedin.datahub.graphql.types.dataset.DatasetType.batchLoad(DatasetType.java:102)
	at com.linkedin.datahub.graphql.GmsGraphQLEngine.lambda$null$102(GmsGraphQLEngine.java:716)
	... 2 common frames omitted
Caused by: java.lang.NullPointerException: null
	at com.linkedin.metadata.extractor.AspectExtractor.extractAspects(AspectExtractor.java:37)
	at com.linkedin.datahub.graphql.types.dataset.DatasetType.lambda$batchLoad$0(DatasetType.java:97)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
	at com.linkedin.datahub.graphql.types.dataset.DatasetType.batchLoad(DatasetType.java:100)
	... 3 common frames omitted
09:02:22 [application-akka.actor.default-dispatcher-63934] ERROR react.controllers.GraphQLController - Errors while executing graphQL query: "query getBrowseResults($input: BrowseInput!) {\n
seems like it has something to do with the SchemaField. As soon as I add a field to my metadata it crashes in the frontend with the NPE
l
@big-carpet-38439 ^
b
Hey David, which version of DataHub are you running?
b
I’m running 0.8.10.
okay I think I can close this thread. This morning I did a complete reinstallation of datahub and databases and now it works. Last week I tried to cleanup my installation (I played around a lot with it) by dropping tabes/indices and seems like this “killed” something in the background 🤦‍♂️
b
oh man -- well im glad you were able to get it working. please let us know if you have any other issues at all
b
I have one short question, maybe it not the correct place but yeah. If I create the entities should I work with 
<http://datahub.metadata.com|datahub.metadata.com>.linkedin.pegasus2avro.schema.StringTypeClass
or 
datahub.metadata.schema_classes.StringTypeClass
or 
<http://datahub.metadata.com|datahub.metadata.com>.linkedin.pegasus2avro.schema.StringType
? Its not really clear for me which is the best way as it even differs in the sources
m
Hi @bland-orange-95847: where are you trying to use these classes? Emitting metadata using python?
b
yes. I am using them in the source connector to build my MetadataWorkUnit
m
datahub.metadata.schema_classes.StringTypeClass
is usually the one
I'm assuming you are building a SchemaMetadata
and adding Fields with types
b
yes correct. Thanks 👍 I tried to build it similar to glue source but there both packages are used. That’s why I wanted to ask 🙂
17 Views
Open in Slack
PreviousNext
Powered by