https://datahubproject.io logo
Join Slack
Powered by
Hi all. I've just pulled latest version and I can'...
# troubleshoot
h
Hi all. I've just pulled latest version and I can't ingest datasets, I get an error 
Caused by: java.net.URISyntaxException: Invalid URN Parameter: 'No enum constant com.linkedin.common.FabricType.PROD)
(full in thread), something changed about the ingestion process or this is a bug?
Copy code
Traceback (most recent call last):
  File "/opt/dwhrepo/datahub_ingestion/basic_ingestion.py", line 449, in <module>
    emitter.emit_mce(dataset.get_mce())
  File "/opt/dwhrepo/dwh/venv/lib/python3.8/site-packages/datahub/emitter/rest_emitter.py", line 145, in emit_mce
    self._emit_generic(url, payload)
  File "/opt/dwhrepo/dwh/venv/lib/python3.8/site-packages/datahub/emitter/rest_emitter.py", line 188, in _emit_generic
    raise OperationalError(
datahub.configuration.common.OperationalError: ('Unable to emit metadata to DataHub GMS', {'exceptionClass': 'com.linkedin.restli.server.RestLiServiceException', 'stackTrace': "com.linkedin.restli.server.RestLiServiceException [HTTP Status:500]: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException\n\tat com.linkedin.metadata.restli.RestliUtil.toTask(RestliUtil.java:42)\n\tat com.linkedin.metadata.restli.RestliUtil.toTask(RestliUtil.java:50)\n\tat com.linkedin.metadata.resources.entity.EntityResource.ingest(EntityResource.java:182)\n\tat sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat com.linkedin.restli.internal.server.RestLiMethodInvoker.doInvoke(RestLiMethodInvoker.java:172)\n\tat com.linkedin.restli.internal.server.RestLiMethodInvoker.invoke(RestLiMethodInvoker.java:326)\n\tat com.linkedin.restli.internal.server.filter.FilterChainDispatcherImpl.onRequestSuccess(FilterChainDispatcherImpl.java:47)\n\tat com.linkedin.restli.internal.server.filter.RestLiFilterChainIterator.onRequest(RestLiFilterChainIterator.java:86)\n\tat com.linkedin.restli.internal.server.filter.RestLiFilterChainIterator.lambda$onRequest$0(RestLiFilterChainIterator.java:73)\n\tat java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670)\n\tat java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683)\n\tat java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010)\n\tat com.linkedin.restli.internal.server.filter.RestLiFilterChainIterator.onRequest(RestLiFilterChainIterator.java:72)\n\tat com.linkedin.restli.internal.server.filter.RestLiFilterChain.onRequest(RestLiFilterChain.java:55)\n\tat com.linkedin.restli.server.BaseRestLiServer.handleResourceRequest(BaseRestLiServer.java:218)\n\tat com.linkedin.restli.server.RestRestLiServer.handleResourceRequestWithRestLiResponse(RestRestLiServer.java:242)\n\tat com.linkedin.restli.server.RestRestLiServer.handleResourceRequest(RestRestLiServer.java:211)\n\tat com.linkedin.restli.server.RestRestLiServer.handleResourceRequest(RestRestLiServer.java:181)\n\tat com.linkedin.restli.server.RestRestLiServer.doHandleRequest(RestRestLiServer.java:164)\n\tat com.linkedin.restli.server.RestRestLiServer.handleRequest(RestRestLiServer.java:120)\n\tat com.linkedin.restli.server.RestLiServer.handleRequest(RestLiServer.java:132)\n\tat com.linkedin.restli.server.DelegatingTransportDispatcher.handleRestRequest(DelegatingTransportDispatcher.java:70)\n\tat com.linkedin.r2.filter.transport.DispatcherRequestFilter.onRestRequest(DispatcherRequestFilter.java:70)\n\tat com.linkedin.r2.filter.TimedRestFilter.onRestRequest(TimedRestFilter.java:72)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:146)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:132)\n\tat com.linkedin.r2.filter.FilterChainIterator.onRequest(FilterChainIterator.java:62)\n\tat com.linkedin.r2.filter.TimedNextFilter.onRequest(TimedNextFilter.java:55)\n\tat com.linkedin.r2.filter.transport.ServerQueryTunnelFilter.onRestRequest(ServerQueryTunnelFilter.java:58)\n\tat com.linkedin.r2.filter.TimedRestFilter.onRestRequest(TimedRestFilter.java:72)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:146)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:132)\n\tat com.linkedin.r2.filter.FilterChainIterator.onRequest(FilterChainIterator.java:62)\n\tat com.linkedin.r2.filter.TimedNextFilter.onRequest(TimedNextFilter.java:55)\n\tat com.linkedin.r2.filter.message.rest.RestFilter.onRestRequest(RestFilter.java:50)\n\tat com.linkedin.r2.filter.TimedRestFilter.onRestRequest(TimedRestFilter.java:72)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:146)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:132)\n\tat com.linkedin.r2.filter.FilterChainIterator.onRequest(FilterChainIterator.java:62)\n\tat com.linkedin.r2.filter.FilterChainImpl.onRestRequest(FilterChainImpl.java:96)\n\tat com.linkedin.r2.filter.transport.FilterChainDispatcher.handleRestRequest(FilterChainDispatcher.java:75)\n\tat com.linkedin.r2.util.finalizer.RequestFinalizerDispatcher.handleRestRequest(RequestFinalizerDispatcher.java:61)\n\tat com.linkedin.r2.transport.http.server.HttpDispatcher.handleRequest(HttpDispatcher.java:101)\n\tat com.linkedin.r2.transport.http.server.AbstractR2Servlet.service(AbstractR2Servlet.java:105)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat com.linkedin.restli.server.spring.ParallelRestliHttpRequestHandler.handleRequest(ParallelRestliHttpRequestHandler.java:63)\n\tat org.springframework.web.context.support.HttpRequestHandlerServlet.service(HttpRequestHandlerServlet.java:73)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:544)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:536)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:494)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)\n\tat org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:367)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException\n\tat com.linkedin.metadata.dao.utils.RecordUtils.invokeProtectedMethod(RecordUtils.java:370)\n\tat com.linkedin.metadata.dao.utils.RecordUtils.getRecordTemplateField(RecordUtils.java:289)\n\tat com.linkedin.metadata.dao.utils.ModelUtils.getUrnFromSnapshot(ModelUtils.java:128)\n\tat com.linkedin.metadata.entity.EntityService.ingestSnapshotUnion(EntityService.java:377)\n\tat com.linkedin.metadata.entity.EntityService.ingestEntity(EntityService.java:312)\n\tat com.linkedin.metadata.resources.entity.EntityResource.lambda$ingest$4(EntityResource.java:183)\n\tat com.linkedin.metadata.restli.RestliUtil.toTask(RestliUtil.java:30)\n\t... 81 more\nCaused by: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat com.linkedin.metadata.dao.utils.RecordUtils.invokeProtectedMethod(RecordUtils.java:368)\n\t... 87 more\nCaused by: com.linkedin.data.template.TemplateOutputCastException: Invalid URN syntax: Invalid URN Parameter: 'No enum constant com.linkedin.common.FabricType.PROD). Url: None: urn:li:dataset:(urn:li:dataPlatform:exasol,main.dds.test13,PROD). Url: None\n\tat com.linkedin.common.urn.DatasetUrn$1.coerceOutput(DatasetUrn.java:78)\n\tat com.linkedin.common.urn.DatasetUrn$1.coerceOutput(DatasetUrn.java:69)\n\tat com.linkedin.data.template.DataTemplateUtil.coerceOutput(DataTemplateUtil.java:954)\n\tat com.linkedin.data.template.RecordTemplate.obtainCustomType(RecordTemplate.java:365)\n\t... 91 more\nCaused by: java.net.URISyntaxException: Invalid URN Parameter: 'No enum constant com.linkedin.common.FabricType.PROD). Url: None: urn:li:dataset:(urn:li:dataPlatform:exasol,main.dds.test13,PROD). Url: None\n\tat com.linkedin.common.urn.DatasetUrn.createFromUrn(DatasetUrn.java:55)\n\tat com.linkedin.common.urn.DatasetUrn.createFromString(DatasetUrn.java:38)\n\tat com.linkedin.common.urn.DatasetUrn$1.coerceOutput(DatasetUrn.java:76)\n\t... 94 more\n", 'message': 'java.lang.RuntimeException: java.lang.reflect.InvocationTargetException', 'status': 500})
Curl ingestion not working either
b
could be an upper/lower case issue. Have you tried 
prod
instead of 
PROD
?
h
Tried 
prod
and 
Prod
, same result
Just random search through repository showed me this:
Copy code
/**
     * Convert fabric String to FabricType
     * @param fabric PROD, CORP, EI, DEV, LIT, PRIME
     * @return FabricType
     */
    @Nonnull
    public static FabricType toFabricType(@Nonnull String fabric) {
        switch (fabric.toUpperCase()) {
            case "PROD":
                return FabricType.PROD;
            case "CORP":
                return FabricType.CORP;
            case "EI":
                return FabricType.EI;
            case "DEV":
                return FabricType.DEV;
            default:
                throw new IllegalArgumentException("Unsupported Fabric Type: " + fabric);
        }
    }
Now I've tried ingesting into an older version of datahub and got the same error, so it seems that problem is in the ingestion part of latest acryl-datahub package
I've downgraded acryl-datahub, but still getting the same error. Now I have no idea, where the problem is, and I don't know how to make the ingestion work again šŸ™ˆ
b
Oh no!!
Iā€™m wondering if something got messed up on the Java service class path. Can you rebuild the metadata service and redeploy?
šŸ‘€ 1
h
I've pulled latest version and ran docker quickstart, same result. If you want me to rebuild some containers, I can't do that, sorry šŸ˜“
b
Can you send the payload that you are emitting ?
h
šŸ¤¦ā€ā™‚ļø After looking at the payload I've realized that I've accidentally changed the urn a little. Now everything is working. Sorry for this, my mistake.
b
Well I'm glad you were able to get it working, nonethless!
5 Views
Open in Slack
PreviousNext
Powered by