dry-hair-98162
11/21/2022, 4:18 AMmicroscopic-room-90690
11/21/2022, 9:41 AMsource:
type: datahub-lineage-file
config:
file: /path/to/file_lineage1.yml
file: /path/to/file_lineage2.yml
file: /path/to/file_lineage3.yml
preserve_upstream: False
sink:
# sink configs
crooked-optician-38646
11/21/2022, 12:45 PM08:49:02 [application-akka.actor.default-dispatcher-15] WARN o.p.o.profile.creator.TokenValidator - Preferred JWS algorithm: null not available. Using all metadata algorithms: [PS384, ES384, RS384, HS256, HS512, ES256, RS256, HS384, ES512, PS256, PS512, RS512]
08:49:02 [application-akka.actor.default-dispatcher-15] ERROR auth.sso.oidc.OidcCallbackLogic - Unable to renew the session. The session store may not support this feature
aloof-leather-92383
11/21/2022, 3:35 PMcurved-apple-55756
11/21/2022, 5:18 PMstraight-mouse-85445
11/21/2022, 5:18 PMcurved-apple-55756
11/22/2022, 8:25 AMError while pulling images. Going to attempt to move on to docker compose up assuming the images have been built locally
. Do you know how to fix it ?curved-apple-55756
11/22/2022, 8:38 AMUnable to run quickstart - the following issues were detected:
- kafka-setup is still running
- schema-registry is not running
- broker is not running
- zookeeper is not running
- datahub-gms is still starting
- mysql-setup did not exit cleanly
Any help would be appreciated !acceptable-terabyte-34789
11/22/2022, 8:39 AMbreezy-portugal-43538
11/22/2022, 11:11 AM[6]: index [containerindex_v2], type [_doc], id [urn%3Ali%3Acontainer%3A25826ac8da9e99f119c577ef4ccc2194], message [[containerindex_v2/0uhknRQkSmOD3pb23xdi0g][[containerindex_v2][0]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][urn%3Ali%3Acontainer%3A25826ac8da9e99f119c577ef4ccc2194]: document missing]]]
A little bit later this error in logs pops up, but I'm not sure if it's connected to the elasticsearch issue:
10:56:59.617 [pool-13-thread-1] ERROR c.l.g.factory.telemetry.DailyReport:110 - Error reporting telemetry:
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:305)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:507)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:602)
at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266)
at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:207)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:193)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1367)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1342)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:246)
at com.mixpanel.mixpanelapi.MixpanelAPI.sendData(MixpanelAPI.java:134)
at com.mixpanel.mixpanelapi.MixpanelAPI.sendMessages(MixpanelAPI.java:172)
at com.mixpanel.mixpanelapi.MixpanelAPI.deliver(MixpanelAPI.java:103)
at com.mixpanel.mixpanelapi.MixpanelAPI.deliver(MixpanelAPI.java:83)
at com.mixpanel.mixpanelapi.MixpanelAPI.sendMessage(MixpanelAPI.java:71)
at com.linkedin.gms.factory.telemetry.DailyReport.ping(DailyReport.java:108)
at com.linkedin.gms.factory.telemetry.DailyReport.dailyReport(DailyReport.java:97)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Logs from the ingestion:
'warnings': {},
'failures': {},
'filtered': [],
'start_time': '2022-11-22 10:31:13.916942 (1.15 seconds ago).',
'running_time': '1.15 seconds'}
Sink (datahub-rest) report:
{'total_records_written': '32',
'records_written_per_second': '17',
'warnings': [],
'failures': [],
'start_time': '2022-11-22 10:31:13.248079 (1.82 seconds ago).',
'current_time': '2022-11-22 10:31:15.069306 (now).',
'total_duration_in_seconds': '1.82',
'gms_version': 'v0.9.2',
'pending_requests': '0'}
Pipeline finished successfully; produced 32 events in 1.15 seconds.
So far as I understood the pipeline has finished, but there is some weird error within the elasticsearch itself, can you guys help resolve the issue?
docker prune did not help to resolve the problem as was suggested somewhere else.
Thanks in advance!few-sunset-43876
11/22/2022, 1:48 PMlate-ability-59580
11/22/2022, 3:44 PMlate-book-30206
11/22/2022, 4:09 PMancient-library-85500
11/22/2022, 4:12 PM15:58:40.956 [pool-3-thread-1] ERROR c.l.m.m.r.PluginEntityRegistryLoader:182 - Tried 5 times. Failed to load registry mycompany-cl-model with java.lang.UnsupportedOperationException: Failed to find Snapshot associated with Config-based Entity
at com.linkedin.metadata.models.ConfigEntitySpec.getSnapshotSchema(ConfigEntitySpec.java:73)
at com.linkedin.metadata.models.registry.MergedEntityRegistry.mergeEntitySpecs(MergedEntityRegistry.java:110)
at com.linkedin.metadata.models.registry.MergedEntityRegistry.apply(MergedEntityRegistry.java:59)
at com.linkedin.metadata.models.registry.PluginEntityRegistryLoader.loadOneRegistry(PluginEntityRegistryLoader.java:147)
at com.linkedin.metadata.models.registry.PluginEntityRegistryLoader.lambda$null$5(PluginEntityRegistryLoader.java:100)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at com.linkedin.metadata.models.registry.PluginEntityRegistryLoader.lambda$start$6(PluginEntityRegistryLoader.java:100)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
brave-businessperson-3969
11/22/2022, 4:33 PMtall-furniture-10080
11/22/2022, 7:06 PMpurple-sugar-36357
11/22/2022, 9:20 PMfew-sunset-43876
11/23/2022, 4:29 AMlittle-waitress-21103
11/23/2022, 7:25 AMbetter-fireman-33387
11/23/2022, 12:45 PMdatahub delete --urn urn:li:container:3b0c3facea93ec09b88aff16b12634d0 --soft
it asks for
Do you want to delete these references? [y/N]: y
after choosing yes I get the following Error (log attached inside)
Failed to execute operation
java.lang.UnsupportedOperationException: Aspect and aspect name is required for create and update operations
Could you please assist?worried-branch-76677
11/23/2022, 3:05 PM{
"servlet": "apiServlet",
"message": "Service Unavailable",
"url": "/api/graphql",
"status": "503"
}
limited-sundown-13355
11/23/2022, 8:25 PM{
"exceptionClass": "com.linkedin.restli.server.RestLiServiceException",
"stackTrace": "com.linkedin.restli.server.RestLiServiceException [HTTP Status:404]\n\tat com.linkedin.restli.server.RestLiServiceException.fromThrowable(RestLiServiceException.java:315)\n\tat com.linkedin.restli.server.BaseRestLiServer.buildPreRoutingError(BaseRestLiServer.java:202)\n\tat com.linkedin.restli.server.RestRestLiServer.buildPreRoutingRestException(RestRestLiServer.java:254)\n\tat com.linkedin.restli.server.RestRestLiServer.handleResourceRequest(RestRestLiServer.java:228)\n\tat com.linkedin.restli.server.RestRestLiServer.doHandleRequest(RestRestLiServer.java:215)\n\tat com.linkedin.restli.server.RestRestLiServer.handleRequest(RestRestLiServer.java:171)\n\tat com.linkedin.restli.server.RestLiServer.handleRequest(RestLiServer.java:130)\n\tat com.linkedin.restli.server.DelegatingTransportDispatcher.handleRestRequest(DelegatingTransportDispatcher.java:70)\n\tat com.linkedin.r2.filter.transport.DispatcherRequestFilter.onRestRequest(DispatcherRequestFilter.java:70)\n\tat com.linkedin.r2.filter.TimedRestFilter.onRestRequest(TimedRestFilter.java:76)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:146)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:132)\n\tat com.linkedin.r2.filter.FilterChainIterator.onRequest(FilterChainIterator.java:62)\n\tat com.linkedin.r2.filter.TimedNextFilter.onRequest(TimedNextFilter.java:55)\n\tat com.linkedin.r2.filter.transport.ServerQueryTunnelFilter.onRestRequest(ServerQueryTunnelFilter.java:58)\n\tat com.linkedin.r2.filter.TimedRestFilter.onRestRequest(TimedRestFilter.java:76)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:146)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:132)\n\tat com.linkedin.r2.filter.FilterChainIterator.onRequest(FilterChainIterator.java:62)\n\tat com.linkedin.r2.filter.TimedNextFilter.onRequest(TimedNextFilter.java:55)\n\tat com.linkedin.r2.filter.message.rest.RestFilter.onRestRequest(RestFilter.java:50)\n\tat com.linkedin.r2.filter.TimedRestFilter.onRestRequest(TimedRestFilter.java:76)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:146)\n\tat com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnRequest(FilterChainIterator.java:132)\n\tat com.linkedin.r2.filter.FilterChainIterator.onRequest(FilterChainIterator.java:62)\n\tat com.linkedin.r2.filter.FilterChainImpl.onRestRequest(FilterChainImpl.java:106)\n\tat com.linkedin.r2.filter.transport.FilterChainDispatcher.handleRestRequest(FilterChainDispatcher.java:75)\n\tat com.linkedin.r2.util.finalizer.RequestFinalizerDispatcher.handleRestRequest(RequestFinalizerDispatcher.java:61)\n\tat com.linkedin.r2.transport.http.server.HttpDispatcher.handleRequest(HttpDispatcher.java:101)\n\tat com.linkedin.r2.transport.http.server.AbstractR2Servlet.service(AbstractR2Servlet.java:105)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat com.linkedin.restli.server.spring.ParallelRestliHttpRequestHandler.handleRequest(ParallelRestliHttpRequestHandler.java:63)\n\tat org.springframework.web.context.support.HttpRequestHandlerServlet.service(HttpRequestHandlerServlet.java:73)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)\n\tat org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)\n\tat com.datahub.authentication.filter.AuthenticationFilter.doFilter(AuthenticationFilter.java:88)\n\tat org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)\n\tat org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\n\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\n\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\n\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: com.linkedin.restli.server.RoutingException\n\tat com.linkedin.restli.internal.server.RestLiRouter.process(RestLiRouter.java:111)\n\tat com.linkedin.restli.server.BaseRestLiServer.getRoutingResult(BaseRestLiServer.java:181)\n\tat com.linkedin.restli.server.RestRestLiServer.handleResourceRequest(RestRestLiServer.java:224)\n\t... 68 more\n",
"status": 404
}
purple-sugar-36357
11/23/2022, 10:55 PMfierce-electrician-85924
11/24/2022, 7:01 AM./gradlew :datahub-frontend:dist
but it's throwing this error
datahub/datahub-frontend/app/auth/AuthModule.java:8:33: error: cannot access Actor
any solution for this?happy-magazine-16314
11/24/2022, 11:24 AMdatahub docker quickstart
on fresh Ubuntu 22.04 install. I just wanted to let you know how I resolved it. I simply had to run sudo apt install sasl2-bin
and after that I managed to start all containers without any further errors. I'm using the latest datahub version v0.9.2.brief-ability-41819
11/24/2022, 1:39 PMDATAHUB_TELEMETRY_ENABLED=false
.
Documentation is only for the Docker Compose: https://datahubproject.io/docs/deploy/telemetry/#overview-of-datahub-telemetry.limited-sundown-13355
11/25/2022, 3:06 AMhelpful-librarian-40144
11/25/2022, 5:15 AMacceptable-terabyte-34789
11/25/2022, 12:56 PMacceptable-terabyte-34789
11/25/2022, 12:56 PM