https://datahubproject.io logo
Join Slack
Powered by
Hello Team, I was working on custom authenticator ...
# troubleshoot
b
Hello Team, I was working on custom authenticator plugin. It’s throwing the below issue Everytime I tried to deploy the plugin. I had implemented the Authenticator interface in my class A3Authenticator. could you please have a look and let me know what is causing the error.
Copy code
java.lang.ClassCastException: class com.apple.datahub.authentication.authenticator.A3Authenticator cannot be cast to class com.datahub.plugins.Plugin (com.apple.datahub.authentication.authenticator.A3Authenticator is in unnamed module of loader com.datahub.plugins.loader.IsolatedClassLoader @30cce476; com.datahub.plugins.Plugin is in unnamed module of loader org.eclipse.jetty.webapp.WebAppClassLoader @5aa9e4eb)
        at com.datahub.plugins.loader.IsolatedClassLoader.instantiatePlugin(IsolatedClassLoader.java:101)
        at com.datahub.auth.authentication.filter.AuthenticationFilter.lambda$registerPlugins$2(AuthenticationFilter.java:187)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at com.datahub.auth.authentication.filter.AuthenticationFilter.lambda$registerPlugins$3(AuthenticationFilter.java:179)
        at java.base/java.util.Optional.ifPresent(Optional.java:183)
        at com.datahub.auth.authentication.filter.AuthenticationFilter.registerPlugins(AuthenticationFilter.java:158)
        at com.datahub.auth.authentication.filter.AuthenticationFilter.buildAuthenticatorChain(AuthenticationFilter.java:134)
        at com.datahub.auth.authentication.filter.AuthenticationFilter.init(AuthenticationFilter.java:75)
        at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:140)
        at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:731)
        at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
        at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:755)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
        at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
        at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.server.Server.start(Server.java:423)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.server.Server.doStart(Server.java:387)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.runner.Runner.run(Runner.java:519)
        at org.eclipse.jetty.runner.Runner.main(Runner.java:564)
2023-01-10 03:47:55.357:INFO:oejs.AbstractConnector:main: Started ServerConnector@4387b79e{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2023-01-10 03:47:55.378:INFO:oejs.Server:main: Started @157619ms
03:48:01.420 [pool-10-thread-1] ERROR c.d.m.ingestion.IngestionScheduler:244 - Failed to retrieve ingestion sources! Skipping updating schedule cache until next refresh. start: 0, count: 30
com.linkedin.r2.RemoteInvocationException: com.linkedin.r2.RemoteInvocationException: Received error 503 from server for URI <http://localhost:8080/entities>
        at com.linkedin.restli.internal.client.ExceptionUtil.wrapThrowable(ExceptionUtil.java:135)
        at com.linkedin.restli.internal.client.ResponseFutureImpl.getResponseImpl(ResponseFutureImpl.java:130)
        at com.linkedin.restli.internal.client.ResponseFutureImpl.getResponse(ResponseFutureImpl.java:94)
        at com.linkedin.common.client.BaseClient.sendClientRequest(BaseClient.java:36)
        at com.linkedin.entity.client.RestliEntityClient.list(RestliEntityClient.java:361)
        at com.datahub.metadata.ingestion.IngestionScheduler$BatchRefreshSchedulesRunnable.run(IngestionScheduler.java:220)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.linkedin.r2.RemoteInvocationException: Received error 503 from server for URI <http://localhost:8080/entities>
        at com.linkedin.restli.internal.client.ExceptionUtil.exceptionForThrowable(ExceptionUtil.java:98)
        at com.linkedin.restli.client.RestLiCallbackAdapter.convertError(RestLiCallbackAdapter.java:66)
        at com.linkedin.common.callback.CallbackAdapter.onError(CallbackAdapter.java:86)
        at com.linkedin.r2.message.timing.TimingCallback.onError(TimingCallback.java:81)
        at com.linkedin.r2.transport.common.bridge.client.TransportCallbackAdapter.onResponse(TransportCallbackAdapter.java:47)
        at com.linkedin.r2.filter.transport.FilterChainClient.lambda$createWrappedClientTimingCallback$0(FilterChainClient.java:125)
        at com.linkedin.r2.filter.transport.ResponseFilter.onRestError(ResponseFilter.java:79)
        at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:96)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
        at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
        at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
        at com.linkedin.r2.filter.compression.ClientCompressionFilter.onRestError(ClientCompressionFilter.java:286)
        at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:96)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
        at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
        at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
        at com.linkedin.r2.filter.message.rest.RestFilter.onRestError(RestFilter.java:84)
        at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:96)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
        at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
        at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
        at com.linkedin.r2.filter.message.rest.RestFilter.onRestError(RestFilter.java:84)
        at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:96)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
        at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
        at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
        at com.linkedin.r2.filter.message.rest.RestFilter.onRestError(RestFilter.java:84)
        at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:96)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
        at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
        at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
        at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
        at com.linkedin.r2.filter.transport.ClientRequestFilter.lambda$createCallback$0(ClientRequestFilter.java:102)
        at com.linkedin.r2.transport.http.common.HttpBridge$1.onResponse(HttpBridge.java:82)
        at com.linkedin.r2.transport.http.client.rest.ExecutionCallback.lambda$onResponse$0(ExecutionCallback.java:64)
        ... 3 common frames omitted
Caused by: com.linkedin.r2.message.rest.RestException: Received error 503 from server for URI <http://localhost:8080/entities>
        at com.linkedin.r2.transport.http.common.HttpBridge$1.onResponse(HttpBridge.java:76)
b
@gentle-hamburger-31302 Can hopefully help on this one - He led the charge to change how we load Authenticator plugins recently
g
Hi @best-wire-59738 As discussed issue is resolved now. cc: @big-carpet-38439
b
I have another doubt, For CLI based ingestion we have token config to be entered for metadata token authentication. For our custom authenticator how does that work?
@big-carpet-38439 @gentle-hamburger-31302 could you also let me know if we can disable the default token authentication. I see it is included in the Authenticator chain.
g
@best-wire-59738 We can't disable the default token authentication
b
Hello @big-carpet-38439 @gentle-hamburger-31302 I was having a small doubt. We have implemented the custom authenticator plugin, we have implemented in a way that we return different user URN for different domain related users, so that user wouldn’t be able to change datasets that belong to other domain. This was working fine for graphQL API. But when user hit openAPI for adding or deleting dataset he is able to do it without any domain restriction. I would like to know if policies aren’t considered when we use OpenAPI?
b
That is correct. Authorization is ONLY performed at the "high level" API (GraphQL). The low level data apis (Rest.li + Open API) do NOT have authorization. cc @astonishing-answer-96712 since you've been thinking about this
b
@big-carpet-38439 Is there any chance we can restrict the enduser from using the openapi and rest.li api
b
We currently have a PR in place to do this!
@orange-night-91387 is working on it, but it's still in progress 🙂
b
great will look in to it. Thanks for your quick reply
b
It's definitely on our radar as a key issue
b
@big-carpet-38439 I have a doubt with datahub cli based ingestions as we have enabled metadata Authentication and implemented custom Auth plugin, How can we pass our custom configs like we have an env variable “DATAHUB_GMS_TOKEN” 
DATAHUB_GMS_TOKEN="my-datahub-token" datahub ingest -c recipe.dhub.yaml
to pass datahub tokens, but we have other configs to be passed to complete our custom Authentication. Any pointers on this how can we Authenticate in this case
g
any data available in 
configs:
get passed to plugin
b
@gentle-hamburger-31302 For our custom Auth when the user access any Api (GraphQL API) we enforce them to add 2 headers ( token and AppId) so that we could perform our auth successfully. In the same way the user want to use Datahub CLI for Ingestion, then as GMS Authentication is enabled it restricts user until Auth is successful. In that case how the user will pass token and AppID.
g
Currently headers haven’t passed to custom plugin
b
When we can expect this update for the custom plugins
c
Any idea when that PR for REST API Authorization will be released?
a
Hi @cuddly-dinner-641, no ETA yet- i’ll see if it’s in the next release
w
@orange-night-91387 Any update on when this will be merged?
o
b
Just discussed this at office hours @orange-night-91387 - The main question was the following (perhaps you can provide guidance) • How do we restrict users from being able to use ANY of the lower level apis (Rest.li, OpenAPI) Please advise, thanks!
o
The policies that apply to the APIs are the same that apply to the rest of the application, so if you block access to any of the underlying APIs you would also not be able to use the app.
b
Can you provide an example of the privileges a user should have or not to not access any of the OpenAPI or Restli APIs? Which privileges are honored at that layer?
(vs which are not)
o
Read/Write permissions around entities primarily as those are the primary REST endpoints: https://datahubproject.io/docs/authorization/policies/ Things that won't apply are things that don't translate well to the REST APIs (i.e. Manage X vs Edit X -> Edit will apply at the REST APIs, Manage X are specific to several GraphQL endpoints that do additional things). There are a few specific ones added for APIs that are marked with a footnote which were not present before.
Like I said though, access through the APIs is synonymous with access through the frontend for these so less focus should be put on accessing the API layer and policies should emphasize what you want your users to be able to do. They will be able to perform those actions through the UI or through APIs.
w
The goal is to restrict access of endusers to low-level unauthorized rest apis. Would an alternative solution be to hide the access from menu or to blacklist the url for rest api endpoints from users (but not from ingestion)
o
low-level unauthorized rest apis
With the flag enabled, they are authorized in the same way that the rest of the application is, i.e. if someone can edit a dataset on the UI then they can edit it through the API. Do you have a specific use case to block all API access regardless of user permissions?
w
Okay, I'll try enabling the flag first and get back on the situation. Our cybersec team raised critical issues that user access policies were not transfered to low-level APIs. Hopefully enabling the flag fixes the issue
g
Hi @wonderful-jordan-36532 Did you get a change to try it out?
w
Yeah worked!
3 Views
Open in Slack
PreviousNext
Powered by