https://datahubproject.io logo
Join Slack
Powered by
Hello, so I am trying to integrate Datahub with Ap...
# troubleshoot
m
Hello, so I am trying to integrate Datahub with Apache Ranger. I have followed the guide that is in the documention (this) I have configured both Ranger and Datahub but when I redeploy the gms container with the Ranger options enabled i get the error of the file. Does anyone know why it is happening??
Datahub_Ranger_error.txt
Copy code
INFO  org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:312) - Operation error. response=VXResponse={org.apache.ranger.view.VXResponse@64bc3642statusCode={1} msgDesc={datahub failed to find service class com.datahub.authorizer.plugin.ranger.DataHubRangerAuthPlugin. Resource lookup will not be available. Please make sure plugin jar is in the correct place.} messageList={null} }
javax.ws.rs.WebApplicationException
	at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
	at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:311)
	at org.apache.ranger.rest.ServiceREST.createService(ServiceREST.java:775)
	at org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:61)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692)
	at org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$aa07b19f.createService(<generated>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:232)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:194)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
	at org.apache.ranger.security.web.filter.RangerSecurityContextFormationFilter.doFilter(RangerSecurityContextFormationFilter.java:141)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter$ServletFilterHttpInteraction.proceed(RangerCSRFPreventionFilter.java:226)
	at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.handleHttpInteraction(RangerCSRFPreventionFilter.java:171)
	at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.doFilter(RangerCSRFPreventionFilter.java:181)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.apache.ranger.security.web.filter.RangerKRBAuthenticationFilter.doFilter(RangerKRBAuthenticationFilter.java:436)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.apache.ranger.security.web.filter.RangerSSOAuthenticationFilter.doFilter(RangerSSOAuthenticationFilter.java:257)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:149)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:194)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:624)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1651)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
I think the former error might be related to the latter one which was obtained from Ranger. The thing is that I have placed the jar in the location said in the guide, but for some reason it is not able to find it.
g
Are you following Kubernetes steps or docker ?
^ @microscopic-mechanic-13766
m
Docker as my datahub is deployed in docker
g
Could you please run below command and share the output, run this command on host machine
Copy code
ls -l ~/.datahub/plugins/auth/resources/ | grep -ir "ranger-datahub-audit.xml" -
m
I don't have the xml file in that path, I have it in another route but mapped to that path inside the container. 
/var/volumes/datahub/plugins/auth/resources/ranger-datahub-security.xml:/home/datahub/.datahub/plugins/auth/resources/ranger-datahub-security.xml
g
The path ~/.datahub/plugins/auth/resources/ is internally mapped to /etc/datahub/.....
m
So mapping it manually to etc/datahub/plugins/auth/resources should work too right?
g
Yup, you need to map to below path inside the container
Copy code
/etc/datahub/plugins/auth/resources
The plugin is tested against Privacera Platform v6.3.0.1.
Yup only against that
I think you first try the doc with datahub docker quickstart without changing the host machine directory path
m
I have managed to make it work. The only thing that I have changed from previous tries is disabling the property 
AUTH_POLICIES_ENABLED
. As in the guide it said it was optional, I haven't disabled it until now. I will test if it works properly now.
Thank you so much for the help @gentle-hamburger-31302!!
Hello again @gentle-hamburger-31302, would you mind helping me with this error I am getting trying to define the 
ranger_datahub
service inside of Ranger? As said in previous messages, I have defined the "general" service 
DATAHUB
via curl command. I have also downloaded the ranger plugin and saved it in the following path: 
/opt/ranger-2.2.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/datahub
For some reason I ignore, I am getting this error trying to define the 
ranger_datahub
service:
Copy code
Error! Datahub failed to find service class com.datahub.authorizer.plugin.ranger.DataHubRangerAuthPlugin. Resource lookup will not be available. Please make sure plugin jar is in the correct place.
Am I skipping some steps or is the ranger integration incomplete (as I have seen here that it has been moved from completed back to planned)??
g
Did you made any change in service file ? and Please share the output of ls command on /opt/ranger-2.2.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/datahub
also ping the curl command you have executed to register the plugin
m
And what do you exactly mean by "ping the curl command"??
I downloaded the service file and made the curl command. No modification was made on the file.
g
share the curl command you have used to register the datahub plugin
m
curl -u admin:<password> -X POST -H "Accept: application/json" -H "Content-Type: application/json" --data @servicedef.json http://<host>:6080/service/public/v2/api/servicedef
g
We had verified on ranger-2.1.0-admin, which come default with Privacera Platform v6.3.0.1.
m
I have ranger-2.2.0-admin, but I doubt that there would be a great difference between versions 2.1.0 and 2.2.0
Is there anything else needed apart from the curl command and the jar?? Is 
/opt/ranger-2.2.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/datahub
the only path where the jar is supposed to be?
g
As per 2.1.0 the path is correct
it should able to find it
m
Theorically, as no major changes were done in 2.2.0 (just some improvements on what was already in 2.1.0), I am surprised it is not able to find the jar
Hello again @gentle-hamburger-31302, I have been able to create the service called 
ranger-datahub
, there were just some mistakes in the way I tried to do it. Although I have created it and I have created a policy to give the user datahub all the permissions needed to be a root user, I keep getting this error as the policies are not captured by datahub:
Copy code
[PolicyRefresher(serviceName=ranger_datahub)-42] ERROR o.a.r.a.client.RangerAdminRESTClient:1220 - Error getting Roles; service not found. secureMode=false, user=datahub (auth:SIMPLE), response=404, serviceName=ranger_datahub, lastKnownRoleVersion=-1, lastActivationTimeInMillis=1664349567076
[PolicyRefresher(serviceName=ranger_datahub)-42] WARN  o.a.r.a.client.RangerAdminRESTClient:1228 - Received 404 error code with body:[null], Ignoring
[PolicyRefresher(serviceName=ranger_datahub)-42] WARN  o.a.r.a.client.RangerAdminRESTClient:868 - Error getting policies. secureMode=false, user=datahub (auth:SIMPLE), response={"httpStatusCode":400,"statusCode":0}, serviceName=ranger_datahub
[PolicyRefresher(serviceName=ranger_datahub)-42] WARN  o.a.r.plugin.util.PolicyRefresher:393 - cache file does not exist or not readable '/tmp/datahub_ranger_datahub.json'
I am currently trying to do it with version v0.8.44 as with version v0.8.45 there are some errors and I can't see the home page correctly.
g
Please cross check the service name present in ranger-datahub-security.xml and name on ranger portal
So datahub is connected to ranger but service name is not matching
m
I have been checking and in the ranger-datahub-security.xml the property 
ranger.plugin.datahub.service.name
was duplicated, the first appareance had 
datahub
as the value, but the last one had 
ranger_datahub
. I have erased the first one, as I thought it might have been a mistake
g
I will fix it in next release
m
Would the service needed to be specified like 
datahub.ranger_datahub
or something like that? I have tried changing the name to see if it was a problem of the name, but the error persisted.
g
Please check document, I haven't tried different combination, What I had verified it is mentioned in doc
m
Is there any way to activate the secure mode connection from Datahub to Ranger? I think the source of the problem is that datahub is trying to connect to 
/service/plugins/policies/download/datahub
but, as my ranger is Keberized, the path should be 
/service/plugins/secure/policies/download/datahub
g
Ok, actually we have set of prerequisites where we have verified the integration, please check them on below link https://datahubproject.io/docs/datahub-ranger-plugin/
m
And are there any future plans on trying to intregate it without basic authentication?
I will try to implement the authentication with Kerberos, although I don't promise much. I will keep you updated!
70 Views
Open in Slack
PreviousNext
Powered by