Hi Datahub Team, Sorry for the duplicate post. Th...
# troubleshoot
g
Hi Datahub Team, Sorry for the duplicate post. There is no reply for the following questions in the "getting-started" channel. 1. Will Datahub Ingest cli (Python), Datahub UI ingestion and GMS rest API be governed by access policies (https://datahubproject.io/docs/authorization/policies/) or apache ranger plugin? 2. On a related question related to Apache Ranger plugin, are you able to provide screenshots/examples on how to configure metadata privileges via the Ranger UI? I was able to verify that platform privileges could be offloaded to Ranger and Datahub was able to sync the policies from Ranger correctly. However, it was not the case for metadata privileges and I was not able to get Datahub to apply the metadata related policies from Ranger. Can the team verify that datahub integration with ranger is working with metadata privileges and we can offload them to Ranger? 3. What is the use of this env variable "REST_API_AUTHORIZATION_ENABLED"? It is not clearly documented in the project
✅ 1
b
@astonishing-answer-96712 can we get some assistance with the ranger config? my colleague wasnt able to set metadata policies using Apache ranger
a
Thanks for the patience, we were out last week for a conference- I responded to questions in the other thread, but @hundreds-photographer-13496 may be able to provide some more info here around ranger
g
Hi @glamorous-microphone-33484 Please find the responses #1 I need to verify whether REST API can be govern with Apache Ranger policies. However flag REST_API_AUTHORIZATION_ENABLED is used to enable authorization on REST API. @big-carpet-38439 Could you please add more information on REST_API_AUTHORIZATION_ENABLED #2 Could you please give an example #3 It is used to control REST API authorization
a
Lets continue in this thread instead. Yup, i am trying to find out if rest api can be enforced by apache ranger policies. For #2, there is no example on how to create metadata access policies with ranger in the documentation. @gentle-hamburger-31302 @astonishing-answer-96712
a
I wanted to say what use-case you wanted to achieve , I will figure it out how to setup the policy
a
Maybe we can start with a simple use case where a user can view/edit dataset/tag. I want to verify that metadata policies set in apache ranger can be enforced in datahub. I was not able to get it to work previously.
@gentle-hamburger-31302 any updates?
a
Since @gentle-hamburger-31302 is out this week, let’s see if @hundreds-photographer-13496 can provide some insight
p
@gentle-hamburger-31302 @hundreds-photographer-13496 any updates for #2?
g
Hi @wonderful-jordan-36532 Please post question related to setting up
REST_API_AUTHORIZATION
in helm chart on #all-things-deployment I resume work today, I will update you on #2 soon
Hi @wonderful-jordan-36532 In Apache Ranger create a policy where resource type is dataset and resource value is * and in permission section you can add required permission like
EDIT_DATASET_COL_TAGS
EDIT_DATASET_COL_DESCRIPTION
. Also disable the default datahub policy authorizer by setting environment variable
AUTH_POLICIES_ENABLED
to false Please refer below policy screenshot, here in drop-down menu select
dataset
All possible privileges can be found at https://github.com/acryldata/datahub-ranger-auth-plugin/blob/main/datahub-ranger-plugin/conf/servicedef.json refer
name
key
a
Have you verified it against the datahub ui?
g
No, I don't have privacera setup, however As per code flow above combination should work. If you have a setup then it would be quick for you to verify
a
That's what i did previously. It was not working and hence i was asking the developers to verify over at your side.
g
Did you set AUTH_POLICIES_ENABLED to false ?
a
I was able to get platform privileges working but not metadata privileges.
g
What was value of AUTH_POLICIES_ENABLED when you tried metadata privileges ?
w
@gentle-hamburger-31302 Thanks for your response. I had a larger question at hand. By default, all users regardless of role or permissions seem to have unlimited access to create, modify, and delete any and all objects in Datahub through the OpenAPI/Swagger, which is accesible from the UI. What's the recommended way of applying Datahub's user policies to OpenAPI, or failing that, disabling user access to it entirely? That's why I was thinking if setting REST_API_AUTHORIZATION to True would solve this issue.
a
I believe i set it to false for AUTH_POLICIES_ENABLED.
a
Thanks @ambitious-pharmacist-14608, let me setup the privacera instance, I will check and fix if there is any bug
b
Being discussed elsewhere
g
Hi @wonderful-jordan-36532 The REST API is discussed on this thread: https://datahubspace.slack.com/archives/C029A3M079U/p1683048344045099?thread_ts=1673338726.422449
@ambitious-pharmacist-14608 / @wonderful-jordan-36532 I tried to setup privacera in docker deployment mode, however I am facing setup issue, so I posted the issue to privacera team and waiting for their reply Error message:
Could not replace file: b''/home/mohd/.ansible/tmp/ansible-tmp-1683188675.4791825-4695-147321046862942/source'' to /home/mohd/privacera/docker/solr/bin/solr: [Errno 21] Is a directory: b''/home/mohd/.ansible/tmp/ansible-tmp-1683188675.4791825-4695-147321046862942/source'' -> b''/home/mohd/privacera/docker/solr/bin/solr'''
Hi @wonderful-jordan-36532 As discussed your issue is resolved on thread: https://datahubspace.slack.com/archives/C029A3M079U/p1683048344045099?thread_ts=1673338726.422449