https://datahubproject.io logo
Join Slack
Powered by
Are there any other permissions needed to ingest u...
# troubleshoot
r
Are there any other permissions needed to ingest users from azure AD? I am having trouble ingesting both users and groups.
b
You do need User.readall Group.readall and GroupMember.readall
If your service account has these things should work
What errors are you seeing?
r
When I try to ingest only users via this config
Copy code
ingest_users: True
    ingest_groups: False
    ingest_group_membership: False
I get these errors
Copy code
[2021-11-09 16:18:06,157] ERROR    {datahub.ingestion.source.identity.azure_ad:269} - Response status code: 403. Response content: b'{"error":{"code":"Authorization_RequestDenied","message":"Insufficient privileges to complete the operation.","innerError":{"date":"2021-11-09T15:18:06","request-id":"fbfba51e-b6e7-4f26-86ff-5bfee56af753","client-request-id":"fbfba51e-b6e7-4f26-86ff-5bfee56af753"}}}'
But when i remove those configs, I get these weird Unable to emit data to GMS errors
Copy code
[2021-11-09 16:04:20,246] ERROR    {datahub.ingestion.run.pipeline:83} - failed to write record with workunit urn:li:corpGroup:Excluded%20from%20MEM%20Temp with ('Unable to emit metadata to DataHub GMS', {'exceptionClass': 'com.linkedin.restli.server.RestLiServiceException', 'stackTrace': "com.linkedin.restli.server.RestLiServiceException [HTTP Status:400]: Conversion = 'M'\n\tat
keep in mind I use the same auth to setup SSO and that works fine
b
#1 is very unexpected - we are doing the same operations either way as far as I know. Can you Google this one and see if azure details why they’d be sending it? #2 I will try to repro this on our end and see if there’s something obvious going on here
I can also look at that line in the code from #1 and see if I find anything
r
For #1 the problem is that this error is not seen on Azure’s side. I had my IT team turn on debugging and they didnt see any exceptions when i ran this ingestion. Is there a way to turn up logging and see what is missing where? 😄
also would it matter if the type of access is Application vs delegated? i am seeing that when it is Application i can aur least read the groups but the users are delegated and i cannot read them, maybe that is a hint?
b
I’m not an expert in azure so I’m not positive
But sounds like it is on the right track yes!
r
yeah me neither and unfortunately I am not getting lot of help on the issue, so would be great of you can find some insight. May be it will help to see what the app is doing with these configs and how it uses the permissions
b
Okay looking into this one
It seems you simply need one of these
Copy code
User.Read.All, User.ReadWrite.All, Directory.Read.All, Directory.ReadWrite.All
It says this about delegated access for these endpoints
Copy code
Delegated (work or school account)	User.ReadBasic.All, User.Read.All, User.ReadWrite.All, Directory.Read.All, Directory.ReadWrite.All, Directory.AccessAsUser.All
Delegated (personal Microsoft account)	Not supporte
My guess is that you cannot use a delegated permission here
Trying to investigate the other issue you are seeing
q
If anyone else hits this: You do indeed need Application type access for the permissions listed in the AD integration guide, Delegated is not sufficient.
b
@quaint-branch-37931 Any chance you'd be willing to update the Azure Guide to include this piece of information?? Would be greatly appreciated!!!
q
Most certainly! I'll create a PR tomorrow.
b
Thank you so much!
Open in Slack
PreviousNext
Powered by