On-demand AzureAD users ingestion Sorry if this was already answered, but I couldn't find it.. Here it goes: I have DataHub deployed in Azure using OIDC. When a user logs in, its profile is read from the IDToken and a CorpUser is created inside DataHub. That's all good. But then I have users that have never logged in DataHub and are running pipelines that create tables and their username are tagged as owners of that table. When I ingest that table, an ownership aspect is created and it tries to link with a CorpUser. But since the user has never logged in, that link goes to void... I understand that's why there is an AzureAD source to ingest all users (or members of a group) pro-actively, to avoid the situation I just described. For security reasons I won't go into, my organization is reluctant in having a copy (or a partial copy) of the AD users stored in an external database. We understand that when you use a tool, your identity will be exposed and we are ok with this. What we are not fan of is storing identities of users that 1. Never logged in DataHub 2. Never created a pipeline/dataset/dashboard/etc... What we would like is to ingest only users that are relevant to DataHub and the things it keeps track, on-demand. Has this been done already? One way I thought this could be fixed is by creating a transformer and looks for user identities and fetches user info from AD when the CorpUser is missing from DataHub. Obviously this transformer would have to appear in pretty much every recipe we run, but we are ok with this. • Is there a better way to solve this? • Would there be bad side-effects of operating like this?

you can use API to get owners from all entities and run a script to ingest only them from azure AD or manually enter the users through API https://datahubproject.io/docs/how/add-user-data Run this regularly as a separate process and you should not need to do changes in all your recipes. This isn't strictly a better way just an alternate which might be simpler.

Yes, I like that. Thank you @square-activity-64562

you can use search with "*" https://datahubproject.io/docs/graphql/queries#search