https://datahubproject.io logo
Join Slack
Powered by
General - Trying to Integrate another app to manag...
# all-things-deployment
h
General - Trying to Integrate another app to manage the Groups and users . How to we achieve this. ? Currently I see we have https://datahubproject.io/docs/metadata-service/#ingesting-entities . Also if we were to use this, how do authenticate such request (OAuth2, API token) ?
b
Hey hey
Typically people batch ingest from the same system where they are doing authentication.. for example: 1. LDAP batch extractor with an LDAP authentication via jaas module 2. Azure / Okta batch extractor with OIDC authentication
h
@big-carpet-38439 - we have a system which handles user creation, accessing groups etc. and we see the following API present in Datahub currently API to create user - https://datahubproject.io/docs/metadata-service/#create-a-user API to delete user - https://datahubproject.io/docs/metadata-service/#soft-deleting-an-entity API to add groups - https://datahubproject.io/docs/metadata-service/#create-a-group . we wanted the following details How to we delete users from a group ? Are there any pagination limits ? for creating users/groups Any other header information that needs to be passed ?
b
Hi @handsome-football-66174 you can of course use those APIs, or you can use our GraphQL API which may be more well suited for you use case. Notice that we support all of these operations: https://datahubproject.io/docs/graphql/mutations
h
@big-carpet-38439 - Thank you for the details, looks like GraphQL API has everything we are looking for. How can we use the above API, can you share some examples for the above.
b
You can play with an interactive version of the API at demo.datahubproject.io/api/graphiql
And there is some documentation here around issuing sample queries:
h
Thank you @big-carpet-38439. I tried the GraphQL version, they seem to be running fine, but the curl commands are erroring out
b
oh my
h
Is there a way to convert the GraphQL to the curl version
curl --location --request POST '<https://<hostname>/api/graphql>' \ --header 'X-DataHub-Actor: urnlicorpuser:datahub' \ --header 'Content-Type: application/json' \ --data-raw '{ "query":"{\\ncorpUser(urn: \"urnlicorpuser:datahub\"){\\nusername\\nurn\\n}\n}","variables":{}}' looks like it doesnt like the new lines
Also dont seem to be getting any results via the curl...
@big-carpet-38439 - Anything I am missing here with the curl version ?
b
It seems likely that you’ll need to remove the new line characters
Have you tried that
h
@big-carpet-38439 - removed the new line characters and tried, still no output.
b
What’s the output saying
Nothing?
Note that we have working curls in the GraphQL docs
Can you try to follow as an example
h
@big-carpet-38439 - Tried the curl commands from the this link - https://datahubproject.io/docs/api/graphql/querying-entities/
b
Taking a look
h
This one works (the generic curl commands)- curl --location --request POST '<https://<gms-hostname>/entities?action=search>' \ --header 'X-RestLi-Protocol-Version: 2.0.0' \ --header 'Content-Type: application/json' \ --data-raw '{ "input": "analytics_iha_dx_codes", "entity": "dataset", "start": 0, "count": 10 }' The curl commands from GraphQL (have tried using both GMS hostname & Frontend hostname)- curl --location --request POST '<https://<gms-hostname>/api/graphql>' \ --header 'X-DataHub-Actor: urnlicorpuser:datahub' \ --header 'Content-Type: application/json' \ --data-raw '{ "query":"{ \n corpUser(urn: \"urnlicorpuser:datahub\") { \n username \n urn \n } \n}", "variables":{}}'
b
I'm seeing this also -- Let me give you a query that works and then update the docs
For the previously query, this should work
Copy code
curl --location --request POST '<http://localhost:8080/api/graphql>' \
--header 'X-DataHub-Actor: urn:li:corpuser:datahub' \
--header 'Content-Type: application/json' \
--data-raw '{ "query":"{ corpUser(urn: \"urn:li:corpuser:datahub\") { username urn } }", "variables":{}}'
Let me go ahead and update those docs
h
@big-carpet-38439 - Noticed the following - In this page lot of curl commands are referring to urnlicorpUser:datahub, they may need to be modified to urnlicorpuser:datahub https://datahubproject.io/docs/metadata-service Getting this error Caused by: java.net.URISyntaxException: Urn entity type should be 'corpuser'.: urnlicorpUser:datahub at com.linkedin.common.urn.CorpuserUrn.createFromUrn(CorpuserUrn.java:34) at com.linkedin.common.urn.CorpuserUrn.createFromString(CorpuserUrn.java:27) at com.linkedin.common.urn.CorpuserUrn$1.coerceOutput(CorpuserUrn.java:61) But when I use urnlicorpuser:datahub, I see the group gets created, but no members added to it.
b
Thank you!
Great catch... Would you be interested in making these updates?
h
Sure. I can make the changes. But the curl doesn't seem to work( creates the group, but with no members)
b
Which CURL?
h
b
Thanks! This is using the deprecated "members" field. Currently, we use the GroupMembership aspect to store the groups that a user is a part of
I will need to update the docs accordingly. Thanks!
👍 1
h
@big-carpet-38439 - Please let me know once the docs have been updated.
b
Sure.
h
@big-carpet-38439 - Hi John, Wanted to check the tentative timelines when the docs would be updated.
@big-carpet-38439 - Also need the what fields can be displayed here ( tried adding displayname, fullname , active and email )
b
Entity will return an "interface" type:
You can fetch fields from any entities that are returned in the search results:
Copy code
{
    ... on Dataset {
       // Dataset fields!
       urn
       name
    }
    ... on Chart {
       // Chart fields
    }
    ... and so on
}
As for the first question, I think by end of day tomorrow I'll be able to have something there
👍 1
Would it be helpful to include a search query in our GraphQL docs?
h
@big-carpet-38439 - Would be really helpful !!
@big-carpet-38439 - Any update on the docs ?
nvm, Using the GraphQL API now.
Open in Slack
PreviousNext
Powered by