Does datahub provide a list of APIs (similar to - ...
# troubleshootm
microscopic-musician-99632
09/13/2021, 9:53 AMDoes datahub provide a list of APIs (similar to - https://atlas.apache.org/api/v2/index.html ).
b
bland-orange-95847
09/13/2021, 9:59 AMRestli interface of GMS offers something like that if you e.g. go on
/restli/docs/rest/entitiesm
microscopic-musician-99632
09/13/2021, 10:10 AMThank you for the pointer. Most of the links from the path are not working hence I was not able to explore much (com.linkedin.restli.server.RoutingException: Invalid documentation path /restli/docs/rest/restli/docs) . I wanted to check if there are APIs available using which I can view the details of my metadata . (Say I ingest from SQL server I am able to see in the datahub UI the details, but I want the API which would give me that info (preferably as JSON) so that I can use it my further downstream logic/pages.
b
bland-orange-95847
09/13/2021, 10:43 AMyeah I also experienced that sometimes the base path is not correct and modifying the URL manually helps.
So you want to do a search by platformType?
m
microscopic-musician-99632
09/13/2021, 11:51 AMI would like to get the "aspects" (I hope the terminology is correct) of the entities I loaded. Say clicking on a entity, currently the UI displays the following - http://myip:9002/dataset/urn:li:dataset:(urn:li:dataPlatform:mongodb,utilitiesdb.well,PROD)/Schema?is_lineage_mode=false and it shows me the Schema definition. How do I get the corresponding JSON data . I tried few options as given - https://datahubproject.io/docs/metadata-modeling/metadata-model#querying-an-entity but unable but keep getting 404. Hence asking if there are any api docs/swagger available. I might be missing something basic but unable to understand. Atlas allows searches like : /api/atlas/v2/search/dsl?typeName=hive_table&query=where.
g
green-football-43791
09/13/2021, 3:09 PMHey @microscopic-musician-99632 - you can try checking out the graphiql interface by navigating to
/api/graphiqlgreen-football-43791
09/13/2021, 3:10 PMor try it out on demo: https://demo.datahubproject.io/api/graphiql
green-football-43791
09/13/2021, 3:11 PMthis lets you play around with querying Datahub's Graphql api. You can explore the schema by hitting
Docsm
microscopic-musician-99632
09/13/2021, 3:51 PMSorry I am not familiar with GraphQL (will try to check it ) , simply put I want to get the schemameta data for my mongo collection (well) that I have ingested from database (utilitiesdb) => urn:li:dataPlatform:mongodb,utilitiesdb.well,PROD . Along the lines of curl 'http://localhost:8080/aspects/urn%3Ali%3Adataset%3A(urn%3Ali%3AdataPlatform%3Afoo%2Cbar%2CPROD)?aspect=schemaMetadata&version=0' given on https://datahubproject.io/docs/metadata-modeling/metadata-model/#querying-an-entity
g
green-football-43791
09/13/2021, 4:08 PMgot it! Is that command you showed not working for you?
b
big-carpet-38439
09/13/2021, 6:19 PM@microscopic-musician-99632 I would recommend you try to use the "dataset" GraphQL query... It will allow you to fetch this information... Here's an example curl:
big-carpet-38439
09/13/2021, 6:19 PM Copy code
curl --location --request POST '<http://localhost:8080/api/graphql>' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query dataset {\n dataset(urn: \"urn:li:dataset:....\") {\n schemaMetadata {\n fields {\n fieldPath\n }\n }\n }\n}","variables":{}}'m
microscopic-musician-99632
09/16/2021, 8:02 AMThank you. I was able to get the metadata from - curl --location --request GET 'http://myip:8080/entities/urn%3Ali%3Adataset%3A(urn%3Ali%3AdataPlatform%3Amongodb%2Cutilitiesdb.well%2CPROD)' . I will check out more regarding the graphQL APIs since I can filter on it.
b
big-carpet-38439
09/16/2021, 4:27 PMAwesome - I will also be adding some richer documentation for GraphQL in coming days