is it possible to ingest metadata with a regular H...
# ingestionp
powerful-egg-69769
02/22/2021, 4:11 PMis it possible to ingest metadata with a regular HTTP request on the DataHub REST API?
c
curved-magazine-23582
02/22/2021, 4:14 PMyes, I have done that. for example: to ingest dataset, use
<http://localhost:8080/datasets?action=ingest>p
powerful-egg-69769
02/22/2021, 4:14 PMthanks, can i post the sample JSON from the repo to that URL?
c
curved-magazine-23582
02/22/2021, 4:16 PMhere is the example api request: https://github.com/linkedin/datahub/tree/master/gms#create-dataset
p
powerful-egg-69769
02/22/2021, 4:17 PMthanks thats really useful
b
big-carpet-38439
02/22/2021, 4:41 PMyep, thanks Ming. @powerful-egg-69769 would it be helpful if we provided client libraries to ingest? if so, what languages would be best?
p
powerful-egg-69769
02/22/2021, 4:45 PMprobably Java & Python would be most useful, as long as they are thin libraries (i.e. not many dependencies)
powerful-egg-69769
02/22/2021, 4:45 PMfor now i think the most helpful thing would be some more examples on dataset metadata
👍 2
powerful-egg-69769
02/22/2021, 4:47 PMwe can see some super simple ones in the repo; would be interesting to try more complex metadata
b
big-carpet-38439
02/22/2021, 5:29 PMabsolutely. agree the metadata included by default is pretty boring... if you're feel inspired feel free to contribute something a bit more comprehensive 🙂
big-carpet-38439
02/22/2021, 5:29 PMcc. @mammoth-bear-12532 @gray-shoe-75895 on client lib discussions
p
powerful-egg-69769
02/22/2021, 5:49 PMif you get swagger on your API that is probably enough
powerful-egg-69769
02/22/2021, 5:49 PMsince you can auto-generate most client APIs from a swagger endpoint
g
gray-shoe-75895
02/22/2021, 5:55 PMWe’re also working on simplifying the install process of the python ingestion libraries - they have a fully typed python class hierarchy which is used to send http requests internally
p
powerful-egg-69769
02/22/2021, 6:44 PMgreat! if we can set them up without having a dependency on a github repo that would be awesome
powerful-egg-69769
02/22/2021, 6:45 PMright now i think one of the pip install dependencies pulls from a github repo
g
gray-shoe-75895
02/22/2021, 7:03 PMyep I've already asked the maintainer to publish a new version to PyPI