Hello newbie question again. I see there is GMA AP...
# ingestionc
curved-magazine-23582
02/04/2021, 4:23 AMHello newbie question again. I see there is GMA API for adding/updating datasets, users, etc. I assume this is equivalent of sending corresponding MCE data to that kafka topic, i.e. Elasticsearch and neo4j will be populated eventually with either of the ingestion methods?
b
big-carpet-38439
02/04/2021, 4:25 AMYep that should be correct! Which endpoint are you looking at?
c
curved-magazine-23582
02/04/2021, 4:27 AMthanks!
<http://localhost:8080/datasets?action=ingest>b
big-carpet-38439
02/04/2021, 5:12 AMyes thats the one
big-carpet-38439
02/04/2021, 5:12 AMCurrently it's only really used by the MCE Consumer code, which consumes incoming kafka messages and hits that endpoint to persist new metadata!
c
curved-magazine-23582
02/04/2021, 5:21 AMCool thanks. I am thinking about POC a potential custom integration with AWS Glue, either through this API or through kafka topic. Haven't decided which way to go.
b
big-carpet-38439
02/04/2021, 6:02 AMWhich language will you be writing data from?
c
curved-magazine-23582
02/04/2021, 6:31 AMprobably netcore/c#
b
big-carpet-38439
02/04/2021, 11:06 PMgot it..it might be easier to use a kafka client for ingestion
c
curved-magazine-23582
02/05/2021, 2:35 AMok. might try that first then