<@UV0M2EB8Q> Is there a list of platforms supporte...
# ingestion
s
@mammoth-bear-12532 Is there a list of platforms supported? I ingested a mariadb using mysql connector. It worked. But it is showing mysql in datahub which I would like to correct to "mariadb". I was thinking adding
underlying_platform
as an option in mysql source. What would be the correct thing here "mariadb" or something else?
Is it present for
S3
? Was thinking of allowing
s3
as
underlying_platform
in the glue data source
l
@square-activity-64562 any luck figuring this out? trying to do the same
m
@late-father-17108 what have you been trying to do?
l
@mammoth-bear-12532 I'm also trying to ingest some data from s3 (mostly files to manually tag), and am wondering how the s3 datasets like this are populated in the demo. When I ingest via glue crawler it shows up as "glue" data, so am wondering if the ingestion pattern used in the demo is different?
m
Got it… this S3 dataset gets populated thru lineage edges emitted by the Airflow tasks that read / write to it
👍 1
In your case, do you have an s3 folder that a glue table points to?
or is it a specific s3 file?
l
we have a folder that it points to
i guess the desire is to surface to end users that the "dataset" they're looking at is an s3 dataset as opposed to a glue artifact (if possible)
m
makes sense! would it be okay to see lineage from the Glue dataset to the S3 dataset (@late-father-17108)?
l
might be okay - just trying to get a sense of what's possible for now. I could imagine us wanting to add our own custom lineage data (non-glue) in the future as well.
m
We def will be adding Glue -> S3 auto-lineage-edge very shortly
👍 1
adding custom lineage is quite easy… you just need to use the python sdk and emit away 🙂
💯 1