Join Slack
Powered by
How to ingest s3 so that the icon shows S3 like <h...
# ingestion
s
square-activity-64562
08/24/2021, 12:59 PM
How to ingest s3 so that the icon shows S3 like
https://demo.datahubproject.io/dataset/urn:li:dataset:(urn:li:dataPlatform:s3,datahubpro[…]-pipelines.entity_aspect_splits.all_entities,PROD)/schema
?
square-activity-64562
08/24/2021, 1:02 PM
I don't want to crawl our S3. It is going to be too costly crawl. I am ok just having the bucket names so I can add owners, lineage etc.
square-activity-64562
08/24/2021, 1:02 PM
So cannot use
https://datahubproject.io/docs/metadata-ingestion/source_docs/s3
b
bland-orange-95847
08/24/2021, 1:14 PM
I guess you then need to emit dataset MCEs
manually
and set the platform to
s3
Open in Slack
Previous
Next