Hello, what will be the difference between the new...
# ingestions
square-solstice-69079
07/06/2022, 11:49 AMHello, what will be the difference between the new Delta Lake ingestion that you are working on compared to using Glue to crawl Delta Lake tables to make them available to datahub?
m
mammoth-bear-12532
07/07/2022, 3:07 AM
Hi Sindre, the new Delta Lake ingestion allows you to connect to s3 directly and read delta tables directly, skipping the Glue layer.
mammoth-bear-12532
07/07/2022, 3:08 AM
Reading delta metadata directly gives us a few advantages like being able to read operational history, schema history, transactions etc… which maybe be harder to pass thru the Glue layer