@modern-monitor-81461 Can you raise a PR with what you have so far? @red-lizard-30438 and folks are interested in trying to extend for S3 🙂

There is one already: https://github.com/datahub-project/datahub/pull/4431. It has a few tests failing, but I'm working on fixing those with @mammoth-bear-12532 's help. The tests are failing because there is something wrong with the Avro mapping not returning what I expect. Once we fix that, the PR would be ready for a "real" review. One idea I had was to take a look at the datalake source and see if we could merge both. From my understanding, the source already crawls the datalake for assets, so we could recognize that a folder is an Iceberg table and call my code on it... Just a thought, we can look at this later once my PR is green.