A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hi DataHub! Is it possible to send Azure blob data to DataHub? I cannot find it....

support for adls exists via the iceberg source

Well, we have successfully ingested AWS S3 to DataHub, but i could not find a specific one for Azure Blob storage

But i guess that with the Iceberg source i should achieve the same result?

Ah got it... we've got one for S3 and one for GCS ... we would love a contribution for Azure Blob Storage... if you checkout the GCS source code, you will see it uses a S3-wrapper api to do its work, and I'm sure ABS / ADLS ecosystem would have a similar wrapper library

GCS source: <https://datahubproject.io/docs/generated/ingestion/sources/gcs/>

Let us know how this works for you :slightly_smiling_face:

Hi Guys, picking up from where Richard left. Any news on this front?

Are you aware of any internal/external ongoing initiative?

Hello Joel! I'm not aware of any community-led contributions here yet.. if it's something you're interested in building out, do let us know &amp; we can provide guidance along the way!

Hi Maggie, azure blob ingestion is indeed a requirement for us. We are currently doing a spike on it so any guidance will be welcome.

<@U0121TRV0FL> any way we could trigger a more in-depth discussion?

<@UV0M2EB8Q> <@U01GCJKA8P9> <@U04QRNY4ZHA> maybe you have some tips/pointers?

Hi Joel &amp; Richard! Many apologies for the delayed response here - how are things going with building out Azure Blog support? If you need help along the way, we always have someone with ingestion expertise joining us at our weekly office hours on Tuesdays! If that's not an option &amp; you still need some direction, please let me know!

<@U0121TRV0FL> Hi Maggie, we already are in close contact with Tamas :slightly_smiling_face: