https://datahubproject.io logo
#integrate-iceberg-datahub
Title
# integrate-iceberg-datahub
r

red-lizard-30438

04/26/2022, 5:38 AM
Hi Team, I am looking for Iceberg solution in Datahub and ingest metadata from Iceberg. I came across this channel, so wanted to know does we have working solution in Datahub? How can we integrate and please share the integration document.
m

modern-monitor-81461

04/26/2022, 3:48 PM
Hi @red-lizard-30438, the Iceberg integration is not quite ready for primetime yet, but we are close to have something incubating. May I ask how you store your Iceberg tables? (S3, Azure, Local filesystem?)
f

future-byte-46314

04/26/2022, 9:17 PM
Sorry for hijacking this thread. We are also looking to integrate datahub with lakehouse people building in our org. I believe they are looking to store iceberg tables on s3. Hope it helps.
m

modern-monitor-81461

04/26/2022, 9:33 PM
@future-byte-46314 My code only supports Azure datalake (Storage Account) right now since I do not have access to an AWS env (or S3). I know that the underlying Iceberg Python API supports S3, so it should be fairly simple (of course we all know what this really mean!) to make my code work with S3.
thank you 1
f

future-byte-46314

04/26/2022, 9:50 PM
Thank you Eric! 🙏 We will provide our testing results from s3. It is expected to happen somewhere in the end of Q2.
b

bitter-noon-86800

04/27/2022, 11:21 AM
Also hijacking 🙂 We use iceberg with s3/minio in k8s
r

red-lizard-30438

04/27/2022, 3:03 PM
We use Iceberg with S3 and Local file system
w

wide-optician-47025

03/21/2023, 4:04 PM
hello, we also use iceberg with S3