@most-tent-28381
1. Can we ingest our data as well or does datahub only support metadata ingestion ?
Are you asking "can I call the
rest.li API to ingest data directly instead of using Kafka event"? If so the answer is YES
2. How is the metadata updated in datahub ? Is it real-time ?
It's real-time if you're updating via
rest.li API. It's near real-time in the case of Kafka events due async nature of Kafka processing
3. How is the lineage generated - do we have to manually specify parent and children relationships ?
Yes. We have internal integration with systems like Gobblin which automatically emits events for lineages. However, if such integration isn't available you'll need to do your own processing to derive the lineage
4. Is there any future plan to include statistics about the data present in the databases ?
Yes. We're actively working on that internally. Will open source that once it's stabilized.