Hi - Is it able to use datahub to visualize the do...
# getting-started
e
Hi - Is it able to use datahub to visualize the downstream consumers of a kafka topic? For example in the demo https://demo.datahubproject.io/dataset/urn:li:dataset:(urn:li:dataPlatform:kafka,SampleKafkaDataset,PROD)/Lineage?is_lineage_mode=true . how is datahub knowing that Baz Chart 1 and SampleHDFSDatabase are built on using SampleKafkaDataset ?
g
Hey @elegant-machine-39016! How are you consuming the Kafka topics? If you use Kafka Connect for example, DataHub has a connector that is able to automatically infer lineage edges: https://datahubproject.io/docs/metadata-ingestion/source_docs/kafka-connect
e
@crooked-market-47728 So in our enterprise the consumers will be consuming the stream (topic) in multitude of ways -- kafkacat, kafka-consumer, some java/python etc. apps, kafka connect, and others. We'd like data hub to show the lineage of a topic like which are all the consumers consuming from it. Is this possible?