Hi all, I'm evaluating datahub for a POC and had a few questions about how different metadata revisions are reflected in the UI. All of the questions assume we're directly hitting gms and synchronously emitting metadata to datahub (not pulling/crawling any services):
• At t=0, I push the following lineage metadata: A --> B
• At t=1, I push A' --> B (with A no longer existing/deleted from the system)
B's authoritative source (from the POV of the code, not necessarily the data warehouse) at t=1 is now A', but the edges A --> B and A' --> B will both be reflected in the UI, and I only see the option to filter lineage by time (at the granularity of days).
Does datahub have a concept of a particular "version" of the world / the ability to filter entities in the UI by that? Trying to think about how to reconcile the "CI-time" view of our data system where we know the full DAG of pipelines, vs the "runtime" view where a dataset may not materialize until some time after deploying pipelines