Hi everyone!
I just wanted to note here some bugs...
# troubleshooth
hallowed-dog-79615
07/11/2022, 12:58 PMHi everyone!
I just wanted to note here some bugs (at least that is what we think) we are finding lately in DataHub.
• Since we installed .40 version, dbt tests are not shown on datasets' entity pages. Formerly they were found under "Assertions", which now is disabled.
• We are not sure since what specific release, but our "impact analysis", the list-like lineage section available in each entity's details page, does not show nodes deeper than 1st order. We can select 2nd and 3+ nodes options in the filters, but they are not shown in the list nor the downloaded csv. Only 1st order nodes in both.
• Column lineage section is algo missing. We have some screenshots from our team from weeks ago and we could find this section under the Properties tab in a dataset's page. Now Properties tab contains different information (related with dbt even if its a snowflake object). It is dbt object overwriting snowflake in a sibling association?
thanks!
plus1 1
thanks bear 1
i
incalculable-ocean-74010
07/11/2022, 3:33 PMThank you for the report Daniel! I’ll forward this to the team internally.
cc @green-football-43791 @little-megabyte-1074
thank you 1
g
green-football-43791
07/11/2022, 3:35 PM1. we’ll look into this Daniel- thanks for raising
2. this may be because you are using neo4j as a graph database- is that possible?
thank you 1
green-football-43791
07/11/2022, 3:35 PM3. we’ll look into this as well, thanks for raising
thank you 1
h
hallowed-dog-79615
07/14/2022, 8:04 AMGabe! It seems we are using elasticsearch for the graphs, any idea about why we are not seeing nodes properly in the impact analysis?
Thanks!!!
l
little-megabyte-1074
07/19/2022, 10:26 PM
Hi @hallowed-dog-79615! Following up here - I believe we have addressed the 1st and 3rd bugs in v0.8.41. For the 2nd one - can you confirm that this is only happening with dbt “source” models? cc: @green-football-43791 - please take a look once you're back!
thank you 1
h
hallowed-dog-79615
07/20/2022, 10:51 AM@little-megabyte-1074 will have a look on .41 and check points 1 and 3.
But, regarding my 2nd point, I fear that the "Impact Analysis" is bugged in any kind of entity I can check (which are dbt/snowflake datasets, airflow tasks and Looker explores). In any of those I cannot see any order 2 or 3+ node when checking the Lineage tab (although the connections are fine in the lineage graph).
Anyway I'll update when we check v0.8.41.
Thanks!
hallowed-dog-79615
07/26/2022, 10:37 AMHi @little-megabyte-1074 and/or @green-football-43791 !
I could check our platform after updating to v0.8.41 and now can confirm that:
-Regarding point 1, seems to be solved, dbt test are there, both in the dbt-part of the sibling object and in the snowflake one.
-Regarding point 3, we still cannot see column lineage. If I check a dataset object in which we could formerly see it (and I am sure cause I have a screenshot!), if I go to Properties tab I see the following:
• dbt-related properties in the dbt object.
• nothing at all in the snowflake object.
hallowed-dog-79615
07/26/2022, 10:38 AM-Regarding point 2, stills the same. There is no difference when marking 2nd and 3rd+ checkboxes. Only 1st order nodes are listed in the impact analysis (of course I am talking about object that have 3rd+ nodes in lineage)
hallowed-dog-79615
08/30/2022, 8:02 AMHi @little-megabyte-1074 and @green-football-43791, we are now on v0.8.43 and still cannot see nodes beyond 1st order in the impact analysis.
Gabe pointed that this could be related with us using neo4j, but we are using elasticsearch.
As I pointed in former messages, urns are properly connected and listed in the lineage graph, no matter the order of connection/inheritage.
Did you make any advance in clarifying this? Is there any detail I may provide in order to help you?
thanks!
g
green-football-43791
08/31/2022, 5:37 AMIf you try to navigate to the lineage tab from one of these entities with incorrect impact analysis, are you able to navigate many layers down that way?
3 Views