https://datahubproject.io logo
Join Slack
Powered by
This is the image
# ingestion
l
This is the image
h
Hey can you paste the image in the thread of earlier message instead ? thanks.
hmm.. are there any errors/warnings in ingestion ? 1. could you share the urns for two datasets -> with name 
t_alphagen_marketaxess_maagbondticket
2. Is the snowflake table corresponding to doe_edp_stg_db.alphagen.
t_alphagen_marketaxess_maagbondticker
and 
t_data_registration_catalog
not ingested during snowflake recipe ?
l
Not its corresponding to doe_edp_db.alphagen.
t_alphagen_marketaxess_maagbondticker
Unable to see the dbt snapshots
Its not the problem with snowflake table that had been ingested but snapshots are not getting up in the lineage. Same problem was with 0.8.43
@hundreds-photographer-13496 can you please tell me what can i do to get all the lineage properly
h
I am not sure yet. In which CLI version was the lineage working as expected for you ?
l
0.8.41
h
so using 0.8.41 CLI version for ingesting dbt and snowflake still gives lineage as expected ?
l
Yeah in one of my instance still i am using 0.8.41 to compare the lineage with latest version
h
can you ingest dbt using 0.8.41 and snowflake using 0.8.45.1 and confirm if its working as expected. Just trying to find the root cause of unexpected behavior - whether its dbt or snowflake source.
l
Yeah can test
h
is I1 the idea lineage you expect ?
l
Yes it’s showing all the dbt transformations
But with the latest version(0.8.45.2) unable to see all the dbt snapshots
@hundreds-photographer-13496 any suggestion?
h
Is snapshot dataset ingested at all (though not in lineage) when using 0.8.45.2 dbt version ? If yes - can you compare its urn with older version ?
l
I am following the same process and files are also same in both the ingestion but in 0.8.41 everything is perfect but with latest version issue is there
h
Got it. Can you confirm if snapshot dataset ingested at all as part of ingestion ? I am not asking about lineage page but on datasets search page ? I suspect if this issue is due to casing of generated urns.
l
I checked the urn is same in both the cases
We are not ingesting the snapshots separately we have pipeline designed in a way that has do all the transformation. Just need to trigger our dbt ingestion cron job
g
What is the full urn for that dbt node that doesn’t have upstream lineage - 
t_eng_msci_globalsanctions
?
l
@gray-shoe-75895 This is the urn of t_esg_msci_controversies table
We are not getting dependencies such as: t_raw_esg_msci_controveries , t_stg_esg_msci_controversies and t_stg_raw_esg_msci_controversies
Hey any suggestion on this dbt lineage
@hundreds-photographer-13496
Open in Slack
PreviousNext
Powered by