This message was deleted.

Is this for db’s or data warehouses? It should be existing by going to the lineage -> column tab on a table

I am not sure it is what i'm looking for. I was wondering if you had a tool to map the whole db organisation like this

We do. Here’s how you access it

We got an issue with this feature, it is not available for all the tables + we can't click on Columns (it is present on the top right corner but not actionable)

CC. @lively-helicopter-98030

Hi @little-printer-16683, which integration are you trying to generate this for?

It is for a BigQuery db

Hi @little-printer-16683, my apologies about the late response, it won't be able to extract the column lineage for all tables due to a few reasons: • we currently parse the SQL queries used to generate these tables, and if we are unable to discern which table/column is responsible for creating the downstream tables, the column lineage will not exist for the particular table • if you have any sort of PK/FK relationship set on these tables, we can also determine column lineage that way 1. Are there a large number of tables where you are expecting to see column lineage? 2. If so, are these tables being generate/created/populated from BigQuery or using some tool such as DBT? 3. Are you not seeing column lineage on tables where you have Primary and Foreign Keys defined? 4. Do you have sample table creation queries for the tables missing the column lineage? Thanks 😄

Hi Sonam, no worries 🙂 ! 1. yes all of them 2. they are created through DBT + BQ 3. yes even those tables 4. yes but what do you need with them? i.e do you want some example of table creation queries without lineage?

Hi @little-printer-16683, thanks for the speedy reply! I am curious to go through some example queries generated by DBT to see if our parser is having issues with them.

We have not connected DBT yet

@little-printer-16683 Let's continue this discussion in a DM 🙂

@limited-solstice-12595 i know this question was asked two months ago, but...did the feature change since you recorded that video? I'm on Redshift and my lineage tables only show the nodes/arcs, not the column level information the way it's displayed in your video. How do I access the view your video displays?

Did you toggle the columns on?

I don't have that option. Is it unique to snowflake?

CC. @lively-helicopter-98030

Hi @adamant-forest-45768 It will hide that toggle, if the columns on the table do not detect any references to the Table Nodes currently on the Lineage View. This is to cut down on the clutter.

Hi @lively-helicopter-98030 thanks for getting back to me - I think I understand that, but if the lineage view is showing arcs between the table nodes, which I assume are foreign key connections, then wouldn't that count as a reference to another table node? Or am I misunderstanding?

Hi @adamant-forest-45768 does your Redshift Tables have PK and FK defined? When there are connections between table nodes without columns showing it’s due to queries being parsed and lineage being created from that.

@lively-helicopter-98030 No, they don't, though I can easily add that. We've been relying on DBT's docs/lineage for that information. Because we also have the DBT integration set up in Secoda, I suppose I assumed that metadata was somehow also reflected in Secoda's catalog. I'll add the PK/FKs in Redshift and see how it responds. If I still have any issues I'll let you know, but otherwise thanks again for the help!

I will make sure that our Redshift Integration is pulling in the PK and FK 👍 Sounds good 😄 Let me know if I can help with anything else, cheers!