Secoda Community

<@U01MHQR7DE2> we should add this to one of the upcoming cycles. Definitely confusing

I’ve added it to the backlog, thanks for the ping <@U01PM6182AX>

Can I piggyback here? "Related" resources in Dictionary terms could use a mouseover to show the table, currently it's impossible to identify them by name if the column names match

Yes, as an option for text blocks wherever you allow markup/embed

<@U02LH3NCG65> requested Latex as a block so this pairs nicely with that request <@U03PB4YJ7GW>

Screen Shot 2022-09-19 at 2.40.35 PM.png

Screen Shot 2022-09-19 at 2.07.01 PM.png

Hey <@U02QJG595KM> you’re correct, you should be able to select the sigma workspaces to choose which areas you’d like to bring into Secoda. cc <@U03113A05T6> this would be good to do on the Sigma integration for the latest changes.

Sounds great, thanks for the tip <@U02QJG595KM> I’ll keep you both posted on the progress :smile:

<@U03113A05T6> Any updates on whether this is scheduled for development? We're about to start prototyping a lot of things in sigma that we don't want brought into Secoda.

<@U02QJG595KM> Hi I’m squeezing this into the current sprint later today. It’ll most likely be wrapped up and tested by next week :smile:

Hey <@U02QJG595KM>, Sonam’s spent some time this week looking into the possibility of bringing the workspaces into Secoda as dashboard groups. Unfortunately, the Sigma REST API is quite limited and doesn’t supply the workspace information on the dashboard object.

However, we can use their GraphQL API which is much better. There are two issues with this though
1. It would require a refactor of the integration. This is ok, it just takes longer on our end
2. It would require you to input username/password for Sigma
Is point #2 ok with you?

<@U01MHQR7DE2> Point 2 is fine for us. Would this be similar to setting up login info in the integrations area?

Just here to say that I second this request, hopefully we can also get columns in for datasets through GraphQL API.

Hello Ryan.  I was trying to understand this issue as our organization is looking into DBT, but I am not sure I fully understand the problem from the Redshift side.  I can create Redshift tables without PK/FK constraints, and then add them after the fact.
```create table foo (id bigint not null);
create table bar (id bigint not null, foo_id bigint);

insert into foo values (1);
insert into foo values (2);
insert into foo values (3);

insert into bar values (1, 1);
insert into bar values (2, 1);
insert into bar values (3, 1);


alter table foo add primary key(id);
alter table bar add primary key(id);
alter table bar add foreign key(foo_id) references foo(id);```
This works fine natively in Redshift.  So then I wonder, is this more of a DBT limitation? Or am I miss understanding the problem you are describing?

Hello <@U03U3BM9TMG> :slightly_smiling_face: Your code definitely works as written, although if you omit the "not null" constraints when creating the tables, it produces the same error I'm now getting: you can't make a nullable column the primary key. Unfortunately, the "not null" constraint is not something you can tell DBT to apply when it initializes a table. Like I said, there _are_ potential workarounds, but they all kind of suck. (side note, I do hope y'all end up going forward with DBT...it's been a game changer for us)

Thanks for the feedback Ryan.  I like to understand these gotchas whenever I can, really helps out when we start implementing.

Hey <@U02P9HZGUJH> this is a reasonable request, just to make sure that I’m understanding the scope correctly, you’d have the ability in the UI to specify that a column is a primary key, as well as the foreign keys that are associated to the primary key?

Hey <@U01MHQR7DE2> thanks for getting back! Yes, that's correct. I'd love to be able to:
1. Specify primary/foreign keys from within the UI
2. Always have the option to display the columns in each table (when viewing the ERD). Sonam recently told me in another thread: _*"[Secoda] will hide that toggle, if the columns on the table do not detect any references to the Table Nodes currently on the Lineage View. This is to cut down on the clutter."*_ Personally, I'd like to have that option always available.

That’s good to know that it would be helpful. I guess the ideal situation would be to be able to filter by this value as well on the search page?

It could be a nice add on too. But a dashboard detailing % of each category + list of table in each category would be a first good step in my opinion.

<@U01MHQR7DE2> can we add this to the log? I think it’s a good idea

100%, I’ve added this to our backlog, thanks for the suggestions <@U03NJMGE4Q4>

Hey <@U03C85WHYSX> we’ve added the issue where the exported data does not play nicely with the import functionality. I’ve also added a ticket for the export functionality to only include the columns that are in your workspace. Thanks for flagging these issues

<@U026C0012KA> seems similar to what you were looking for regarding filters for “null”

True. <@U03C8DUQEHF> I think this would be a good improvement on the widget

<https://www.loom.com/share/0345504b10274d5593f09a6d99a3b98b>

Thanks <@U01PM6182AX>. The text block gives me a good option to highlight collections I want to functionally pin to the top of the list for all teams to see.

As a feature request, it would be great if I could customize this page per membership group. For example, I want my Secoda users who are in the "sales team" group to be able to log in and see the pinned collection of sales asset documentation, their most used dashboards, etc.  When a user who is part of the "engineering" group logs in, I'd want them to see the pinned collection of QA queries or maybe have a text block linking them to the video that shows where they can find important table level documentation. I don't want my sales person to log in and see a bunch of pinned technical enablement and not be able to find their sales asset documentation.

Not a deal breaker or anything, but definitely a nice to have since Secoda is really set up to be approachable to both technical and non-technical users in other ways :slightly_smiling_face:

<@U041UMTNUV7> could we add this to our feature backlog? I think i’ve heard it from a few folks now :slightly_smiling_face:

Hey <@U04C7Q1H022> which page are you referring to?

When I am working in the Catalog -&gt; Table -&gt; Columns.

You can click on a magic wand next to the column description to apply the description to similarly named columns in other tables.

Ok that's helpful, thanks Andre. I'll add this request to our backlog

You can currently drag and drop similar to excel. This may speed up your workflow

<@U01PM6182AX> Thanks for the tip on drag and drop that makes it easier. I did notice that if you drag it over a lot of rows sometimes one of the rows skips staging its changes. If I go in and edit it slightly different it will get picked up. Maybe some kind of latency/concurrency issue in the code

Happy holidays! <@U041UMTNUV7> can we add this to linear?

Thanks for the feedback Daniel - I've added this to our ticketing system for prioritization and we'll keep you in the loop

No high priority for sure, just an improvement

<@U02H00X2XV3> do you mean automatically or manually? And would the ERD just show the relationships between the associated columns?

Manually would be fine. I want to visualize few views side by side and then infer relationships.

That should already be happening automatically, do you have an example view that it's not working for. Sharing the URL would be most helpful

image.png

Hey <@U03C85WHYSX> this is great feedback, improving the usability of queries is something that we'll be focused on in Q1, so we'll factor in this feedback,

In terms of a naming convention that's more useful, do you have any examples of what you think would be a good name?

Would be great to have the query name = *name of the table(s) it's creating*, I know that for us that use one model = one table it's perfect for multiple table in one model I don't have any recommendations
Let me know if you need any more feedback or explanations

<@U01MHQR7DE2> <@U04HYM31CBV> we should add this to linear for redesign feedback on dictionary pages.

Pretty good ideas <@U04HBMDJVJB>

Glad it helps. More in my backlog… will share them when properly structured

<https://www.loom.com/share/35415e6495a54fef84b1adbd86cf7d83>

<https://www.loom.com/share/449d322ab8744fb18514208c86658533>

Oh man. Thank you. I didn't know we could add icons to tags. This absolutely solves it. Secoda is ahead of the game once again!

Hi Sharada, I replied to this in the <#C01N376S9A5|> channel

In BigQuery lineage two tables related by a query have a box in the connectiion that describe the transformation…

When clicking the small box with &lt;/&gt; the relationship is described, in this case a SQL query…

it would be great to extend this &lt;/&gt; box concept to the manual lineage so it can contain not only SQL data but python code, links to documents, external links, etc

As of now lineage is created but all context, both business and tech, is lost (it does answer the question what is related but not why is related)

Oh interesting. <@U03113A05T6> / <@U01MHQR7DE2> these are great requests

<@U03113A05T6> also thought about this feature, but we didn't include it in the initial scope for manual lineage. However, it is something we've want to build. I've created a ticket here <https://linear.app/secoda/issue/SEC-2818/describe-lineage-relationship>

Glad to have a quick meeting with you along the week if needed to share my exact view on this feature. I basically see things in terms of graphs, nodes and edges, so i have tons of suggestions… here to help <@U01MHQR7DE2> <@U03113A05T6>

Yes, this is coming in the update next week. If you remove the vertex from the table, it will delete the node relationship, but we realize this isn’t that obvious and are adding an x and an edit button

Nice, what would you want to add about “dummy” data?

For me this dummy data are ‘empty nodes’ meaning boxes I can use to ‘simulate’ the lack of integration… Example down here…

*Use Case: 1*
I have Periscope as BI tool. Periscope has Views, Dashboards and Tiles/Widgets. A View can serve n Tiles. A dashboard can contain n Tiles… a dummy data schema would be…
• Click a button, new node
• I edit that node and I manually inform minimum metadata like: 
    ◦ Name of the asset
    ◦ Type: Periscope Widget (its my type, is up to me to keep my taxonomy, not Secoda’s)
    ◦ URL: link to the widget, 
    ◦ Description: can be text or a link to some internal confluence or some internal Secoda document, Anything that help me keep the knowledge together in one single box (and being displayed inline or in a side bar, like ‘official’ data assets)
    ◦ Any other standard Secoda elements (add tag, add collections, add owners, add governance, add verified, etc)
    ◦ I should have an icon that says it is a manual, dummy node box. Maybe the ability to add an icon (I assume the layout of that box would be the same one as the official Secoda’s integration nodes (e.g BigQuery icon))
• I add another manual dummy box, fill it same way and link both together + have the &lt;/&gt; icon in the edge or vertex so i can describe there the relationship
    ◦ Eg one box is a Periscope tile and the other a Periscope dashboard, the &lt;/&gt; box would allow me to add a relationship   ‘a belongs to b’ (its my type, is up to me to keep my taxonomy, not Secoda’s) and some extra metadata (url, a description box etc). 

I would like to ~click~ hover on a table and all tables dependant from it should be highlighted in a wider view, so I can find at sight what downstream tables are related (when I click now it opens the side bar, what is good as well and you should keep it)

In the same way, if this source table affects a big bunch downstream tables, maybe not all of them belong to the same business process or the same domain or the same data meshes… having the chance of flagging the different objects now in Secoda (via related or collections or tags), it would be great that, when selecting a source table, all dowstream ones are colored depending on the business process it belongs to

if a table is involved in several business process in early stages, maybe you can have double or triple colored lines around the node, or maybe multicoloring the node border…

eg, fx rates table will affect to many business process: finance, sales, etc

Makes sense. Noted in the same ticket as the other requests :slightly_smiling_face: Thanks Daniel

Welcome. This is for sure a nice to have to boost user adoption, rather than a real, impacting feature

Great idea! I’ve created an improvement ticket for this in our backlog :smile:

Hey <@U01TLJ5NDC2> once your analytics page is working (we're currently working on a fix) you'll be able to see this breakdown.

As in like a “calendar” to schedule extractions?

<@U01PM6182AX> :Thats not a bad idea though. But what I was thinking is like a drop down list where it says Monday, Wednesday... etc

Does anyone feels the same especially from an admin perspective?

Hey <@U04HBMDJVJB> can you connect me with your Okta administrator, we'll set it up!

Thanks <@U01MHQR7DE2> I will find some slots to do that

Hey Amit! This is coming very soon :sunglasses: I'll update once it's out!

Hi.
Would be cool if you have API Endpoint to run the Monitor. That could be convenient to execute then from the ETL tool. Having the time-based schedule of the Monitors sometimes is not enough, and ETL process should be the place where everything is orchestrated, including this Data Quality layer once the data are loaded

Evening everybody, I had a nice chat the other day with one colleague about dbt semantic layer governance, andI realised that maybe Secoda will support dbt Semantic Layer queries and/or semantic yml files in Glosary section, when a Metric is created under Glossary... And as a bonus i was thinking, are you entering the Metric Tree battle as part of a wider governance scope?

Hi Daniel!
Thank you for reaching out. I’ll share your feedback with the product team and will get back to you regarding our roadmap for these features.

any news about the Metric Tree battle? you planning to have something like this?