Secoda Community

image.png

It's likely a problem with the on prem setup. Let me take a look at your logs

Thanks for letting us know Fred, we will take a look at it today and have a fix in the next release on Sunday. I'll update you when I've found the issue

Thanks for pointing this out! We're releasing an update this afternoon and a fix for this issue will be in there

We're so sorry, looking into it as I type this!

Hey <@U0289RK09ST>, the application is back up. Apologies about that

<@U02AGDQRZ19> this is now available in the "Questions" section of the application

Screenshot 2021-08-11 at 14.42.26.png

Thanks for pointing this out <@U0289RK09ST>. Two questions:
1. When you reload is the owner added?
2. Do you have the Slack app enabled?

Hey Sarin, would you be able to confirm if this has been fixed or not. Thanks!

When is the next release expected? And how often are updates done to the product?

We do weekly updates so the next release will be out this weekend. Although we do release patches for bug fixes throughout the week

In your dashboard (workbook), what are you running queries on? Right now we pull table information by parsing the SQL from the queries and from customSQL tables defined in Tableau.

BigQuery. I am able to see the underlying query used when a SQL query extract is used for the dashboard, but the tables section is still not populated.

The lineage step is likely the step of the BigQuery extraction process that is running into errors, which would cause this bug.

Can you try manually running an extraction on your BigQuery integration. We pushed some updates over the weekend and you can test if that resolved the issue

Neither do I see the, upstream and downstream data for the tables.

This is pretty important for the demo that I want to present internally

Okay let me prioritize this today and make sure it's working for you by tomorrow at the latest. Apologies about the delays

Hey Sarin, we've just pushed a change to fix this issue. Please let me know if it's been resolved. Thank you for your patience. If you need your free trial extended please let me know!

Thanks a lot Andrew.! Yes the extraction was successful and I can now see the downstream/upstream details.
On another note, what’s the best way to interpret this data?

That's great that it's working now. Here's how you can interpret the data through two examples:

If you have two tables table A and B, and B is downstream of A that means that table A was used to make table B through a JOIN for example. And then there is a depth number which means how many JOINs the table is away.

Another example with table A, B and C. Where A is used to make B and B is used to make C. If you were on Secoda and looking at table A then B and C would be downstream of A where B has depth 1 and C has depth 2.

Thanks for the example. Say if I’m viewing table A in the catalog, B would appear under downstream right? Please correct if I’m wrong

Ok.. so I feel like details shown is opposite of what you described for the BQ integration I created. We have Table A which which is then used to create B. However B shows as Upstream and not Downstream.

Also, please extend the trail period again. Thanks :pray:

Sorry Sarin, you're correct it is the opposite of how I described it. Even I get it mixed up sometimes :sweat_smile:

What's the name of your workspace again, so I can extend the trial?

I've extended your free trial by a week Sarin

Screenshot 2021-08-17 at 08.28.54.png

You're correct, I will modify the Redshift integration to also extract the views. I'll circle back on this thread when it's been completed Rodrigo.

Is it supposed to appear similar to Postgres where it shows if the extraction was successful or not?

Hi Krizi, can you refresh and try the extraction again. The most likely reason why it's failing and not showing why is because there is an invalid username and password being used, or your session token has expired on Secoda (refresh would fix this).

But either way you're correct that this should be showing an error of why it's not extracting, which is a bug on our end.

Hi Kriza, I can see some errors coming in from your extraction attempts. Our Metabase integration makes the assumption that your Metabase dashboards are organized by groups, and it looks like there's an attempt to get a dashboard without a group which is causing an error. I can fix this issue so that you're able to perform the extraction. Apologies about that problem. And I will circle back here when it's fixed

Thank you, Andrew! Is there a limitation on the Metabase user where it has to be an admin? Another possibility might be an issue with SSO. I logged in Metabase using the Gmail account to confirm I have the correct credentials and I was able to proceed.  I've been trying to connect using an account with access to just a few tables, but I am unable to extract data

Thanks for the additional info Kriza, that would be the problem. It assumes that the user is an admin and has permission to view all of the resources in Metabase. When I update the integration I'll remove that assumption so it works for users that only have a few tables

I see, got it. Thank you! Also, a suggestion for deleted integrations, can we exclude them in this view? I also tried renaming the latest integration but it still shows Postgres

Yes that's a bug on our end. Thanks for pointing it out Kriza. We will let you know when it's been fixed

Thanks for letting us know Will, which page is this on?

Okay thanks, I will let you know here when the bug is fixed

Prety sure I had this same bug on the demo - wasn't able to change bezos, musk or myself as the owners. After multiple clicks it seemed to work sometimes. Didn't see anything in the console or network that was telling about what the error might've been!

It is back and forth for me tbh, recently was working again then started bugging out

Thanks for pointing out these bugs Chris! We'll be fixing them asap

the collections tags lists seem to eventually dedupe.

Screen Recording 2021-09-22 at 5.43.45 PM.mov

Thanks for pointing this out. It looks like it's a little sensitive, so you have to make sure you're pressing on the right spot to add the group. I've added a video of myself going through this. As you can see, depending on where I'm clicking on the group box it will either add the group or not. We'll change this so it registers that you're adding the group nomatter where you select on the box, but you should be able to add a group if you give it a couple shots.

Thanks for pointing this out Chris, the resource count should only be what's indexed in your catalog. So if you exclude schemas it will be excluded in the resource count. We'll get a fix out for this asap

Screen Recording 2021-09-29 at 10.08.58 PM.mov

Also experiencing this - we’re planning on evaluating Secoda this week, so hopefully this is temporary?

Hi there, this is definitely a bug on our end and will fix it asap this morning. Thanks for pointing this out. I’ll let you know when it's been resolved

<@U02FR9DEVFG> <@U02G68D73UJ> this issue has been resolved. Thank you for flagging it

<@U01MHQR7DE2> this bug is back for some reason. <@U02G68D73UJ> are you experiencing the same issue?

I'll push another fix asap. Thanks for flagging this

I was just able to reproduce it on my end. But then I refreshed the page and it was working okay again. Can you see if it's the same for you <@U02FR9DEVFG>. Either way, I'll have to dig to see why it's sometimes happening but I just wanted to see if it was still happening for you

Ah yes ok refreshing the page resolves the issue.

Very strange, if you can reproduce it again let me know. There's clearly something causing it I'm just having issues reproducing it locally

I am experiencing this issue now trying to create any Dictionary term. Says Something Went Wrong, but get an empty term created in the list view.

We're working on a fix for this issue <@U02CMV9BNP5>. As a temporary workaround, navigating away from the page, refreshing, and then navigating back to the dictionary term should resolve the issue

Hi Mark, you need to whitelist the IP address `54.175.53.193/32` on your Postgres database. I missed updating the Postgres documentation here.

The extractions are run on a different service than the analysis query engine, which is why you need to whitelist both addresses.

Perfect. Works now. Thanks. Figured it was something like that.

*More details:*  I did the change on Microsoft Excel on a Mac. When I edited the csv in a text editor, it then uploaded fine

Thanks for pointing out this issue <@U028E7B1U31>. I'm curious about the use case you have for this feature? I know people who are migrating from Amundsen have used it but would love to hear your use case

We started building up the dictionary on a spreadsheet before we got secoda. Once we move everything to secoda, we'll manage the dictionary in secoda

<@U01MHQR7DE2> I tried to bulk update the description via this method but it seems that I need to at least have something in each of the column description before I can bulk update them via the import feature. What’s the best way of bulk updating descriptions?

You're correct, the bulk upload was designed for people migrating from Amundsen. What I can do is make the upload functionality more intelligent and try to automatically match the resource description you're uploading based on column/table name.

If you could send a sample to <mailto:andrew@secoda.co|andrew@secoda.co> of what your spreadsheet looks like I can see how much effort it would be. How many descriptions are you migrating over?

Let me take a look into this Chris, I'll get back to you in a couple of minutes once I figured out the issue.

<@U02F1A61UEQ> which user is it to help me troubleshoot

I think I figured it out.  I added a default share setting

That's great that you found the workaround. But I'll still need to get to the root cause so that this doesn't happen in the future.

Do you know if the viewer refreshed after joining the workspace? I want to make sure that it's not an issue with the invitation flow before diving into sharing.

In her case, I had given her the invite link.

May be worth taking a few minutes to make sure I have things set up sensibly.

That may be the issue (bug on our end). If it is I'll let you know. But it would be great to make sure things are setup sensibly. Feel free to book a time tomorrow or early next week and I can offer any pointers. <https://calendly.com/andrewmcewen/30min>

Screenshot 2021-10-08 at 11.01.18.png

Hi there. We have the following error when trying to export data from Redshift:
```psycopg2.InternalError) Result size exceeds LISTAGG limit DETAIL: ----------------------------------------------- error: Result size exceeds LISTAGG limit code: 8001 context: LISTAGG limit: 65535 query: 2102954 location: string_ops.cpp:116 process: query0_117_2102954 [pid=28836] ----------------------------------------------- [SQL: WITH queries AS ( SELECT query, LISTAGG(CASE WHEN LEN(RTRIM(text)) = 0 THEN text ELSE RTRIM(text) END) WITHIN GROUP (ORDER BY sequence) as query_statement, COUNT(*) as row_count FROM stl_querytext GROUP BY query ORDER BY query desc ) SELECT DISTINCT(sqt.query_statement) as query, CURRENT_DATABASE() as cluster FROM queries AS sqt LEFT JOIN stl_scan ssc ON ssc.query = sqt.query LEFT JOIN svv_table_info sti ON ssc.tbl = sti.table_id WHERE sti.schema IS NOT NULL AND sti.database = CURRENT_DATABASE() AND starttime &gt; DATEADD(hour, -24, getdate()) LIMIT 1000 ] (Background on this error at: <https://sqlalche.me/e/14/2j85>)```

If you update the application by running the update script (`update-secoda.sh` ) this bug will have been fixed

after running script and updating .env I have
``` 502 Bad Gateway```
are there maybe some logs I should check out in case of any problems?

Are you able to see the login screen at all?

Sometimes it takes a couple of minutes to start up after a restart. How long ago did you restart it?

I have similar problem to this one
<https://secodacommunity.slack.com/archives/C01PT4WP0TE/p1629957842000200>

can you please support with it?

Yup, this is due to an authentication error. Are you using Google Oauth on Metabase or just standard login?

Oh wait, this is due to a configuration issue with your self-hosted deployment. Let me take a closer look.

You'll have to change the `DOMAIN`  variable in `.env` to be the IP address/domain of your installed instance of Secoda. Then when you reload the Metabase integration should work as expected <@U02FSG2JZ39>

Redsfhit is working, but we still have the problem with Metabase
<@U02FSG2JZ39>

Hey <@U02F6LCE0MB> I'll take a look into this tomorrow. Thanks for letting me know

Screen Shot 2021-10-12 at 10.19.57.png

Screen Shot 2021-10-12 at 10.18.38.png

we have another two problems
• views are not showing lineage
• we cannot upload logo images
cc <@U02FSG2JZ39>

Apologies about the delay here. I’ve added these items to our backlog which shows when we’re working on a particular item that you’ve flagged. We’re tracking the bugs and improvements that you’ve both identified here <https://www.notion.so/secoda/Klar-mx-feedback-10139b22b2694f749c95eb092370e488>.

The image upload issue and Metabase issue are both something I’m looking into right now.

The redshift lineage and popularity is more tricky because information on views are structured differently in Redshift’s system tables so it will require some work to get that information into Secoda.

<@U02F6LCE0MB> <@U02FSG2JZ39> the image upload issue has been fixed!

You’re right <@U02F1A61UEQ>, we’ll update that. Thanks for flagging this!

<@U02F1A61UEQ> I hit this error too. We're looking into why it happened

<@U01PM6182AX> Had the same one momentarily

<@U02F1A61UEQ> we’ve found the root cause of this issue and it’s been resolved so you won’t be experiencing this anymore!

also deleting one ending up deleting all the duplciated ones as well :disappointed:

so we lost some documentation for these collections :confused:

We've just sent you a message about how to recover these!

Hi Aaron thanks for letting us know. If you check back in your catalog are all your tables there? Also if it’s only loading some tables, that usually means that the user you used to connect to Snowflake doesn’t have permission to run `SELECT` statements on that table. You can view our documentation to setup permissions in Snowflake <https://help.secoda.co/en/articles/5013624-snowflake|here>

Thanks Andrew - they are back in the catalog so it all seems to be working as of now.

That’s great, if you have any other issues please let me know!

This is a bug on our end Chris, we’re attempting to convert a string to lowercase that doesn’t exist. We’ll let you know when we’ve fixed the issue

:thumbsup:

Not sure if its related, but I’m not seeing explores/views in the Catalog at the moment.  I do see dashboards

This is 100% related to that issue. We have <@U02F2A1UY20> looking into it now, and will hopefully have a fix out before the end of today

Hey Kevin, we’re just looking into this issue now. I’ll get back to you in a couple of minutes when we’ve found what the problem is.

<@U02LH3NCG65> The reason why it’s not working is because we have a max file size in our server API calls. Do you know the size of your manifest.json so we can increase the limit

We’ve increased the size of accepted files, you should be able to upload it no problem. Make sure to refresh your page if you have had it sitting idle. Let me know if it works now!

You’re correct <@U02739P8464> apologies about that. We’ll get a bug fix out today for that issue. Thanks for bringing it to our attention