Secoda Community

image.png

Thanks for reporting this bug <@U02F1A61UEQ>. We’ll remove those additional data sources, and patch the bug for future users.

This should be fixed now <@U02F1A61UEQ>, let me know if you’re still seeing the additional sources

The 2 middle dashboards are personal folders that I’m trying to exclude.

Ahhh, yes we missed the check that we do for whether or not it should be included. It’s a quick fix, but we’ll need to push a new version of the application so expect a fix for it out tomorrow

I just tossed a request into Notion that’s semi related.  At the end of the day, I’m trying to avoid indexing people’s Personal content

Screenshot 2021-11-16 at 14.07.17.png

Looks like some 500 Server errors on authentification

Untitled.txt

Some example request and response headers if that helps debugging.

We also had the same problem this morning around 10:22AM Paris Time (GMT+1).

<@U02HEA5UD7F> <@U02EC8R1946> <@U02EK1FNY1Y>

FYI it's still not working (more than 30 minutes now).

<@U02ER9HLT7E> <@U02MH0GH0V9> we're looking into this now. Just getting started here (8:30am) Sorry about this. Will have a fix up shortly!

Hey all, the app is back up now. We had some issues with some larger extractions on our database. We're doing a retrospective to make sure this doesn't happen again. Apologies for this guys!

Hey Alex, you’ll get an email when the extraction finishes which will have more information if it runs into an error.

We do think we can improve the experience here though, so I’d be curious what info you’d expect/like to see?

The emails are fine but I think I’d prefer a single page that lists:

• each extraction that is running or has run
• the status for each extraction (failed/success)
• time elapsed for each extraction (since being triggers for ones that a running, time to completion for complete ones)
I think dbt Cloud is a good model for what this should look like (on their homepage when logged in)

This is very helpful, thanks so much Alex!

Have you whitelisted the IP address `54.175.53.193` ? That’s where the Docs calls are made from, which is different from the IP address of where the extractions are run from.

Ah, didn’t realize the IPs were different, would be a good call out in the integrations docs

You’re right Alex, we’ll update the documentation right now!

This is likely because of a bug that happens when our search index update job is running in the background and hasn’t completed when you’re viewing your search page. I’m assuming this happens when you update a description on one of your resources, or is it just random?

I think the only thing I did was add some of the resources to a collection. And it’s not just the search page, it’s also the catalog (but I suppose both of them may be impacted by the same bug)

What is the resolution for the bug? Rerun the extraction maybe?

Note: I can still see these tables in the collections I added them to. I can even click into these tables to see details. But theres no way for me to find tables that are not already saved in the collections

You’re correct, they are both impacted by the same bug. To resolve the issue, I’d recommend clicking into one of the tables on the collection and update the description. If you wait ~1min after doing that your tables should appear in the catalog. If they do not let me know and I’ll take a deeper look into the issue

I checked our Elasticsearch indices for your workspace, and it looks like the tables are populated now. Let me know if you’re able to see them on the frontend

Yep, all looks good now. Thanks!

Also thanks x2 for being so responsive. Apologies for posting things back to back but I’m just eager to get this to a state to share with the rest of my team. I think it could be valuable in closing the knowledge gap between our team and stakeholders. :pray:

Alex, no need to apologize about posting things back to back. The feedback on the product helps us more than you can imagine, so don’t ever hesitate. Happy to hear Secoda could be valuable to your team and stakeholders!

Hey Max, did you receive my reply on Intercom? If not I mentioned there that what you’re seeing is a UI bug and that you’ll receive an email when the extraction succeeds or fails. If it still is showing the timeout error let me know and I’ll take a closer look at the issue

I took a look at our logs, and it appears that the extraction ran successfully. Let me know if you see the new tables on your end

All set. Thanks for your quick assistance on this.

Hey Vishu thanks for reaching out, did this behaviour occur multiple times, or was it a single occurrence? Or put another way, were you able to successfully edit any table documentation?

This happened twice on a single table documentation. Once it saved for real never had an issue.

You’re correct <@U02LVDYD1AS>. When we implemented email verification it seems to have broken this capability, thanks for sharing the issue with us. In the meantime if you want to invite people via the “Invite Teammates” button on the side bar, you can add a role/group and it will work as expected

hmm, we thought we solved this bug. We're looking into it!

Screen Shot 2021-11-26 at 4.43.38 PM.png

Hi Milan, based on your error message it looks like the project name is incorrect. You can find the project name in the red box that’s shown in the screenshot

Awww so its the ID name not the project name, let me try now.

Team, I am looking to create a data dictionary, catalog and willing to give Secoda a try.   I am working on a project with BCG/Bloomberg and would like to demo this product with the client.   Do you have any tutorials, trainings or best collections I can use as reference?

<@U01MHQR7DE2> I was able to connect, however while building queries I get the following error:  *403 Access Denied: BigQuery BigQuery: Permission denied while globbing file pattern.*

That’s great to hear it worked well, for BigQuery you need to make sure you’ve created a service account with the following roles:
```BigQuery Data Viewer
BigQuery Metadata Viewer
BigQuery Resource Viewer
Job Viewer
Logs Viewer
Private Logs Viewer```
In terms of trainings <@U01PM6182AX> would be best to provide getting started tutorials and other good learning materials. Perhaps you could even book a demo with him?

<@U02NRF8ASKU> I can definitely share some information with you about how to best use the product as well as demos / tutorials. Here's my calendar to walk through how you're hoping to use the product so I can help you demonstrate it: <https://calendly.com/etai-secoda/rolling30>

Thanks <@U01PM6182AX>, I set up time for 0900 Monday, I need to demo to the client on Tuesday, so a quick turnaround.

<@U01PM6182AX> any additional training videos you can share prior to Monday?

Looking forward to showing you around. This may be helpful in the meantime: <https://secoda.notion.site/Secoda-Overview-145abc9ec6f14d01afeef27a0d9197ec> (although there's some new features that aren't highlighted in this yet)

Aww man, this is nice, I present to Bloomberg/BCG on Tuesday (Bloomberg is the Client), this data catalog/dictionary is the missing piece.  Love it.

Thanks for letting us know Vishnu, we’ll get a fix out today for this bug

Can you send a screenshot of the results? Is it not ordered correctly?

There are 111 tables and when I searched with schemaname.viewname

The search option did not show the required view result

Is there any chance that there is a column name within those tables that has "schemaname" or "viewname" in the title?

nope, we don't have column names like that

Check out the recent post in <#C01PZJDBJ6M|>. We've found the bug for this and have fixed it

Hey Alex, thanks for sharing this. We’ve just pushed a fix for this bug. Please let me know if you see the query now

Hey <@U02NN9PTP71>, it's been fixed. Sorry about this issue

confirmed i'm back in :slightly_smiling_face:

<@U02LVDYD1AS> &amp; <@U02HEA5UD7F>..<@U01MHQR7DE2> is looking into this right now. We moved some of the infrastructure and it's causing this error with the extractor.

Hey <@U02LVDYD1AS> and <@U02HEA5UD7F> thanks for bringing this to our attention. The problem with the scheduler has been resolved, and you should see your daily extractions show up today. There are lots of extractions queued up, so it might take a few hours but they’ll be there today

<@U01MHQR7DE2> It looks like my extractions have not been running again (last ones were run manually, 12/3)

Hey <@U02LVDYD1AS>, just to confirm is it the Metabase and dbt integrations that aren’t running? It looks like the Postgres and Redshift integrations are running regularly now. I just wanted to confirm because it’s likely a specific issue with those two integrations.

Metabase, Redshift, and dbt don’t appear to be running on daily (based on the “History” tab)

Oh maybe it was a UI thing? The Redshift extractions just showed up

Metabase and dbt still don’t appear to have run though

It was likely a UI thing. I’ll look into the Metabase and dbt extractions now.

Screen Shot 2021-12-07 at 11.36.24 AM.png

Tableau is also not extracting by the looks of it

Screen Shot 2021-12-05 at 9.36.46 PM.png

Thanks for the additional context <@U01TLJ5NDC2>. Two updates on this:

<@U02LVDYD1AS> we’ve found the Metabase / dbt issue and will be shipping a fix shortly.

<@U01TLJ5NDC2> we’ve also fixed the issue you’re seeing with Tableau which will be in the next release. We’ve also revamped the extraction UI to give you more insight into what’s going on, which will also be in the next release.

<@U02LVDYD1AS> we’re currently blocked on deploying the fix due to an AWS outage, Secoda is still working but it’s effecting our deployment pipeline <https://status.aws.amazon.com/>

Ooof not a good day for AWS :grimacing: thanks for the update!

Hey Alex, based on the error shown it does look like it's a whitelisting issue. You’ll have to whitelist 35.175.75.15 for your database

<@U01MHQR7DE2> Still debugging this. It still does not work after whitelisting. Is this connecting over SSL?

This was on our end, apologies about that. We’ve been able to fix it without having to redeploy. Let us know if it’s working now for you!

sent a tweet to my tiny following in support of all the hard work you all do :heart:

Let me take a look quickly Alex, and I will let you know when I’ve found what the issue is

<@U02PBHLUDU6> could you provide some details on what the inputs were when you tried to send the invitations? How many emails did you input, what type of user, and were there any groups associated with them?

I’m having trouble reproducing the issue, so any details would be helpful.

Thanks for following up. It appears to be working fine now. :shrug:

You’re correct Alex, we’ve fixed this issue as well and it will also be in the next release

Hey Gerald. Sorry about this. A couple of us are debugging this right now and just need <@U01MHQR7DE2> to deploy the new version. Unfortunately, he's afk at the moment (11:40 PM our time) so a fix will be pushed as soon as he's back online

Hey <@U028E7B1U31> this has been fixed now, apologies about the delay!

You’re correct Alex, thanks for pointing this out. We’ll fix it over the weekend

Not urgent just noticed this is still showing up this way. (Also happy new year <@U01MHQR7DE2>!)

Happy new year <@U02LVDYD1AS>! Thanks for bringing this back up Alex. I will let you know when it’s resolved

It’s taking the date and trying to localize the time to the user’s current timezone. We did this to make it more readable to non-technical users but perhaps it would be good to provide a setting that disables this formatting

Apologies about this one Kevin, you’re correct and we’ll fix it for the end of today

This one is also fixed now <@U02739P8464>!

Hey Kevin, <@U02GZEAV18D> will be working on this issue and the one that you mentioned yesterday. I’ll update you when they are fixed. Apologies about that

Tackled it now. I've added it to the deployment scheduled for later today.

Hey <@U02739P8464> this issue is fixed now!

Hey Alex, for now I’d suggest following this process to link questions. <https://www.loom.com/share/ca4f3f744a6c486199ce7171858eb3cd>

In the future we’ll be adding linking by doing @question title similar to how you link dictionary terms. That will be coming early in the new year when we improve the functionality around linking resources.

Got it, that works for now. I guess what I was getting at is that it would be nice to see all the questions that referenced a table/model by tagging it. Is that within the scope of the linking you’re talking about?

Hey Julie thanks for reaching out. Let me take a look at this for you. I’ll keep you updated on what I find.

Thank you! If Secoda is observing new year's day today, this can absolutely wait until later

We’re back to work today! So I’ll submit a patch after my meeting from 3:30pm - 4pm EST and let you know when it’s live

The issue is a null reference for a python dictionary term, so a mistake on our end

Hey <@U02QJG595KM>, this should be resolved now. If it continues to have issues please let me know and I’ll take another look!

Hey William, happy new year! Which piece of functionality are you looking to leverage for the Slack integration. Are you trying to use the questions feature? The reason I ask is because for some pieces of functionality, like the questions feature, you need to allow Slacks servers to communicate with Secoda inside of your VPN.

Was just going to test it out and see what offerings they are, we arent using questions right now

It’s likely that there’s a connection issue on the installation for Slack. Because the requests goe Secoda -&gt; Slack -&gt; Secoda through the standard oauth2 flow. On the third step the VPN is probably blocking the request from Slack.

I’ll try one quick update that might fix it, what’s the url that you’re using for Secoda <@U01TB6G50RY>?

Hey Benoit, could you provide the name of a table that should have documentation but does not in Secoda. I think the issue is that there’s an existing blank documentation field and the extractor does not want to override that. If that’s true then we can push a patch that fixes this issue. But I wanted to confirm first that is the issue.

After removing the BQ integration and running another extraction with the dbt integration, did it bring in the descriptions?

`dim_business` is an example. Removing BQ integration did not help. Note that both the table description and the columns descriptions are missing from the prod tables, and both are there on the preprod table.

Okay I will investigate now and let you know what I find

There is a blank description like I expected, I’ll push a patch so that this works properly <@U02HEA5UD7F>

blank description in both the table description and the fields description?

No it’s on our end. I’m fixing the bug now and will let you know when you can run an extraction that will work properly.

<@U02HEA5UD7F> this should be fixed now if you run the extraction again

just tested right now, and it seems to be working, everything looks documented :slightly_smiling_face: Thanks a lot

Hey Mohan, thanks for reaching out. We’ve temporary removed the magic wand due to a large issue but we should have the feature back in before the end of today cc <@U02GZEAV18D>

Hi Mohan, I've prioritized getting this resolved. It will be in production shortly.