Secoda Community

Hey <@U02QJG595KM> thanks for reporting this. Are all of the users that are having this experience editors/admins?

It is missing for 2 editor/admin users, but visible for a different editor

Hey Julie, just following up on this. Could you DM me the emails of the 2 edit/admin users that it’s not working for please

So it appears to be working now for all users. This might be resolved?

We did push a fix, and I was about to update you <@U02QJG595KM>!

Hi <@U032MHAQVMJ> - I'm taking a look at this now. I will send you a DM.

Hey <@U03JXMCBC2X>, you can reach out to me at <mailto:etai@secoda.co|etai@secoda.co>. We're happy to show you what a complete workspace looks like and how you can best use Secoda. Here's my calendar if it's easier: <https://calendly.com/etai-secoda/rolling30|https://calendly.com/etai-secoda/rolling30>

```Dbt metadata extractor: 
'data_type'
Traceback (most recent call last):
  File "/app/databuilder/job/job.py", line 75, in _launch_job
    self.task.run()
  File "/app/databuilder/task/task.py", line 80, in run
    record = self.extractor.extract()
  File "/app/databuilder/extractor/database/dbt/dbt_metadata_extractor.py", line 146, in extract
    return next(self._extract_iter)
  File "/app/databuilder/extractor/database/dbt/dbt_metadata_extractor.py", line 321, in _get_extract_iter
    tbl_columns: List[ColumnMetadata] = self._get_column_values(
  File "/app/databuilder/extractor/database/dbt/dbt_metadata_extractor.py", line 487, in _get_column_values
    col_type=manifest_col_content["data_type"],
KeyError: 'data_type'```

Hey <@U03HF9AAJ72> thanks for sharing the logs. Could you try re-running your dbt cloud extraction now. We’ve pushed a fix for what’s shown in your logs. Let me know if it works for you now.

<@U03HF9AAJ72> are you part of the Too Good To Go team? If so we’ll need to coordinate with your devops team so they get the software update as well

Hello Matt, do you happen to know the name of the lost document by any chance?

Hello Kash, yes it was something close to "SSO Python Connection"

<@U02739P8464> did this document get renamed/deleted? We’re not seeing any history of the document based on the name that <@U0324KSPYFR> has sent.

None of us deleted it. I think that it got dropped somehow after an update. We didn't notice it until recently so not sure when exactly that happened.

How long ago did you create the document, roughly?

Was it a dictionary term originally? I found a deleted dictionary term named “Connect Via Python SSO” in the database that I restored. The URL for it should be: <https://app.secoda.co/dictionary/8d56630c-327a-4b62-81b3-cd21eb3831e1>

That's the correct one! Yes I must have created it as a dictionary term in the first place. At some point we moved all of our other articles into Documents and this one got left out somehow. Thank you so much for your help!

Hey <@U03H5HV3BQE> let me take a look at this for you. I should have an answer shortly

There’s an exception thrown on our query parser that’s not being caught <@U03H5HV3BQE>. It’s a quick fix that should be resolved shortly. I’ll update you when it’s completed.

Hey <@U03H5HV3BQE> this should be working now. Let me know if that’s the case for you

It’s running now <@U01MHQR7DE2>, I’ll update you it finishes.

<@U02739P8464> which schema are you getting the 404 on?

All of them... to replicate.. click on a schema in the catalog panel and then try to click on a schema name in the table.

It looks like it’s setting the id as `undefined`. We’ll push a fix today. Thanks for reporting Kevin

<@U02739P8464> a fix has been pushed. There’s still an outstanding issue where the schema doesn’t show all of the tables associated with the schema but it doesn’t throw a 404 anymore

Hey <@U01TLJ5NDC2> that sounds like a React rendering issue on our end. <@U03113A05T6> this was fixed in v3 wasn’t it? If you could DM the fix I’ll cherry pick it for v2

Hey <@U01TLJ5NDC2> this is fixed now, so the next time you run the update script it will be resolved

Hey <@U03M9F8GP8X> dbt is considered a “secondary” data source. We only pull the tables from the place they actually are persisted to such as BigQuery, Snowflake, Postgres, etc. The purpose of the dbt integration is to decorate those tables with metadata around lineage, descriptions, etc. Let me know if you had any questions about that.

Ah, I see now that if I click on the main "catalog" menu option, I see it; I was expecting a dbt section underneath it like I see for Snowflake. All good; thanks!

That’s great, if you have any other questions let me know Amanda

Unrelated question - I've set up my Slack integration, but am having trouble connecting a collection to a Slack channel - the "pick a channel" dropdown doesn't show any options, and if I try typing it in, nothing happens. Am I missing something there?

Have you connected the Slack integration from your settings (<https://app.secoda.co/settings/slack>) <@U03M9F8GP8X>?

<@U02N4NXTTJR> is there anything that <@U03M9F8GP8X> would be missing that would be causing the Slack channels to show up blank?

Can you try visiting this link and re-authorizing <@U03M9F8GP8X> <https://slack.com/oauth/v2/authorize?client_id=1692512662368.1977698162288&amp;scope=chat:write,chat:write.public,commands,incoming-webhook,channels:read,groups:read,im:read,mpim:read&amp;user_scope=|https://slack.com/oauth/v2/authorize?client_id=1692512662368.1977698162288&amp;scope=[…]-webhook,channels:read,groups:read,im:read,mpim:read&amp;user_scope=>

We’re going to push a bug fix, and that should resolve the issue. I’ll update you when that’s available Amanda

<@U03M9F8GP8X> We just pushed an update for this, could you try integrating slack again please?

Still not cooperating - I've disconnected/reconnected, logged out/logged in, a couple times

Trying a different browser now
ETA: that didn't help, either

Looking at our logs, it seems like Slack keeps blocking our access to read the channels list and reauthorizing it doesn’t seem to help.

cc: <@U01MHQR7DE2> When we ran into a similar issue before, we had to go to Slack’s admin page (<http://slack.com/admin|slack.com/admin>) for all the installed apps and then click on “Configure Apps” on the sidebar. Then in the list of apps, find Secoda to see if there are any alerts from Slack indicating they are blocking Secoda (usually in the form of a yellow banner)

Gotcha; I'll check in with our Slack admins

Update - I have one of our Slack admins working on this; I disconnected and had her redo the connection and it now shows some, but not all of the channels in the dropdown. She's going to keep working on it, but may reach out to you by email, <@U01MHQR7DE2>

<@U03M9F8GP8X> are the channels not showing, private channels? 

Hey <@U03C09NL529> I’ve DMd you in our other chat

(I can scroll down, but not to the right)

Hey <@U03M9F8GP8X> that’s a bug on our end, we’ll have to update the iframe so it can scroll horizontally

Checking in for an update on timing for this, please!

Let me see if I can quickly fix it <@U03M9F8GP8X>. I’ll update you in ~30 mins if it takes longer

<@U03M9F8GP8X> do you have an example of a document that contains an embed that doesn’t scroll horizontally

(Though I suppose it depends on your monitor size/settings - I can see the whole thing if I reduce my browser zoom enough)

(I thought the `*` option might do that, but it doesn't seem to have worked)

Hey Amanda, could you send the name of the document please

Just to confirm, are you not able to see the complete list of channels on the dropdown? Are all the channels that you want to see “public” channels?

Looks like we're only seeing about 40% of existing channels

<@U01PM6182AX> Would love to get them setup today. What would you suggest?

Hey <@U03M9F8GP8X> <@U03NEUQRJV7> I’m seeing ~200 channels that were pulled from your Slack. Do you have an example of a Slack channel that we aren’t pulling?

There are 11 `analysis-*` channels that don't show up in the dropdown

Hey <@U03M9F8GP8X>, the issue is that Slack defaults the endpoint to return only 200 channels. So we’ll increase that and push a fix for it. cc <@U02N4NXTTJR>

image.png

Hey <@U03M9F8GP8X> we’ve updated the API call to be paginated but it doesn’t look like the `analysis-*` channels are included. Can you send a screenshot like the following of the info about a channel like `analysis-marketing`

Screen Shot 2022-07-04 at 4.09.56 pm.png

Hey <@U028E7B1U31> they shouldn’t be able to “edit” the column but they should be able to view it. We’ll change the wording of that.

Screen Shot 2022-07-07 at 10.01.01 pm.png

I was pointing out that when a viewer click on the `edit in sidebar`  against a column, the sidebar appears but keeps loading

Ah apologies on the misinterpretation <@U028E7B1U31>. I’ve created a ticket for this bug and will hopefully have it resolved today.

You’ll have to run the following queries on the `SECODA` role (or other role you’ve used to connect to Secoda) on your warehouse and database <@U03M9F8GP8X>

```GRANT USAGE ON WAREHOUSE "&lt;warehouse&gt;" TO ROLE SECODA;
GRANT USAGE ON DATABASE "&lt;database&gt;" TO ROLE SECODA;```

Here’s the full guide <https://docs.secoda.co/integrations/snowflake>

We had done that, but got it sorted by setting up a default.
Separately - query running permissions for editors aren't sticking - I click the checkboxes next to their names and hit "submit" - but if I navigate away and come back, the boxes clear (and they report not being able to run the queries)

We’ve updated you to an Enterprise trial <@U03M9F8GP8X>, let me know if you’re able to add editors now :slightly_smiling_face:

I can add editors, but my editors can't run queries in documents - only admins can.

I’m just going to DM you the workaround <@U03M9F8GP8X>

Hey <@U031Z4XUNMD>, this should be fixed now. Let us know if you continue to experience issues

cc <@U02N4NXTTJR> do you want to take this one? I’ll create a ticket

Pushing a fix for it. Should be live in ~10 minutes

It has not been requested in the past. Is it critical for your setup?

<@U01MHQR7DE2> Can you please review my full question

In our extractor, we’re pulling the schema/description directly from the `schema` and `description` field for a particularly `node` which are the models inside of the `manifest.json`.

Do you have an example of a `manifest.json` that you can share in a DM along with the table/schemas that the switch is happening on. That will make it easier for me to debug.

Hey <@U03NZNE5TSQ> I’m currently looking into the issue, I’ll let you know when we’ve found what the cause is.

Hey <@U03NZNE5TSQ> could you retry the integration connection

Apologies, we deployed the wrong service for the fix. The fix is now ready. Let me know if it’s resolved now.

It works now. I'm waiting to see if my jobs get imported now .

I'm not seeing the Glue Jobs. Or does the Glue integrations not supposed to import the job as well?

The Glue jobs do not currently get imported, but this is a relatively straight forward change. Let me add it to the additional changes that we’re making this week for glue <@U03NZNE5TSQ>

This is a mistake on our end. The Hex integration is still just an embed into Secoda, which you can do by pressing /Hex on a doc. What would you want to see brought in from Hex?

Hey <@U031Z4XUNMD> thanks for reporting the issue. I’ve created an issue for this week to resolve this. <https://linear.app/secoda/issue/SEC-1188/schema-and-group-visibility>

Hey Daniel, this is a bug on our end — apologies about the issue. The engineering team is strictly focused on bug fixes this week and we’ve prioritized this issue so it should be fixed within the next day. I’ll keep you updated

Hi <@U01MHQR7DE2>, thank you so much for looking into this. Let me know! :slightly_smiling_face:

hey <@U01MHQR7DE2>, any luck with this bug?

Hey <@U03PQCSRNMU>, we’ve pushed a partial fix for it, I’ll move this thread over to our shared Slack channel so that Dan can update the instance to the latest version. I’ll provide context over there of why it’s a partial fix.

Hey <@U037JGLBGF5> thanks for reporting the issue, we’re looking into it immediately and will let you know when it’s resolved.

<@U037JGLBGF5> this issue has been resolved now

We’re taking a look at this Kevin, I’ll update you shortly when we’ve fixed the issue

<@U02739P8464> can you try re-running the extraction now please, we’ve just pushed a fix

Also, don't see our `analytics` database under the catalog drop down

Hey <@U031Z4XUNMD> I’m taking a look at these two issues now

<@U031Z4XUNMD> all of the tables should be removed and you should see the analytics database in the sidebar again. Let me know if you have any other issues

We do have a backlog item that’s high in priority for supporting GCP  and Azure for the dbt core integration <@U03RP147P1A>. I can keep you updated when it’s available

Hey <@U02P9HZGUJH> this is an issue all users are experiencing. We’ve added a ticket to resolve the problem

Hey <@U03S8TT8UHY> if you try re-running the extraction now it should work

Hey <@U03SJERETDL> it’s currently not possible via the API but we have a roadmap item for this. I imagine it will be prioritized before the end of the month. When we do I’ll let you know.

A couple of other self-hosted dbt core users upload continuously via the API <@U01TB6G50RY>. I’ll send over some instructions on how to do that

<@U01TLJ5NDC2> we probably want to look at the api for this use case at least then

<@U01MHQR7DE2> so without the api its just reads the file on an ad-hoc basis. I inspected the file and i dont think they are any connection details and on prem so not worried about the IP address issues.

But i did try uploading it and got some errors :stuck_out_tongue:

```Dbt manifest extractor: 

'NoneType' object is not subscriptable

Traceback (most recent call last):

  File "/app/databuilder/job/job.py", line 75, in _launch_job

    self.task.run()

  File "/app/databuilder/task/task.py", line 67, in run

    record = self.extractor.extract()

  File "/app/databuilder/extractor/database/dbt/dbt_core_extractor.py", line 149, in extract

    return next(self._extract_iter)

  File "/app/databuilder/extractor/database/dbt/dbt_core_extractor.py", line 225, in _get_extract_iter

    manifest_list = get_manifest_list_from_machine_iam_role(

  File "/app/databuilder/utils/s3.py", line 78, in get_manifest_list_from_machine_iam_role

    if bucket_name[-1] != "/":

TypeError: 'NoneType' object is not subscriptable```

```Initialization: 

Job extraction failed, Error msg: 'NoneType' object is not subscriptable

Traceback (most recent call last):

  File "/app/databuilder/job/job.py", line 75, in _launch_job

    self.task.run()

  File "/app/databuilder/task/task.py", line 67, in run

    record = self.extractor.extract()

  File "/app/databuilder/extractor/database/dbt/dbt_core_extractor.py", line 149, in extract

    return next(self._extract_iter)

  File "/app/databuilder/extractor/database/dbt/dbt_core_extractor.py", line 225, in _get_extract_iter

    manifest_list = get_manifest_list_from_machine_iam_role(

  File "/app/databuilder/utils/s3.py", line 78, in get_manifest_list_from_machine_iam_role

    if bucket_name[-1] != "/":

TypeError: 'NoneType' object is not subscriptable



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

  File "/app/databuilder/models.py", line 105, in process

    getattr(Extractors, integration_type)(

  File "/app/databuilder/extract/database/dbt/dbt_extract.py", line 117, in _extract_dbt_core

    metadata_job.launch()

  File "/app/databuilder/job/job.py", line 123, in launch

    self._launch_job(

  File "/app/databuilder/job/job.py", line 87, in _launch_job

    raise JobFailedError(f"Job extraction failed, Error msg: {e}")

databuilder.utils.exceptions.JobFailedError: Job extraction failed, Error msg: 'NoneType' object is not subscriptable```

<@U01TB6G50RY> it’s likely because there hasn’t been a private bucket where the file gets uploaded to, but it’s much easier to just use the API

please hide the IP in the original post <@U01TB6G50RY>