Hi all, I’ve granted all the Snowflake permissions as stated here https://datahubproject.io/docs/generated/ingestion/sources/snowflake#prerequisites. Does the Snowflake database show up as one of the nodes in the lineage? Or is it because it’s a data share that it wouldn’t show up? I ask because I noticed that one of our table queries the

snowflake.account_usage.tag_references

table but I don’t see this table in the lineage. The

snowflake.account_usage.tag_references

also never gets ingested by our Snowflake ingestion recipe. Lastly, when I try to just ingest the

SNOWFLAKE

database I get an error like below:

"source": {
    "type": "snowflake",
    "report": {
      "events_produced": 0,
      "events_produced_per_sec": 0,
      "entities": {},
      "aspects": {},
      "warnings": {},
      "failures": {
        "permission-error": [
          "No tables/views found. Please check permissions."
        ]
      },

I can definitely see and query the

snowflake.account_usage.tag_references

table using the Snowflake UI though so I’m not sure if it’s really a permission error at all. Thanks.

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: 1️⃣ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? Yes button 2️⃣ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Yes button

What do you mean that “one of our tables queries…”? Tag_references is a special, snowflake internal view that the ingestion queries to figure out where you have applied Snowflake tags.

We have a table that directly queries the

snowflake.account_usage.tag_references

table. So what I was expecting was this

Oh, interesting. Unfortunately im not able to answer your question right now, as i dont have the code in front of me at the moment. I wonder if the system views are filtered out somehow? 🤔

@gray-shoe-75895 any ideas here?

I believe we treat

snowflake.account_usage.tag_references

as a “system table” and hence automatically filter it out from both schema ingestion and lineage. I don’t think there’s a way to override that behavior, but cc @hundreds-photographer-13496 to confirm

that's right, the snowflake system tables are ignored during ingestion. However, as of now, this is configurable by overwriting

database_pattern

config. If not specified in recipe explicitly it takes the form below. You can override this in recipe by explicitly setting the deny pattern and not including SNOWFLAKE in it.

database_pattern:
    allow:
      - .*
    deny:
      - ^UTIL_DB$
      - ^SNOWFLAKE$
      - ^SNOWFLAKE_SAMPLE_DATA$