On Elementary Cloud, I cannot consistently get lineage or catalog pages to load. I am not sure how to provide any further information about this, but it seems that these pages arbitrarily load, or sit with a spinner indefinitely, which makes demoing or presenting the tools difficult to do, because I can't reliably predict when pages will load.

Hi @jeremy thomas! sorry for the inconvenience here, we are looking into it

No, it's intermittent (which I know is the most fun to troubleshoot). And if it's not working, none of them work, not just some models. It seems to be working now, but I have a presentation tomorrow morning and I'm concerned that it will show back up

For sure, I'm sure we will find the issue soon

Hey @jeremy thomas, we have good news 🙂 We found the problem and @Elon Gliksberg and @Itamar Hartstein deployed a fix and validated it works in your instance. The issue was that there are circular dependencies in your pipelines. As this is not allowed in dbt, our lineage algorithm assumes there aren't any, and it went into endless loops. Our graph still does not support circular dependencies, but it will no longer fail due to loops. What we did is to prioritize the direct

ref

in case there is a circular dependency. We assumed that the direct ref better represents your model. For example: If there is

model_a

select * from {{ref("model_b")}}

and

model_b

select * from {{target.database}}.{{target.schema}}.model_a

The lineage will only show

model_b -> model_a

. Hope this makes sense, let us know if we can help with something else 🙏 @Elon Gliksberg@Itamar Hartstein @Nimrod Neuberg - thank you all for the super quick investigation and fix! 🐕

@Maayan Salom and team, thank you for the incredibly fast fix! I will admit, some of our dbt practices have been less than perfect up until now (another good reason to have better monitoring) and so I am unsurprised to find this was less than optimal on our end. I will work with the team to find a better solution, and I appreciate your flexibility and helpfulness!