This looks like the same Databricks pattern others hit where

edr

hangs or intermittently fails while running the internal

dbt deps

for the Elementary internal project. The logs you pasted are exactly that step. There are a few known Databricks‑specific gotchas: 1. Permission / filesystem issues on the internal dbt project path On Databricks, the internal project lives under something like:

/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/elementary/monitor/dbt_project

When

dbt deps

tries to write

package-lock.yml

or

dbt_packages

there, it can fail or hang if that location isn’t writable for the job user. This is documented in similar errors:

Permission denied: .../elementary/monitor/dbt_project/package-lock.yml

. [[Databricks perms](https://elementary-community.slack.com/archives/C02CTC89LAX/p1716798424976069); [GitHub 1584](https://github.com/elementary-data/elementary/issues/1584)] Mitigations from the community: - Set

DBT_PACKAGES_FOLDER

to a writable location (e.g.

/tmp/edr_dbt_packages

) so

dbt deps

doesn’t try to write under

site-packages

. [[DBT_PACKAGES_FOLDER](https://elementary-community.slack.com/archives/C02CTC89LAX/p1716798424976069)] - Or pre‑run

dbt deps

for the internal project in your image / job so

edr

doesn’t need to do it at runtime:

dbt deps --project-dir /local_disk0/.ephemeral_nfs/.../site-packages/elementary/monitor/dbt_project

[[Pre‑deps in Docker](https://elementary-community.slack.com/archives/C02CTC89LAX/p1697103899679929)] 2. Intermittent resource (memory / CPU) issues There are multiple reports where

edr report

/

send-report

fails or dies only sometimes, and the root cause was memory / resource pressure (e.g. dbt subprocess killed with SIGKILL, or report failing “20% of the time”). [[Intermittent report fail](https://elementary-community.slack.com/archives/C02CTC89LAX/p1703268110543749); [SIGKILL example](https://elementary-community.slack.com/archives/C02CTC89LAX/p1718142965337009)] On Databricks, that can manifest as: - The

dbt deps

subprocess being killed (no further logs, job “stuck”). - Very long hangs on

dbt run-operation elementary.log_macro_results

or

dbt deps

before failing. 3. How to debug what’s actually happening in your case Since your log stops at

dbt deps

, you’ll need to look at the underlying dbt logs for the internal project: - Check the internal dbt logs directory on the cluster:

/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/elementary/monitor/dbt_project/logs

(path will match your

project-dir

root). [[Internal dbt logs](https://elementary-community.slack.com/archives/C02CTC89LAX/p1703686732161589)] - Also check

edr.log

under your

target-path

(often

target/edr.log

or similar) if you’ve configured one; that’s where Elementary logs more detail. [[edr.log hint](https://elementary-community.slack.com/archives/C02CTC89LAX/p1703268110543749)] 4. Concrete steps to try in your Databricks job Based on the above community fixes: 1. Set a writable packages folder before calling `edr`:

python       import os       os.environ["DBT_PACKAGES_FOLDER"] = "/tmp/edr_dbt_packages"

Then run your

edr send-report

command. [[DBT_PACKAGES_FOLDER](https://elementary-community.slack.com/archives/C02CTC89LAX/p1716798424976069)] 2. (Alternative) Pre‑install internal deps once per cluster: In a setup cell or cluster‑init script:

bash       dbt --log-format json deps \         --project-dir /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/elementary/monitor/dbt_project

Adjust the path to match your actual

project-dir

. This is the same pattern others used in Docker/Airflow so

edr

no longer tries to run

dbt deps

at runtime. [[Pre‑deps in Docker](https://elementary-community.slack.com/archives/C02CTC89LAX/p1697103899679929)] 3. If it’s still intermittent, increase resources / reduce load: - Try running the job on a larger cluster or with fewer concurrent tasks. - If you later see failures at the

run-operation elementary.log_macro_results

step with SIGKILL, that’s a strong sign of memory pressure. [[SIGKILL example](https://elementary-community.slack.com/archives/C02CTC89LAX/p1718142965337009)] Because your logs stop exactly at

dbt deps

, I strongly suspect either a permissions issue on the internal project path or the subprocess being killed under resource pressure. The steps above (especially setting

DBT_PACKAGES_FOLDER

and checking the internal

dbt.log

) are the recommended next actions from similar Databricks cases in the knowledge sources.