Hi, I'm trying to run elementary from python and calling the main function, similar to running dbt from python. Currently i'm getting the following error: Permission denied: '/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/elementary/monitor/dbt_project/package-lock.yml' Can you help me figure out why its trying to write to this package-lock file in the installed library? The script is currently running on a databricks cluster. We are using dbt-core and dbt-databricks 1.7.13 and elementary-data[databricks] 0.15.1 Here is the full stack trace of the error: > 2024-05-27 061843 — INFO — Running with edr=0.15.1 2024-05-27 061843 — INFO — Installing packages for edr internal dbt package... 2024-05-27 061843 — INFO — Running dbt --log-format json deps --project-dir /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/elementary/monitor/dbt_project --target [REDACTED] 2024-05-27 061847 — INFO — Running with dbt=1.7.13 > 2024-05-27 061847 — INFO — Encountered an error: [Errno 13] Permission denied: '/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/elementary/monitor/dbt_project/package-lock.yml' 2024-05-27 061847 — INFO — Traceback (most recent call last): File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/dbt/cli/requires.py", line 91, in wrapper result, success = func(*args, **kwargs) File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/dbt/cli/requires.py", line 76, in wrapper return func(*args, **kwargs) File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/dbt/cli/requires.py", line 152, in wrapper return func(*args, **kwargs) File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/dbt/cli/requires.py", line 198, in wrapper return func(*args, **kwargs) File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/dbt/cli/main.py", line 492, in deps results = task.run() File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/dbt/task/deps.py", line 213, in run self.lock() File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/dbt/task/deps.py", line 200, in lock with open(lock_filepath, "w") as lock_obj: PermissionError: [Errno 13] Permission denied: '/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/elementary/monitor/dbt_project/package-lock.yml' The command in python is:

from elementary.cli.cli import ElementaryCLI
run = ElementaryCLI()
run.main(
    args=[
        "report",
        "--env",
        "prod",
        "--open-browser",
        "false",
        "--project-name",
        f"governance_dbt-prod",
        "--exclude-elementary-models",
        "true",
        "--disable-passed-test-metrics",
        "true",
        "--profile-target",
        "prod",
    ],
    standalone_mode=False,
)

EDR CLI tool uses internal DBT project (installed in PIP VENV folder as a part of elementary-data package). EDR runs

dbt deps

for that DBT project to download/update DBT dependencies, so EDR needs to have enough permissions to write to a folder. But the good news are that you can control where EDR will install DBT deps with

DBT_PACKAGES_FOLDER

env var. This var is not documented, I found it digging into EDR source code. So you can just set

DBT_PACKAGES_FOLDER=/tmp/edr_dbt_packages

and it should work

Thanks i will try it

Oh, sorry, you are right,

package-lock.yml

is created inside DBT project, and not in

dbt_packages

directory. To overcome that you need to use Python VENV in writable directory. Then EDR will be able to create required file. If I remember correctly if your DBT project is not writable, you need to use both tricks: VENV and DBT_PACKAGES_FOLDER together.