Jordy Weening
12/11/2025, 1:20 PMkapa.ai
12/11/2025, 1:21 PMkapa.ai
12/11/2025, 1:21 PMedr report from a Databricks notebook using %sh and a dbt profile, which you can adapt into your bundle job/task. ### 1. Ensure packages are installed on the cluster In a notebook cell: python %pip install elementary-data %pip install "elementary-data[databricks]" This is required so the edr CLI can talk to Databricks. [[Databricks install](https://elementary-community.slack.com/archives/C02CTC89LAX/p1680765973633019?thread_ts=1680765973.633019&cid=C02CTC89LAX)] ### 2. Make sure you have an elementary profile You need a profiles.yml with an elementary profile that points to the schema where Elementary dbt models live. You can generate the profile from your dbt project with: bash dbt run-operation elementary.generate_elementary_cli_profile Then copy the output into profiles.yml (e.g. in your repo or workspace) and fill in secrets like token, host, http_path. [[Generate profile](https://docs.elementary-data.com/oss/guides/generate-report-ui)] In Databricks, you’ll typically: - Store profiles.yml in the repo/workspace path, and - Point edr to it with --profiles-dir. ### 3. Run edr report from the notebook Here is a minimal notebook cell that runs the report against a dbt project checked out in the workspace (paths are examples; adjust to your bundle’s repo path): python # Python cell that shells out to edr dbt_project_dir = "/Workspace/Repos/<user>/<repo>/path_to_dbt_project" profiles_dir = dbt_project_dir # or another folder where profiles.yml lives target_dir = f"{dbt_project_dir}/edr_target" cmd = f""" cd {dbt_project_dir} edr report \ --project-dir "{dbt_project_dir}" \ --profiles-dir "{profiles_dir}" \ --target-path "{target_dir}" """ print(cmd) python # Actually execute the command result = dbutils.notebook.run("dummy", 0) # placeholder to keep cell type Python Or more simply with `%sh`: bash %sh cd /Workspace/Repos/<user>/<repo>/path_to_dbt_project edr report \ --project-dir "/Workspace/Repos/<user>/<repo>/path_to_dbt_project" \ --profiles-dir "/Workspace/Repos/<user>/<repo>/path_to_dbt_project" \ --target-path "/Workspace/Repos/<user>/<repo>/path_to_dbt_project/edr_target" This pattern (using --project-dir, --profiles-dir, --target-path) is what users successfully run from Databricks notebooks. [[Shared-cluster example](https://github.com/elementary-data/elementary/issues/1584)] The HTML report will be created under edr_target/elementary_report.html by default (or under the folder you pass via --target-path). [[Report location](https://elementary-community.slack.com/archives/C02CTC89LAX/p1687983933201809?thread_ts=1687983933.201809&cid=C02CTC89LAX)] ### 4. (Optional) Upload to S3 or other storage One common pattern is to generate the report in the notebook and then upload it to S3 (or another static hosting location) using Python (e.g. boto3) or cloud CLIs, instead of edr send-report. [[S3 upload pattern](https://elementary-community.slack.com/archives/C02CTC89LAX/p1681955298320329?thread_ts=1681955298.320329&cid=C02CTC89LAX)] --- To integrate with Databricks Asset Bundles, wire this notebook as a task in your bundle’s job; the core logic is the same: install elementary-data[databricks], ensure profiles.yml is available, and call edr report with the correct paths. The knowledge sources don’t provide a bundle-specific YAML example, so I can’t give a fully accurate bundle config snippet.