Overriding Elementary macros from your dbt project is generally not possible in the standard way, because Elementary explicitly calls its macros using the package namespace (e.g.,

elementary.macro_name

). This approach prevents users from overriding Elementary's macros via the usual dbt macro dispatch or by defining a macro of the same name in your project. This is a deliberate design choice to avoid accidental overrides and ensure stability of the package. If you need to change the behavior of Elementary macros (for example, to add additional fields to models like

dbt_invocations

or

dbt_run_results

), you have a couple of options: 1. Fork the Elementary package: You can fork the Elementary dbt package, make your required changes to the macros directly in your fork, and then use your forked version in your project. This is the recommended approach if you need to make fundamental changes to how the macros work, since overriding via dispatch will not work due to the explicit namespace usage in Elementary's code [Overriding Elementary macros](https://slack.com/archives/C02CTC89LAX/p1701990486227199), [Macro override limitations](https://slack.com/archives/C02CTC89LAX/p1684772727567609). 2. Contribute to Elementary: If your enhancement could be useful to others, consider contributing your changes back to the Elementary open source project. The Elementary team is open to contributions and can provide guidance if you want to add support for additional fields or orchestrator metadata [Contribution discussion](https://slack.com/archives/C02CTC89LAX/p1709420538077629). 3. Workarounds: In some cases, you can temporarily modify the macros directly in your local

dbt_packages/elementary/

directory, but this is not a sustainable solution for long-term maintenance. In summary, you cannot override Elementary macros in the standard dbt way due to their explicit namespace usage. Forking the package or contributing upstream are your best options for customization. If you have a specific use case (e.g., adding orchestrator metadata like

dag_id

,

dag_run_id

, etc.), note that Elementary already supports collecting some job information from orchestrators via environment variables. You can find more details in their documentation: [Collect job data guide](https://docs.elementary-data.com/cloud/guides/collect-job-data) and in the Slack discussions [Orchestrator metadata support](https://slack.com/archives/C02CTC89LAX/p1739193464422719).

To associate each job (or pipeline run) with a specific pipeline ID in Elementary, you can pass custom metadata—such as your pipeline ID—using dbt variables or environment variables when running your dbt commands. This approach is supported by Elementary and is commonly used for similar use cases. How to do it: - Pass the pipeline ID as a dbt variable in your run command:

dbt run --vars '{"pipeline_id": "your_pipeline_id"}'

This value will be captured in the

vars

column of the

dbt_invocations

table, allowing you to later filter or join on this field to identify which pipeline (cube) a particular job run belongs to. - Alternatively, you can set it as an environment variable and reference it in your dbt project or macros. Elementary is designed to capture such custom variables and store them in the

dbt_invocations

table, making it possible to track and differentiate runs by pipeline, tag, or any other custom identifier you provide. This method is recommended for scenarios where you have multiple pipelines or cubes and need to track job runs individually by a unique identifier like pipeline ID. You can then query the

dbt_invocations

table to analyze or report on runs by pipeline ID or join it with other result tables as needed [Custom metadata in dbt_invocations](https://slack.com/archives/C02CTC89LAX/p1666615460329049). If you need this field to appear in other models or views, you can join on

invocation_id

between

dbt_invocations

and other result tables, as suggested in the knowledge sources. Let me know if you need an example SQL query or further details!

Certainly! Here’s how you can fork the Elementary dbt package, make changes, and use your local version in your dbt project, based on the knowledge sources: ### 1. Fork the Repository - Go to the [Elementary dbt package GitHub repository](https://github.com/elementary-data/dbt-data-reliability). - Click “Fork” to create your own copy under your GitHub account. ### 2. Clone Your Fork Locally

bash git clone <https://github.com/<your-username>/dbt-data-reliability.git> cd dbt-data-reliability

### 3. Make Your Changes - Edit the macros or models as needed in your local fork. ### 4. Use Your Fork in Your dbt Project You have two main options: #### Option A: Install from Your Fork via Git In your

packages.yml

, point to your fork and (optionally) a specific branch or commit:

yaml packages:   - git: "<https://github.com/<your-username>/dbt-data-reliability.git>"     # revision: <branch-or-commit-hash>  # Optional: specify a branch or commit

Then run:

bash dbt deps

This will install your forked version into your dbt project. This method is confirmed to work and is commonly used for custom changes [Install from fork](https://slack.com/archives/C02CTC89LAX/p1701280742138359). #### Option B: Use a Local Path (for rapid iteration/testing) If you want to test changes locally without pushing to GitHub, you can use a local path in `packages.yml`:

yaml packages:   - local: ../path/to/your/local/dbt-data-reliability

Then run:

bash dbt deps

### 5. (Optional) For Elementary CLI Users If you are using the Elementary CLI and want it to use your local dbt package, you can link your local package using the CLI’s script:

bash python scripts/link_local_elementary_dbt_pkg.py <path to your local elementary dbt package repo>

This ensures the CLI uses your local changes [CLI local linking](https://slack.com/archives/C02CTC89LAX/p1683017563521789). --- Summary: - Fork the repo, make your changes, and point your

packages.yml

to your fork (or local path). - Run

dbt deps

to install. - Your dbt project will now use your customized Elementary package. Let me know if you need help with a specific step!

To add additional fields like pipeline name or pipeline ID to the

dbt_invocations

model in your forked Elementary package, you will need to: 1. Modify the model SQL file: The

dbt_invocations

model is defined in the Elementary package under

models/edr/dbt_artifacts/dbt_invocations.sql

. You should add your new fields (e.g.,

pipeline_id

,

pipeline_name

) to the

SELECT

statement in this file, making sure to extract them from the appropriate source (such as environment variables or dbt vars). 2. Update the schema YAML: You should also update the corresponding schema file (YAML) to document your new columns. This is typically found in the same directory as the model SQL file. 3. (If needed) Update macros: If the values for your new fields are set via environment variables or dbt vars, ensure that the macro or logic that collects invocation metadata is updated to include these fields. The macro responsible for uploading invocation data is usually called in an on-run-end hook, such as

elementary.upload_dbt_invocation

[see code reference](https://github.com/elementary-data/dbt-data-reliability/blob/167885c50c01b892b45e2f1073d243f80cd6c43b/macros/edr/dbt_artifacts/upload_dbt_invocation.sql#L33-L33). Where to make the changes: - SQL model:

models/edr/dbt_artifacts/dbt_invocations.sql

- Schema YAML:

models/edr/dbt_artifacts/dbt_invocations.yml

(or similar) - Macro (if needed):

macros/edr/dbt_artifacts/upload_dbt_invocation.sql

If you want these fields to be populated automatically, you can pass them as environment variables (e.g.,

DBT_PIPELINE_ID

,

DBT_PIPELINE_NAME

) when running dbt, and then reference them in your model or macro logic. This approach is supported and recommended for custom metadata [Custom metadata in dbt_invocations](https://slack.com/archives/C02CTC89LAX/p1666615460329049). If you need more detailed guidance on the exact code changes, please let me know!

Thank you for your follow-up. Based on the knowledge sources, the location of the

dbt_invocations

model in the Elementary dbt package is referenced as

models/edr/dbt_artifacts/dbt_invocations.sql

[see error logs and documentation](https://slack.com/archives/C08HW57FGN9/p1742322227557429), [dbt artifacts documentation](https://docs.elementary-data.com/dbt/dbt-artifacts). However, if you do not see the models in that directory after forking and cloning the repository, there are a few possible reasons: 1. You may be looking at the wrong repository or branch. Make sure you have forked and cloned the correct repository: [elementary-data/dbt-data-reliability](https://github.com/elementary-data/dbt-data-reliability). 2. The models directory structure may have changed. The knowledge sources reference the path as

models/edr/dbt_artifacts/

. If you do not see this, check for a similar directory or search for

dbt_invocations.sql

in the repository. 3. You may be looking at a minimal or incomplete clone. Ensure you have cloned the full repository and not a shallow or partial copy. If after checking these points you still cannot find the models, the knowledge sources do not provide further details about a different structure or location. In that case, I recommend searching the repository for

dbt_invocations.sql

or reviewing the latest documentation in the repo for any structural changes. If you need more specific guidance, please provide the structure you see, or clarify which files are present in your fork.