What happens if my project have a multi schema par...
# ask-elementary-ai
l
What happens if my project have a multi schema paradigm? Will there be several metrics schemas?
k
No, having multiple schemas in your dbt project does not create multiple Elementary schemas. Elementary creates one schema per dbt project per environment (target), regardless of how many different schemas your models write to. As Elementary support confirmed directly for this scenario: > "Elementary creates a schema for each dbt project + env (target), and works regardless of the different jobs and schemas on the project. Once you install the dbt package in this project, the hooks will run in all your jobs and write to the same unified Elementary schema." [Single schema per project] So even if your
dbt_project.yml
routes models to schemas like
stg1
,
stg2
,
marts1
,
marts2
, etc., all Elementary metadata will be written to a single Elementary schema. Each job run will only update the artifacts relevant to what was run, without overwriting unrelated results. [Multi-job same project] The only cases where you'd end up with multiple Elementary schemas are: • Multiple separate dbt projects (one Elementary schema per project is required to avoid conflicts) • Multiple targets/environments (e.g.,
prod
,
staging
,
dev
) if you choose to enable Elementary in each 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.
l
And the name? Which name will elementary concatenate with the one we choose? @kapa.ai
k
By default, dbt's
generate_schema_name
macro concatenates your target schema (from
profiles.yml
) with the custom schema name you choose, using an underscore:
Copy code
<target_schema>_<custom_schema_name>
So if your
profiles.yml
has
schema: my_project
and your
dbt_project.yml
has:
Copy code
models:
  elementary:
    +schema: elementary
...Elementary tables will land in
my_project_elementary
. [Elementary FAQ] If you want a fully custom name (no automatic prefix), you need to override the
generate_schema_name
macro. For example, using the
meta.schema_name
approach: [Elementary FAQ]
Copy code
{% macro generate_schema_name(custom_schema_name, node) -%}
    {%- set default_schema = target.schema -%}
    {% set config_meta = node.config.get('meta') %}
    {% if config_meta and config_meta is mapping %}
        {% set schema_name = config_meta.get('schema_name') %}
        {% if schema_name and schema_name is string %}
            {{ return(schema_name) }}
        {% endif %}
    {%- elif custom_schema_name is none -%}
        {{ default_schema }}
    {%- else -%}
        {{ default_schema }}_{{ custom_schema_name | trim }}
    {%- endif -%}
{%- endmacro %}
Then in `dbt_project.yml`:
Copy code
models:
  elementary:
    +meta:
      schema_name: "elementary_data_observability"
This gives you full control over the final schema name without any automatic prefix. [Custom schema name] 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.
l
@kapa.ai What if there are two projects writing in the same target schema? How the metrics going to be resolved
k
If two separate dbt projects write to the same Elementary schema, you will run into data conflicts and overrides. This is explicitly not supported by Elementary. The core problem is that some Elementary tables are fully reloaded (recreated) on every dbt run, not incrementally appended. So whichever project runs last will overwrite the artifacts from the previous one, causing missing or incorrect data in your reports and alerts. [Multi-project schema conflict] As Elementary support explained: > "Some of our artifacts are tables which get reloaded each dbt run. The meaning of that is that if you have different models / tests / etc at your projects, running one will override the artifacts of the other which will lead into missing data at Elementary alerts and report." [Multi-project schema conflict] There is also no parameter or variable to enable safe multi-project sharing of a single schema. [No parameter available] Recommended solution Use a separate Elementary schema per dbt project, even if both projects share the same target database. You can do this by setting a distinct schema name in each project's `dbt_project.yml`:
Copy code
# Project 1
models:
  elementary:
    +schema: project1_elementary

# Project 2
models:
  elementary:
    +schema: project2_elementary
[Per-project schemas] If you need a unified view across both projects, Elementary Cloud supports connecting multiple dbt projects (each with its own Elementary schema) into a single environment, handling the unification at the UI level rather than at the schema level. [Cloud multi-env] 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.
r
@kapa.ai I have pretty similar question: Does this mean that if I have a single dbt model (template) that receives parameters like:
Copy code
dbt build --select orders --vars '{"category_name": "project_1"}'
dbt build --select orders --vars '{"category_name": "project_2"}'
dbt build --select orders --vars '{"category_name": "project_3"}'
In that case, should I create separate Elementary schemas for each parameterized run, or is it acceptable to use a single shared schema? Because Airflow passes different parameters to same model to create project_orders (each order table has different schema)
k
This is essentially the multi-tenant / parameterized run scenario, and it maps closely to cases discussed in the community. The core issue Since you're running the same dbt project (same
dbt_project.yml
, same project name) with different variables, Elementary treats all these runs as the same project. The problem is that dbt artifact tables (like
dbt_models
,
dbt_tests
, etc.) are fully reloaded on each run, so each parameterized run will overwrite the artifacts from the previous one. [Multi-project schema conflict] Single shared schema Using one shared Elementary schema for all parameterized runs will cause overrides — the last run wins, and you'll lose observability data from the previous runs. This is the same problem as two projects sharing a schema. Separate schema per parameterized run This is the approach the community has experimented with for multi-tenant setups. You can dynamically set the Elementary schema based on your variable:
Copy code
# dbt_project.yml
models:
  elementary:
    +schema: "elementary_{{ var('category_name', 'default') }}"
This would produce schemas like
elementary_project_1
,
elementary_project_2
, etc. However, this means a separate Elementary report per parameterized run, which may not be what you want. [Per-project schemas] If you want a unified view The community workaround is to create a
union_elementary_schema
that unions the tables from all per-run schemas and point the CLI at that. However, there's a caveat: since it's the same dbt project, identifier clashes are likely. One suggested mitigation is to make the project name dynamic:
Copy code
name: 'my_project_{{ env_var("category_name") }}'
...so that identifiers differ across runs. But this comes with its own maintenance overhead and is not officially supported. [Multi-tenant union approach] Summary The knowledge sources do not contain a clean, officially supported solution for this exact parameterized-run pattern in OSS. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements