@kapa.ai can i have multiple elementary targets/locations in a single dbt project?

Thinking completed Yes, you can have multiple Elementary targets or locations in a single dbt project, typically representing different environments such as

dev

,

staging

, and

prod

[Multi-schema paradigm; Multi-env schema sprawl]. Elementary operates on a model of one Elementary schema per dbt project per environment (target) [Multi-schema paradigm]. How it works 1. dbt Configuration (

dbt_project.yml

) You can define the target database and schema for Elementary models in your `dbt_project.yml`models elementary: +database: your_database_name +schema: elementaryBy default, dbt appends this custom schema to your target schema from

profiles.yml

(e.g.,

<target_schema>_elementary

), resulting in different locations per target environment [Define database and schema; Multi-schema paradigm]. 2. Elementary Profile (

profiles.yml

) When using the Elementary CLI (

edr

), configure multiple targets under the

elementary

profile to match each environment [Define database and schema]elementary outputs: dev: type: warehouse_type database: dev_database schema: dev_elementary # ... rest of credentials prod: type: warehouse_type database: prod_database schema: prod_elementary # ... rest of credentials target: prod 3. Running the CLI for Specific Targets When generating reports or running monitors, specify which target/location Elementary should read from using the

--profile-target

flag [Define database and schema]:edr report --profile-target dev edr report --profile-target prod Important Considerations • Within a single target run: Elementary expects a single unified schema per target run; you cannot dynamically split Elementary’s internal tables across multiple schemas simultaneously during the same invocation [Multi-schema paradigm]. • Cross-project differences: While multiple targets/environments are supported in a single project, Elementary Open Source does not support merging multiple distinct dbt projects into the same Elementary schema [Slack conversation; Slack conversation]. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai can in databricks multiple service principals write into the elementary schema (not concurrently, but one after another?) Does elementary exposes some views that cant be changed by manage/modify permissions?

Thinking completed Based on the provided knowledge sources, here is what is known regarding multiple service principals and permissions in Databricks: Multiple Service Principals Writing to the Elementary Schema The knowledge sources do not explicitly confirm whether having multiple distinct service principals write sequentially to the same Elementary schema in Databricks is officially supported, but they highlight an important caveat regarding ownership and permissions: • Ownership and Grant Overwrites: During dbt runs, Elementary updates and rebuilds artifact tables and views (for example, on

on-run-end

). If different users or service principals execute these runs, replace/recreate operations can flip relation ownership and cause permissions or grants to be dropped or overwritten [Linen thread on permissions]. • Mitigation: If you run jobs using different principals, ensure that role-based permissions or shared ownership (such as a shared group/role or default schema privileges) are set up so that one principal does not lock out the other when replacing views and tables [Linen thread on permissions]. Views That Cannot Be Changed by Manage/Modify Permissions The knowledge sources do not explicitly mention whether Elementary exposes specific views that cannot be altered via

MANAGE

or

MODIFY

permissions in Databricks. The sources do note that: • Elementary builds several internal views (such as

model_run_results

and

snapshot_run_results

) as part of its package models [Linen thread on permissions; Linen thread on dbt-spark]. • When dbt runs with global grant configurations (e.g.,

+grants

) in Databricks, issues such as

KeyError: 'Principal'

can happen if Databricks does not return the expected principal format when standardizing grants on view creation [Linen thread on dbt-spark]. • For Elementary Cloud access, only read-only access (

USE CATALOG

,

USE SCHEMA

,

SELECT

) is required on the Elementary schema, alongside access to Databricks system tables (

system.information_schema

,

system.query

,

system.access

,

system.billing

) [Permissions and security]. Beyond this, there is no information indicating that Elementary views have inherent restrictions that prevent standard Databricks

MODIFY

or

MANAGE

privileges from functioning normally. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai does elementary store the alerts run info and all of that in incremental tables? I m planning to have a sibgle elementary schema for multiple pipelines from a single dbt project. I dont want each pipeline to override the tables via the on hook end macro

Thinking completed Yes, Elementary stores run results, test executions, and alerts incrementally or as log-based appends, and using a single Elementary schema for multiple pipelines from a single dbt project will not wipe out your results [Multi-schema paradigm; Multi-job same project]. Here is how Elementary handles this across your pipelines: 1. Alert and Result Models are Log-Based / Incremental • Alert tables: Tables such as

alerts_v2

(and

alerts

,

alerts_models

,

alerts_source_freshness

) are configured as incremental models [Race condition issue; Slack forum alert tables]. • Run and test results: Result models (such as

dbt_run_results

,

elementary_test_results

, and

dbt_invocations

) store logs of your executions. The

on-run-end

hook runs queries that append new invocation details and run results rather than dropping prior results [On-run-end hooks; Multi-schema paradigm]. 2. Multiple Pipelines from the Same dbt Project If all pipelines belong to the same dbt project (sharing the same project definition and documents), running different pipelines or jobs into one Elementary schema is explicitly supported: • Each run only updates the metadata and artifacts relevant to what was executed, without overwriting unrelated models or previous run logs [Multi-schema paradigm; Multi-job same project]. • For dbt versions 1.4.0 and above, Elementary maintains an artifacts cache for metadata models (

dbt_models

,

dbt_tests

, etc.), meaning only newly changed configurations are updated rather than replacing the full project metadata on every run [On-run-end hooks]. (Note: Schema overwriting only becomes a problem if you have multiple separate dbt projects writing to the exact same Elementary schema, which is not supported [Multi-schema paradigm].) 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.