<@U088WCP3B4P> when running `edr report` in a step...
# ask-elementary-ai
o
@kapa.ai when running
edr report
in a step in GitHub Actions, i'm getting:
Copy code
elementary.exceptions.exceptions.DbtCommandError: Failed to run dbt command.
Runtime Error
  Could not find profile named 'elementary'
despite my
profiles.yml
containing the
elementary
profile and the command working locally. I checkout the repository first, have DBT_PROFILES_DIR="." and a dbt command runs successfully beforehand
k
A message from kapa.ai
Based on the knowledge sources, this is a common issue when running
edr report
in GitHub Actions. Here are the most likely causes and solutions: ## Common Causes 1. Profile location issue: Even though you have
DBT_PROFILES_DIR="."
, the
edr
command may not be finding your
profiles.yml
file in the expected location. 2. Profile configuration mismatch: There may be a discrepancy between your profile configuration and where Elementary tables actually exist in your data warehouse. ## Solutions to Try ### 1. Explicitly specify the profiles directory Add the
--profiles-dir
flag to your
edr report
command:
Copy code
bash edr report --profiles-dir .
This has resolved the issue for several users, as mentioned in [this discussion](https://slack.com/archives/C02CTC89LAX/p1731008431456499) and [this thread](https://slack.com/archives/C02CTC89LAX/p1688066687146099). ### 2. Verify your elementary profile structure Ensure your
profiles.yml
has the correct structure with
elementary
as a top-level key:
Copy code
yaml elementary:   outputs:     default:       type: <your_warehouse_type>       # ... other connection details
As noted in [this discussion](https://slack.com/archives/C02CTC89LAX/p1686013979045139), Elementary requires a dedicated profile named
elementary
(not just using your existing dbt project profile). ### 3. Check schema configuration According to [this resolution](https://slack.com/archives/C02CTC89LAX/p1710456181524819), the schema variable works differently for Elementary. While dbt creates schemas with a prefix, Elementary needs the full schema name as it exists in your data warehouse. Verify that the schema in your
elementary
profile matches exactly where your Elementary tables are located. ### 4. Use absolute paths for credentials If you're using service account files or keyfiles, try using absolute paths instead of relative paths, as suggested in [this GitHub issue](https://github.com/elementary-data/elementary/issues/1702). The most likely fix for your GitHub Actions setup is to add
--profiles-dir .
to your
edr report
command.
o
@kapa.ai this worked, is there a way of making the output much more quiet, so it just creates the html file?
k
A message from kapa.ai
Based on the knowledge sources, there isn't a built-in
--quiet
flag for the
edr report
command to suppress output. In fact, several users have encountered errors when a
--quiet
flag was inadvertently being passed to Elementary commands, as documented in [this thread](https://slack.com/archives/C02CTC89LAX/p1730137138889379) and [this discussion](https://slack.com/archives/C02CTC89LAX/p1736184701459329). However, you have a few options to reduce output in your GitHub Actions workflow: ## 1. Redirect output in your workflow You can redirect the command output to
/dev/null
or a log file in your GitHub Actions step:
Copy code
bash edr report --profiles-dir . > /dev/null 2>&1
Or to capture it in a log file:
Copy code
bash edr report --profiles-dir . > edr_output.log 2>&1
## 2. Use the target-path option According to the [CLI documentation](https://docs.elementary-data.com/oss/cli-commands#cli-options), you can specify where output files (including logs) are saved:
Copy code
bash edr report --profiles-dir . --target-path ./edr_target
This will organize all Elementary output files in a dedicated directory, making your workspace cleaner. ## 3. Configure via environment variable You can also set the target path using an environment variable:
Copy code
bash export EDR_REPORT_TARGET_PATH="./edr_target" edr report --profiles-dir .
Note: Be careful not to use
DBT_QUIET
or similar flags, as Elementary's CLI doesn't support the
--quiet
option and it will cause errors if passed through dbt configurations. The HTML report will still be generated successfully; these approaches just help manage where the output goes and how verbose the console logs are.
o
@kapa.ai by output I'm referring to all of the lines which are in the stdout when you run
edr report
e.g:
Copy code
INFO:elementary.clients.dbt.command_line_dbt_runner:Running dbt command --log-format json deps --project-dir /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/elementary/monitor/dbt_project --profiles-dir /home/runner/work/
2025-10-28 09:51:42 — INFO — Running dbt command --log-format json deps --project-dir /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/elementary/monitor/dbt_project --profiles-dir /home/runner/work/
DEBUG:elementary.clients.dbt.command_line_dbt_runner:Running dbt command --log-format json run-operation elementary.log_macro_results --args ***"macro_name": "elementary_cli.get_latest_invocation", "macro_args": *** --project-dir /opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/elementary/monitor/dbt_project --profiles-dir /home/runner/work/
k
A message from kapa.ai
Based on the knowledge sources, there is currently no built-in option to suppress the stdout logging output from
edr report
. The Elementary CLI does not support a
--quiet
flag or similar option to reduce verbosity. ## Current Limitation According to [this GitHub issue](https://github.com/elementary-data/elementary/issues/732), the Elementary CLI does not support
--log-format json
or similar logging control options like dbt core does. The issue discusses a feature request for JSON log formatting, but there's no mention of a quiet mode. ## Workarounds for GitHub Actions Since you're running this in GitHub Actions, you can suppress the output at the shell level: ### 1. Redirect all output to /dev/null
Copy code
bash edr report --profiles-dir . > /dev/null 2>&1
### 2. Redirect to a log file (if you need to review it later)
Copy code
bash edr report --profiles-dir . > edr_output.log 2>&1
### 3. Suppress only stdout (keep errors visible)
Copy code
bash edr report --profiles-dir . > /dev/null
The HTML report file will still be generated successfully regardless of whether you redirect the output. These approaches simply prevent the verbose logging from cluttering your GitHub Actions console output. Note: As mentioned in [this discussion](https://slack.com/archives/C02CTC89LAX/p1677471254515409), there was a feature request for JSON log formatting support, but it hasn't been implemented yet. The community is welcome to contribute this feature if needed.
o
@kapa.ai also is there a way of monitoring the change in the amount of time it takes to run a model as a potential anomaly? like if it takes 10 seconds normally and it spikes to 30 seconds, that's a 200% increase
k
A message from kapa.ai
Yes, Elementary supports monitoring model execution time anomalies! There are two main approaches you can use: ## 1. Anomaly Detection on Execution Time (Recommended) You can set up an anomaly detection test on the
model_run_results
table to automatically detect unusual increases in execution time. Here's how: ### Step 1: Create a view (if needed) Create a view model on top of
model_run_results
(e.g.,
model_execution_times.sql
):
Copy code
sql {{ config(materialized='view') }}  select name, execute_started_at, execution_time from {{ ref('elementary', 'model_run_results') }}
### Step 2: Add the anomaly test In your
schema.yml
or
properties.yml
file:
Copy code
yaml models:   - name: model_execution_times     columns:       - name: execution_time         tests:           - elementary.column_anomalies:               column_anomalies:                 - max               tags: ['elementary']               timestamp_column: 'execute_started_at'               where_expression: name = '<my_model_name>'
As explained in [this discussion](https://slack.com/archives/C02CTC89LAX/p1673350484278159), this will detect spikes in execution time for specific models. Limitation: Currently, you need to create a separate test for each model you want to monitor (using
where_expression
), as Elementary doesn't yet support grouping by model name automatically. ## 2. Alternative: Test All Models with Dimensions You can also define the test directly on the
model_run_results
source using dimensions:
Copy code
yaml sources:   - name: elementary_models     schema: "your_elementary_schema"     tables:       - name: model_run_results         columns:           - name: execution_time                 tests:               - elementary.column_anomalies:                   config:                     severity: warn                   tags: ["model_performance"]                   column_anomalies:                     - max                   dimensions: ["package_name", "name"]                   timestamp_column: generated_at                   anomaly_direction: spike                   ignore_small_changes:                     spike_failure_percent_threshold: 10
As shown in the [Performance Alerts documentation](https://docs.elementary-data.com/features/performance-monitoring/performance-alerts), this configuration: - Monitors each model individually using dimensions - Only detects spikes (not drops) in execution time - Ignores small changes under 10% to reduce noise - Would catch your 200% increase scenario (10s → 30s) ## 3. Static Threshold Alternative If you prefer a simpler approach with fixed thresholds, you can create a singular test:
Copy code
sql {{ config(     tags=["model_performance"],     meta={         "description": "Alert on models with significant execution time increases"     } ) }}  select name, package_name, status, generated_at, execution_time from {{ ref('elementary', 'model_run_results') }} where CAST(generated_at AS timestamp) >= TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 day)    AND execution_time >= 30 * 60  -- 30 minutes threshold order by execution_time desc
The anomaly detection approach is more dynamic and would automatically adapt to your model's normal execution time patterns, making it ideal for catching relative changes like your 200% increase example.
o
@kapa.ai is bytes processed one of the metrics collected by elementary in the artifact tables? I couldn't find it
k
A message from kapa.ai
Based on the knowledge sources, bytes processed is not currently collected by Elementary in the artifact tables, but this is a frequently requested feature. ## Current Status According to multiple discussions in the community: - Elementary does not currently track bytes processed or billing metrics like BigQuery's bytes processed or Snowflake's query costs ([GitHub issue #420](https://github.com/elementary-data/elementary/issues/420)) - The team is actively researching this feature, as mentioned in [this discussion](https://slack.com/archives/C02CTC89LAX/p1731405406429199) - There are open feature requests for this functionality ([issue #420](https://github.com/elementary-data/elementary/issues/420), [issue #617](https://github.com/elementary-data/elementary/issues/617), and [issue #601](https://github.com/elementary-data/elementary/issues/601)) ## What Elementary Does Collect According to the [Security and Privacy documentation](https://docs.elementary-data.com/cloud/general/security-and-privacy#what-information-does-elementary-collect%3F), Elementary currently collects: - Run results: Including run time, errors, invocation origin - Table metrics: Row count, last modification time, size - Execution time: Available in the
dbt_run_results
table ## Workaround As noted in [this discussion](https://slack.com/archives/C02CTC89LAX/p1711962067673969), you can access some adapter-specific information: > "I finally found the data inside
dbt_run_results
. One could query for
test
materialization and check the
execution_time
as well as the adapter response, which includes bytes processed, etc." The
dbt_run_results
table stores the full adapter response, which may contain bytes processed information depending on your data warehouse adapter (particularly for BigQuery). However, this data is not parsed or displayed in Elementary's UI. ## Future Plans According to [this comment](https://github.com/elementary-data/elementary/issues/420), the suggested implementation approach would be to "add the entire adapter response as a field" to make billing metrics available across different warehouse platforms.