Based on the knowledge sources, test failures from

dbt test

or

dbt build

are not immediately reflected in the UI or trigger alerts. The process works as follows: When you run

dbt test

or

dbt build

, Elementary captures the test results through an on-run-end hook that executes at the end of the dbt invocation. The test results are stored in tables like

elementary_test_results

and then aggregated into alert tables like

alerts_dbt_tests

. For the alerts to be sent, you need to run the

edr monitor

command after your dbt tests complete. This command: 1. Aggregates the test results into the alerts table 2. Identifies new alerts that need to be sent 3. Sends the alerts via configured channels (like Slack) As mentioned in one of the forum discussions, Elementary leverages "the dbt results jinja variable to get the results of all the resources who participated in an invocation. This variable is currently only accessible by dbt at the end of the DAG (in the context of an on-run-end hook). So the answer is that it will alert once the DAG is completed" [Slack Forum Discussion](https://slack.com/archives/C02CTC89LAX/p1674071962667739). This means that for long-running DAGs, you'll only see alerts after the entire DAG completes, not immediately when a test fails in the middle of execution. To ensure alerts are sent promptly, you should run

edr monitor

after your dbt jobs complete. Many users set up their workflows to run

edr monitor

shortly after their dbt jobs finish (e.g., "30 minutes after our datamart job is completed" as mentioned by one user). If you're not seeing alerts despite test failures, you might want to check: 1. If the alerts are being properly recorded in the

alerts_dbt_tests

table 2. If there are any configuration issues with alert suppression or disabling 3. If the

edr monitor

command is running correctly with the proper parameters

Based on the knowledge sources, alerts from dbt tests are not automatically sent through webhooks. Instead, Elementary Data follows a specific workflow for sending alerts: 1. First, dbt tests are run (via

dbt test

or

dbt build

), which populate Elementary's test results tables 2. Then, you need to explicitly run the

edr monitor

command to process and send alerts The

edr monitor

command is what actually sends the alerts through your configured channels (including webhooks or Slack). By default, it looks for alerts from the past 24 hours, but you can adjust this with the

--days-back

parameter [Slack Forum Discussion](https://slack.com/archives/C02CTC89LAX/p1663241888773889). As one user mentioned in a forum discussion, they "run edr monitor 30 minutes after our datamart job is completed" [Slack Forum Discussion](https://slack.com/archives/C02CTC89LAX/p1676383019819779). This indicates that there's typically a deliberate delay built into workflows to ensure all test results are captured before sending alerts. The recommended flow according to Elementary support staff is:

1. dbt run 2. dbt test 3. edr monitor

[Slack Forum Discussion](https://slack.com/archives/C02CTC89LAX/p1686842159294479) If you're not seeing alerts being sent despite having failing tests, you should check: 1. Whether the alerts are properly recorded in the alerts tables 2. If you're running

edr monitor

after your dbt tests complete 3. If there are any configuration issues with your webhook setup Remember that Elementary will only send alerts for new issues that haven't been sent before, as it marks alerts as "sent" after processing them. I hope this helps clarify the process for consuming alerts through webhooks from dbt with Elementary Data!

I'll explain how the Environment Syncs Schedule works with triggered syncs in Elementary Cloud and clarify the alert consumption process. ## Environment Syncs Schedule: Triggered Syncs Elementary Cloud offers two options for synchronizing data from your data warehouse's Elementary schema: 1. Hourly syncs (default): Updates data once per hour, which means your data may not be up-to-date within that hour window 2. Triggered syncs: Uses webhooks to trigger a sync immediately after your data pipeline finishes running, providing near real-time updates ### How Triggered Syncs Work When you configure triggered syncs: 1. You set up a webhook in Elementary Cloud's Environment Settings 2. Elementary provides you with a webhook URL and a secret 3. You configure your orchestration tool to call this webhook after your dbt jobs complete The process differs slightly depending on whether you use dbt Cloud or dbt Core: #### For dbt Cloud: - You create a webhook subscription in dbt Cloud that triggers on "Run completed" events - You select only the main jobs of the relevant environment to avoid unnecessary syncs - dbt Cloud provides a secret token that you enter in Elementary Cloud - When your dbt job completes, dbt Cloud automatically calls the webhook #### For dbt Core: - You use the webhook URL and secret from Elementary in your orchestration tool - You implement a simple HTTP POST request after your dbt job completes - Example code provided by Elementary:

import hmac import requests  request_body = b"" secret = b"<SECRET>" webhook_url = "<WEBHOOK_URL>" signature = hmac.new(secret, request_body, "sha256").hexdigest() <http://requests.post|requests.post>(webhook_url, headers={"Authorization": signature}, data=request_body)

This approach ensures your Elementary Cloud environment updates as soon as new data is available, giving you near real-time visibility into your data quality [Environment Syncs Schedule](https://docs.elementary-data.com/cloud/guides/sync-scheduling). ## Alert Consumption Through Webhooks It's important to distinguish between two different webhook concepts in Elementary: 1. Triggered syncs webhooks: These trigger Elementary Cloud to sync data from your data warehouse 2. Alert webhooks: These are used to send alerts from Elementary to other systems For alert consumption, the process works as follows: 1. When dbt tests run, the results are stored in your Elementary schema through on-run-end hooks 2. To send alerts, you need to run the

edr monitor

command, which: - Aggregates test results into alert tables - Identifies new alerts to be sent - Sends alerts through configured channels The logs from a typical

edr monitor

run show this process:

Running internal dbt run to aggregate alerts Querying test alerts Running dbt run-operation elementary.log_macro_results... Querying model alerts

The time it takes for this process varies based on your data volume and warehouse performance, but from the logs shared in the knowledge sources, it typically takes a few minutes to complete [Slack Forum Discussion](https://slack.com/archives/C02CTC89LAX/p1713364399960069). For near real-time alerting, it's recommended to run

edr monitor

immediately after your dbt jobs complete. As one community member mentioned, they "run edr monitor 30 minutes after our datamart job is completed" to ensure all test results are captured. To optimize your workflow for real-time alerts, you should: 1. Configure triggered syncs in Elementary Cloud 2. Run

edr monitor

after your dbt jobs complete 3. Consider automating this process through your orchestration tool This approach will give you the most timely alerts about data quality issues in your pipelines. Hope this helps clarify how Environment Syncs Schedule works with triggered syncs and the alert consumption process!

I want to clarify the relationship between triggered syncs webhooks and alert consumption through webhooks in Elementary Cloud. ## Triggered Syncs Webhooks vs. Alert Webhooks There are two distinct webhook concepts in Elementary: 1. Triggered Syncs Webhooks: These are webhooks that trigger Elementary Cloud to sync data from your data warehouse. When your dbt job completes, it calls this webhook to initiate a sync of your Elementary schema data to Elementary Cloud. 2. Alert Webhooks: These are webhooks that Elementary uses to send alerts to external systems. This is a beta feature that allows you to consume Elementary alerts in your own systems. ## Triggered Syncs Webhooks As explained in the documentation, triggered syncs webhooks work as follows: 1. You configure a webhook URL in Elementary Cloud's Environment Settings 2. When your dbt job completes, it triggers this webhook 3. Elementary Cloud then syncs the data from your Elementary schema in your data warehouse 4. This ensures your Elementary Cloud environment has the most up-to-date information This is particularly useful for near real-time monitoring, as it updates your Elementary Cloud environment immediately after your data pipeline runs, rather than waiting for the hourly sync [Environment Syncs Schedule](https://docs.elementary-data.com/cloud/guides/sync-scheduling). ## Alert Consumption Through Webhooks For consuming alerts through webhooks (sending alerts from Elementary to other systems), the documentation indicates: > "Consuming alerts through webhooks is supported as a beta feature. Reach out to us to enable it for your instance!" [Webhooks (Beta)](https://docs.elementary-data.com/cloud/integrations/alerts/webhooks) This means that while Elementary can send alerts to various platforms like Slack, MS Teams, PagerDuty, etc., the ability to consume these alerts through your own custom webhook endpoint is still in beta and requires special enablement. The knowledge sources don't provide detailed information about how alert consumption through webhooks works in its beta state, so you would need to reach out to Elementary directly to get this feature enabled and to receive specific implementation details. If you're looking to set up a complete near real-time alerting workflow with Elementary Cloud, you would: 1. Configure triggered syncs webhooks to ensure Elementary Cloud has up-to-date data 2. Set up alert integrations with supported platforms (Slack, MS Teams, etc.) 3. If needed, request access to the beta webhook alerts feature to consume alerts in your custom systems This combination would give you the most timely and flexible alert system for your data quality monitoring. I hope this clarifies the distinction between these webhook concepts!

A message from kapa.ai

I need to clarify that there seems to be some confusion about the relationship between triggered syncs webhooks and alert consumption through webhooks in Elementary. ## Triggered Syncs Webhooks vs Alert Consumption Webhooks These are two separate features in Elementary: 1. Triggered Syncs Webhooks: These webhooks trigger Elementary Cloud to sync data from your data warehouse to the Elementary Cloud platform. This ensures your Elementary Cloud environment has up-to-date information about test results and other metrics. 2. Alert Consumption Webhooks: This is a beta feature that allows Elementary to send alerts to external systems through webhooks. ## Timing for Alert Consumption Through Webhooks Based on the knowledge sources provided, there isn't specific information about how long it takes to consume alerts through webhooks. The documentation only mentions: > "Consuming alerts through webhooks is supported as a beta feature. Reach out to us to enable it for your instance!" [Webhooks (Beta)](https://docs.elementary-data.com/cloud/integrations/alerts/webhooks) Since this is a beta feature, the detailed implementation and timing information isn't documented in the provided knowledge sources. To get specific information about the timing and implementation details, you would need to reach out directly to Elementary's team. ## Triggered Syncs Process For triggered syncs (which is different from alert consumption), the process works as follows: 1. You configure a webhook URL in Elementary Cloud's Environment Settings 2. When your dbt job completes, it triggers this webhook 3. Elementary Cloud then syncs the data from your Elementary schema in your data warehouse This sync process ensures that Elementary Cloud has the most up-to-date information from your data warehouse, which is a prerequisite for generating accurate and timely alerts [Environment Syncs Schedule](https://docs.elementary-data.com/cloud/guides/sync-scheduling). If you're interested in implementing alert consumption through webhooks, I recommend reaching out to Elementary directly as mentioned in their documentation to get access to this beta feature and receive specific implementation details including timing expectations.

Based on the knowledge sources, when using triggered syncs with webhooks in Elementary Cloud, there is minimal time delay in syncing data from the Elementary schema in your data warehouse to Elementary Cloud. According to user experiences shared in the Slack forums, the sync process appears to be quite efficient: One community member mentioned they've "experienced near real-time logging, once on-run-end hook is initiated (to Snowflake)" even with large dbt jobs consisting of "700+ models and 4000+ tests" [Slack forum user question](https://slack.com/archives/C02CTC89LAX/p1737540471837389). An Elementary staff member confirmed this experience, stating that "there is no known latency for a cloud region, and the volume of the dbt artifacts data should not have a significant effect on the time" and that "Elementary aims to be very efficient, it makes sense that you would have near real-time results, even for the large number of models and tests" [Slack forum user question](https://slack.com/archives/C02CTC89LAX/p1737540471837389). The triggered sync process works by having your orchestration tool or dbt Cloud call a webhook immediately after your dbt job completes, which then initiates the sync process from your data warehouse to Elementary Cloud [Environment Syncs Schedule](https://docs.elementary-data.com/cloud/guides/sync-scheduling). This approach is described as being "as close as possible to real time" in the Elementary documentation [Environment Syncs Schedule](https://docs.elementary-data.com/cloud/guides/sync-scheduling). So in summary, when using triggered syncs with webhooks, you can expect near real-time syncing of data from your Elementary schema to Elementary Cloud with minimal delay, regardless of the volume of data being processed.