In Elementary Data, tests and test coverage are related but distinct concepts: ## Tests Tests in Elementary Data are validations that check if your data meets specific quality expectations. Elementary supports several types of tests: 1. dbt Tests: Elementary leverages dbt's testing ecosystem, including standard tests (unique, not_null) and tests from packages like dbt-utils and dbt-expectations. 2. Anomaly Detection Tests: These monitor data for unusual patterns or deviations from historical norms. Examples include: - Volume anomalies (monitoring row counts) - Freshness anomalies (checking data timeliness) - Dimension anomalies (detecting changes in specific data segments) - Column anomalies (monitoring metrics like null counts, min/max values) 3. Schema Tests: These validate the structure of your data, such as detecting schema changes. 4. Custom SQL Tests: You can write your own SQL-based validations. ## Test Coverage Test coverage refers to how comprehensively your data assets are being tested. It's a measure of what percentage of your data models, columns, or quality dimensions have tests applied to them. Elementary provides a Test Coverage page that helps you: - Identify gaps in your testing strategy - Organize tests by quality dimensions (completeness, uniqueness, freshness, validity, accuracy, consistency) - Add tests to improve coverage As mentioned in the documentation, "While it may seem attractive to implement extensive monitoring throughout your data infrastructure, this approach is often suboptimal. Excessive failures can lead to alerts fatigue, potentially causing them to overlook significant issues." [Detection and coverage](https://docs.elementary-data.com/best-practices/detection-and-coverage) The key difference is that tests are the individual validations you apply, while test coverage is the measure of how well those tests span across your data assets and quality dimensions.

I think there might be a misunderstanding about the relationship between tests and test coverage in Elementary Data. Tests don't "override" test coverage - they contribute to it. Test coverage is a measure of how comprehensively your data assets are being tested. When you apply tests to columns or models, you're increasing your test coverage. Each test you add contributes to your overall coverage metrics. Based on the knowledge sources, Elementary Data allows you to configure tests at different levels with a priority hierarchy. For timestamp columns specifically, there's a configuration priority:

column tests > table config > global vars

As mentioned in one of the Slack conversations: > "You can define the timestamp column in a global (tests config), per model config, or per test config. If you have only one test - than yes, all definitions will be equivalent. You can configure the timestamp_column in either level and it would behave the same." [Slack conversation](https://slack.com/archives/C02CTC89LAX/p1681873690201069) This means that if you configure the same parameter (like timestamp_column) at multiple levels, the more specific configuration (column level) will take precedence over more general configurations (table or global level). So rather than tests "overriding" coverage, it's more accurate to say that more specific test configurations can override more general configurations when they apply to the same parameter.

Based on our previous discussion about Elementary Data, here's a summary of tests vs test coverage and configuration priority in chart format: ## Tests vs Test Coverage | Aspect | Tests | Test Coverage | |--------|-------|---------------| | Definition | Individual validations that check if data meets quality expectations | Measure of how comprehensively data assets are being tested | | Purpose | Validate specific quality dimensions of data | Identify gaps in testing strategy | | Types | dbt Tests, Anomaly Detection Tests, Schema Tests, Custom SQL Tests | Percentage metrics across models, columns, or quality dimensions | | Implementation | Added through YAML configs or SQL files | Measured and visualized in Elementary UI | | Outcome | Pass/fail results for specific validations | Overview of testing comprehensiveness | ## Configuration Priority Hierarchy | Priority Level | Scope | Precedence | Example | |----------------|-------|------------|---------| | 1 (Highest) | Column-level tests | Overrides all other levels | Specific test configuration for a single column | | 2 | Table/model config | Overrides global settings | Configuration applied to an entire model | | 3 (Lowest) | Global variables | Default settings | Project-wide configuration in dbt_project.yml | This hierarchy follows the principle that more specific configurations take precedence over more general ones. When you apply tests at different levels, you're not overriding coverage but rather increasing it, while potentially overriding configuration parameters at more general levels.

Based on the knowledge sources, when you add tests through the Elementary Data cloud platform UI, the commits are created under the name "Elementary Data" as the commit author. This is part of Elementary's Configuration-as-Code feature, which allows you to configure tests directly from the UI. When you configure a test through the Elementary Cloud interface, the platform automatically opens a pull request (PR) or merge request in your connected code repository. The process works as follows: 1. You need to first [connect a code repository](https://docs.elementary-data.com/cloud/integrations/code-repo/connect-code-repo) to your Elementary project 2. When you configure tests through the UI, Elementary creates the necessary YAML configurations 3. These configurations are submitted as a PR/MR to your repository with "Elementary Data" as the commit author 4. You can then review and merge these changes through your normal code review process Elementary supports integration with various code repository platforms including [GitHub](https://docs.elementary-data.com/cloud/integrations/code-repo/github), [GitLab](https://docs.elementary-data.com/cloud/integrations/code-repo/gitlab), [Bitbucket](https://docs.elementary-data.com/cloud/integrations/code-repo/bitbucket), and [Azure DevOps](https://docs.elementary-data.com/cloud/integrations/code-repo/azure-devops). This approach combines the ease of UI-based configuration with the benefits of version control and code review processes.

Based on the knowledge sources, when you encounter an error status with a custom SQL test in Elementary Data, there are several ways to access detailed logs to understand what went wrong: ## Finding Error Logs for Custom SQL Tests 1. dbt.log File: The most comprehensive error information is typically found in the dbt.log file. This file contains detailed error messages from dbt's execution of your test query. As mentioned in one of the support conversations: > "Could you please share with me your

edr.log

file? It's in the path where you executed the CLI." [Slack conversation](https://slack.com/archives/C02CTC89LAX/p1677270954575139) 2. edr.log File: This log file contains information about the Elementary Data Report (EDR) CLI execution and can provide insights into why your custom query test failed. 3. Elementary Test Results Table: In some cases, you might need to check the

elementary_test_results

table in your data warehouse: > "You can search for those rows in the

elementary_test_results

table in the Elementary schema and delete them." [Slack conversation](https://slack.com/archives/C02CTC89LAX/p1687167849673609) 4. Data Warehouse Query History: Sometimes the error might be visible in your data warehouse's query history: > "Maybe the error is not from the test, it's from the sample query. Maybe you can locate the failed query on Redshift query history?" [Slack conversation](https://slack.com/archives/C02CTC89LAX/p1684011990045279) Unfortunately, the knowledge sources don't indicate a specific UI location where you can directly view detailed error logs for pending changes with an 'error' status. This suggests that you might need to check the log files mentioned above or your data warehouse query history to find the detailed error information. If you're using the Elementary Cloud platform, you might want to check if there's a specific section in the UI for viewing logs or contact Elementary support for more specific guidance on accessing error details for custom SQL tests. Would you like more information about how to properly format custom SQL tests to avoid errors?

Based on the knowledge sources, both Elementary Cloud and Elementary OSS provide ways to audit logs related to test results, alerts, and monitoring activities, but with different approaches and capabilities. ## Elementary Cloud Platform Elementary Cloud provides comprehensive incident management and alerting capabilities that allow you to track and audit test results and alerts: 1. Incidents Page: The Cloud platform offers a centralized [Incidents page](https://docs.elementary-data.com/features/alerts-and-incidents/incident-management) where you can: - View all open and historical incidents - Track what issues are open and which were resolved - See who is assigned to each incident and the current status - View the timeline of actions and events since an incident started - Check if similar issues happened before and how they were resolved [Incidents in Elementary](https://docs.elementary-data.com/features/alerts-and-incidents/incidents) 2. Incident Deep Dive: When clicking on an incident, you can see: - Test execution history - Result details (pass/fail/warning) - Alert distribution information (which Slack channels received alerts) - Sample of failed rows - Compiled queries [Incidents in Elementary](https://docs.elementary-data.com/features/alerts-and-incidents/incidents) ## Elementary OSS In the OSS version, audit logs are primarily available through: 1. Data Warehouse Tables: Elementary creates tables in your data warehouse that store test results, metrics, and alerts: - `alerts_data_monitoring`: Contains information about alerts that were triggered - `data_monitoring_metrics`: Stores metrics collected by tests - `elementary_runs`: Tracks execution runs 2. CLI Logs: The Elementary CLI generates log files that contain information about command execution: - `edr.log`: Contains detailed logs about Elementary Data Report (EDR) CLI execution 3. Observability Report: You can generate a report using the CLI command:

edr report

This creates a UI that shows test results and model runs [CLI commands and usage](https://docs.elementary-data.com/oss/cli-commands) The main difference is that Cloud offers a more integrated, user-friendly interface for tracking incidents and alerts with advanced features like incident grouping, assignment, and status tracking, while OSS requires more manual querying of the data warehouse tables and log files to audit the same information. If you need more specific audit capabilities that you're not seeing in either version, you might want to reach out to Elementary's support team for guidance.