Ricardo Bravo
04/10/2025, 9:05 PMkapa.ai
04/10/2025, 9:05 PMkapa.ai
04/10/2025, 9:06 PMRicardo Bravo
04/10/2025, 9:08 PMkapa.ai
04/10/2025, 9:09 PMkapa.ai
04/10/2025, 9:09 PMyaml vars: elementary: test_sample_row_count: -1 # Set to -1 to store all failing rows This configuration tells Elementary to store all failing rows instead of just a sample [GitHub Issue #1652](https://github.com/elementary-data/elementary/issues/1652). ### 2. Enable store_failures for tests For individual tests, you can enable the store_failures config: yaml tests: - not_null: config: store_failures: true ### 3. Add required flags (for some environments) Some users have reported needing these flags in their dbt_project.yml to properly store test results: yaml flags: require_explicit_package_overrides_for_builtin_materializations: false source_freshness_run_project_hooks: True ### 4. Run Elementary models Make sure Elementary's models are up to date: dbt deps dbt run -s elementary Then run your tests: dbt test ### Accessing the failed records After running tests, you can find the failed records in: - elementary_test_results - Contains test metadata and results - test_result_rows - Contains samples of the actual failing rows Note that even with test_sample_row_count: -1, some test types (like aggregation tests) won't store all individual failing rows due to how they're structured. For calculating percentages, you'll need to: 1. Query the failed records from Elementary tables 2. Query the total count from your actual models 3. Calculate the percentage manually Several community members have created custom dbt models that extract this information using regex on test messages and joining with table row counts to calculate quality percentages. Would you like more specific guidance on querying these tables or creating a custom model for your percentage calculations? I'm here to help you implement this solution for your open-source Elementary setup. P.S. If you're using specific test types like uniqueness tests, you might also want to explore the fail_calc parameter as mentioned in our previous conversation. P.P.S. If you're using Athena, make sure to check that your test_result_rows table is populated correctly, as some users have reported issues with this specific database. P.P.P.S. If you're using Microsoft Teams for alerts, be aware of a current bug where not all alerts that should be sent are received, which could affect your monitoring of test failures [GitHub Issue #1652](https://github.com/elementary-data/elementary/issues/1652).Noya Offer
04/14/2025, 7:48 AM