Hello! If I wanted to see '% of rows failing', wha...
# community-support
j
Hello! If I wanted to see '% of rows failing', what would be the recommended path to seeing this in the Elementary report? We are using the open-source dbt package. We recently migrated observability dashboards from an in-house solution to using Elementary reports. One in-house feature we need urgently (next 2 weeks) is a metric showing us 'how badly' a given dbt model is failing a given test. We need this to proceed with fixing bad data and reachout out to stakeholders about getting higher quality data in the pipeline. The metric of interest is: Of all the rows that existed in a given model right before filtering for failures, what percentage of those rows were failures? We computed it in our generic tests - ideally, we can show that and other columns of interest in reports as needed.
Code-wise, it looks like a lot of what I need to modify is concentrated within the
run_results-esque
logic. testing - if so, would prefer to still understand the 'best' way to maintain this 🙂
n
Hey Jonathan! Thanks for reaching out. Currently, Elementary doesn’t provide this calculation directly, but here are a couple of options that might help: 1. Elementary Cloud offers a data health dashboard that presents six data quality dimension scores along with an overall data health score. While this provides a different perspective, let us know if it’s something you’d find useful! 2. To calculate this manually, you would need to query both the number of failed rows and the total row count. You can use dbt’s
store_failures
feature to capture failing records, but note that this doesn’t work for every type of test—for example, tests that include aggregations in their queries. If you come up with an in-house solution, we’d love for you to share it with the community! 🙂
d
@Jonathan Delgadillo Lorenzo did you find a solution for calculating percentage of failures metric?