we're finding a mismatch between the test passes a...
# ask-elementary-ai
s
we're finding a mismatch between the test passes and failures reported in the elementary report vs the pass and failure figures posted to slack. why might that be?
k
A message from kapa.ai
Based on the knowledge sources, there are several reasons why you might see a mismatch between test results in the Elementary report versus Slack notifications: ## Common Causes 1. Skipped Tests Being Counted as Warnings: According to a [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1705055739443819), skipped tests were being added to the warnings section in Slack summaries, which could cause discrepancies. This was identified as an issue and fixed in a subsequent version. You can verify if you have skipped tests by checking for
status = 'skipped'
in your
elementary_test_results
table. 2. Timing Issues with
--days-back
Parameter
: As mentioned in a [community thread](https://slack.com/archives/C02CTC89LAX/p1669159603059669), the
--days-back
parameter used for viewing test history in reports can affect how Slack notifications behave, potentially batching results from different days together. 3. Errored Tests vs Failed Tests: According to [this discussion](https://slack.com/archives/C02CTC89LAX/p1701968892485459), Elementary displays errored tests differently from failed tests (grey instead of red in the report). This could lead to discrepancies in how tests are counted between the report and Slack notifications. 4. Report Generation Timing: There was a [known issue](https://slack.com/archives/C02CTC89LAX/p1692848107076739) where the report's logic ran before data finished loading, which could cause inconsistencies. This was fixed by changing the execution order. ## Recommended Actions - Check your
elementary_test_results
table for tests with
status = 'skipped'
- Review your
edr.log
and
elementary_output.json
files (found in the
edr_target
folder) to help diagnose the issue - Ensure you're using the latest version of Elementary, as several bugs related to test result reporting have been fixed in recent releases If the issue persists, the Elementary team typically requests these log files to help diagnose the specific cause of the mismatch.