https://linen.dev logo
Join Slack
Powered by
I’m running into a strange case where I’m seeing t...
# community-support
j
I’m running into a strange case where I’m seeing the wrong 
metric_name
added to the 
data_monitoring_metrics
table. I’m seeing 
row_count
here when the model I have tests for doesn’t have any row count tests defined and only has a 
sum
column anomaly. Very possible I’m doing something wrong though! Has anyone come across this before?
e
Hi! That’s interesting 🤔 You don’t have and never had volume anomalies for this model? Are you sure?
Do you see the sum metric as well?
j
Agreed - very interesting!
Copy code
- name: event_count_column
        description: event_count_column
        tests:
          - elementary.column_anomalies:
              column_anomalies:
                - sum
              where_expression: "event_name = 'X'"
              time_bucket: # Daily by default
                period: day
                count: 1
              training_period:
                period: day
                count: 30
              tags: ["update"]
              config:
                severity: warn
          - elementary.column_anomalies:
              column_anomalies:
                - sum
              where_expression: "event_name = 'Y'"
              anomaly_sensitivity: 4
              tags: ["update"]
              config:
                severity: warn
^^here’s how I have it defined. and the only other thing i have in the model YAML that’s related to Elementary is this below:
Copy code
config:
  elementary:
    timestamp_column: "day_pt"
and nope, not seeing the 
sum
metric_name when i run it right now! only seeing 
row_count
and 
build_timestamp
i may have had 
row_count
rather than 
sum
a few weeks ago. is it possible there’s some kind of cache i need to drop?
e
I don't think so because row_count is a table metric and "sum" is a column metric. So maybe you had a row count test configured (volume / dimension anomalies) and removed it, but I doubt it has anything to do with why you're not getting any metrics in for this column test. when did you configure the test and how many times did you run 
dbt test
since?
j
yes agreed, it doesn’t seem that would be related. i configured the test a week or two ago and have consistently seen 
row_count
rather than 
sum
with the config above each time i’ve run it, both in a testing env and through our orchestrator
another thing that’s odd is that if you look at the 
elementary_test_results
table, i do see 
column_anomalies
listed in in the 
test_short_name
column as expected, rather than 
volume_anomalies
for some other models 🤔
e
what elementary version are you using? I'll try to recreate this
j
thank you!
version: 0.13.2
if it’s helpful, the sequence (based on 
created_at
timestamp) looks to be: 1. Record is added to 
data_monitoring_metrics
with the 
row_count
value for 
metric_name
and none with column sum 2. A few seconds later, record added to 
elementary_test_results
for that model that looks like it is running column sum metrics (as expected) and no row counts 3. Nothing appears in the 
metrics_anomaly_score
view for this model (row counts or column sums), even though things show up for other models here
e
I'm unable to reproduce 😞 can you please try to query data_monitoring_metrics with the condition 
where column_name = 'event_count_column'
? do you see row count metrics there as well?
j
yep! just tried querying that and there are results with 
column_name = 'event_count'
and 
metric_name = 'sum'
from a few weeks ago (I have no idea what changed since then lol), but in the last few weeks, no records in that table with anything besides a null 
column_name
, since it only has 
row_count
and 
build_timestamp
as the two 
metric_name
values
e
you can try running the test with the following var: 
dbt test --vars '{"force_metrics_backfill": true}
also, just in case, you can try upgrading to 0.14.0.
j
got it — we have a somewhat custom way of running tests but i’ll give that a try! what does the 
force_metrics_backfill
portion do?
e
It’s forcing through all the caching mechanisms of the metrics and refreshing all of them. I’m not sure it’s needed but worth a try
34 Views
Open in Slack
PreviousNext
Powered by