Is there a way to disable the data_monitoring_metr...
# ask-elementary-ai
a
Is there a way to disable the data_monitoring_metrics functionality in the project yaml or in vars? Getting a database error: Database Error Too many tables, views and user-defined functions for query: Max: 1000 because I've got over 1000 tests and it's trying to insert all the monitoring metrics for those tests at the same time. So I'd like to just disable that so I can still run the 1k tests but avoid the database error. (edited)
k
A message from kapa.ai
Yes, you can disable the collection of metrics (which includes the population of the
data_monitoring_metrics
table) by setting the following variable in your `dbt_project.yml`:
Copy code
yaml vars:   collect_metrics: false
This will prevent Elementary from running the extra queries that populate
data_monitoring_metrics
, which should help you avoid the database error related to too many concurrent inserts or table/view limits. Disabling this will not impact your ability to run dbt tests, but you will lose the anomaly detection and historical metric tracking features that rely on this table. If your goal is to just run the tests and not collect metrics, this is a safe approach to take [Disable metric calculation](https://slack.com/archives/C02CTC89LAX/p1699602091358719) [Duplicate row_count metric](https://slack.com/archives/C02CTC89LAX/p1704884803109289). If you are running anomaly tests, be aware that they depend on the metrics stored in
data_monitoring_metrics
. If you disable metrics collection, anomaly detection will not function as intended [Disable metrics for performance](https://slack.com/archives/C02CTC89LAX/p1693398462923139). If you only want to run dbt tests and not use anomaly detection or metrics, setting
collect_metrics: false
is the recommended solution.
a
is there a way to chunk the updloads so that I can run 2k tests at the same time without runninginto this error?
is there a way to chunk the updloads so that I can run 2k tests at the same time without runninginto this error? @kapa.ai
n
@Anna Nguyen Were you able to find a solution to this? I am assuming you have a BQ DBT Project. I have the same. Trying to see if I can chunk the queries and load.