Hi, how do I override a macro from elementary? (may consider open source solution once i fix on my own deployment) https://github.com/elementary-data/dbt-data-reliability/blob/0.14.1/macros/edr/materializations/model/metrics.sql#L1 specifically this one. I tried various solutions. Thanks in advance!! In all solutions I set the dispatch in my dbt_project

dispatch:
  - macro_namespace: elementary # overriding macros in elementary package
    search_order: ["<project_name>", "elementary"]

Attempt 1

{% macro query_table_metrics() -%}
    {{ return(adapter.dispatch('query_table_metrics', 'elementary')()) }}
{% endmacro %}

{% macro default__query_table_metrics() %}

 <MY LOGIC GOES HERE>

    {# select
      {{ modules.datetime.datetime.utcnow().timestamp() }} as build_timestamp,
      count(*) as row_count
    from {{ this }}
    #}
  {% endset %}

  {% set metrics = [] %}
  {% for metric_column in elementary.run_query(query).columns %}
    {% set metric_name = metric_column.name %}
    {% set metric_value = metric_column[0] %}
    {% do metrics.append({
      "id": "{}.{}".format(invocation_id, this),
      "full_table_name": elementary.relation_to_full_name(this),
      "column_name": none,
      "metric_name": metric_name,
      "metric_value": metric_value,
      "updated_at": elementary.datetime_now_utc_as_string()
    }) %}
  {% endfor %}
  {% do return(metrics) %}
{% endmacro %}

Attempt 2

{% macro query_table_metrics() %}

 <MY LOGIC GOES HERE>

    {# select
      {{ modules.datetime.datetime.utcnow().timestamp() }} as build_timestamp,
      count(*) as row_count
    from {{ this }}
    #}
  {% endset %}

  {% set metrics = [] %}
  {% for metric_column in elementary.run_query(query).columns %}
    {% set metric_name = metric_column.name %}
    {% set metric_value = metric_column[0] %}
    {% do metrics.append({
      "id": "{}.{}".format(invocation_id, this),
      "full_table_name": elementary.relation_to_full_name(this),
      "column_name": none,
      "metric_name": metric_name,
      "metric_value": metric_value,
      "updated_at": elementary.datetime_now_utc_as_string()
    }) %}
  {% endfor %}
  {% do return(metrics) %}
{% endmacro %}

Attempt 3:

{% macro query_table_metrics() %}
    <MY LOGIC GOES HERE>
    {# select
      {{ modules.datetime.datetime.utcnow().timestamp() }} as build_timestamp,
      count(*) as row_count
    from {{ this }}
    #}
  {% endset %}
  {% set metrics = [] %}
  {% for metric_column in elementary.run_query(query).columns %}
    {% set metric_name = metric_column.name %}
    {% set metric_value = metric_column[0] %}
    {% do metrics.append({
      "id": "{}.{}".format(invocation_id, this),
      "full_table_name": elementary.relation_to_full_name(this),
      "column_name": none,
      "metric_name": metric_name,
      "metric_value": metric_value,
      "updated_at": elementary.datetime_now_utc_as_string()
    }) %}
  {% endfor %}
  {% do return(metrics) %}
{% endmacro %}

{% macro query_metrics() %}
  {% if not elementary.can_query_metrics() %}
    {% do return([]) %}
  {% endif %}

  {% do return(query_table_metrics()) %} {# use override instead #}
  {# {% do return(elementary.query_table_metrics()) %} #}
{% endmacro %}

Hey @Chris Dong, thanks for reaching out! In order to understand better the issue, can I ask why do you want to override this macro?

We don't need to perform a full table scan and can use metadata from information schema (at least for bigquery)

Thanks for the info @Chris Dong! About overriding macros: There are 2 ways to call macros: 1. By the macro name alone for example

query_table_metrics()

- In that case dbt goes over all of the installed packages and uses the first one it matches 2. By using both the name of the macro and the name of the package it is defined at for example

elementary.query_table_metric()

- In that case dbt goes directly to the specified package and uses the macro that is defined there. At Elementary we use the second option. The reason for that is it prevents users overriding Elementary's macros by mistake (if they where using the same macro name as we), and more importantly it prevents from Elementary to override users' macros. The reason you were able to override dbt_utils macros is because they probably were using the 1 option in their package.

@Idan Shavit if i understand your comments correctly, because you use the second option, it is actually not possible to override Elementary's macro intentionally - is that correct?

Hey @Chris Dong, yes you understood correctly. Unfortunately you can't override our macros 😕 If you are willing to contribute, we would be happy to provide guidance and help!

https://github.com/elementary-data/dbt-data-reliability/pull/674/files @Idan Shavit pretty quick change so here's a PR!

@Chris Dong thank you for the contribution! I see that @Itamar Hartstein is already taking a good care of it 🙂

@Itamar Hartstein mind to address the comments when you're online? Also if possible, would like this change in sooner than later. is there a rough ETA for this? and how long it'll take to get into a release patch? given this operation is resulting in unnecessary costs for us

Hi @Chris Dong! sorry for the delay will review on Sunday and yeah makes sense as a temp measure

@Itamar Hartstein so i used this solution but for some reason this test is skipped completely? any idea what could be wrong?

Hey @Chris Dong ! What do you mean by "this test"? which test?

# - package: elementary-data/elementary
  #   version: 0.13.2
  # <https://github.com/elementary-data/dbt-data-reliability/pull/674>
  - git: <https://github.com/elementary-data/dbt-data-reliability.git>
    revision: 5e5ad3555c69982684abc89a457e8c2ae86f4042

so this is what i have in my packages.yml but in production, i dont see the query being ran to get the row count anymore

Hi @Chris Dong! Sorry for the delay. Unfortunately we didn't release since then due to other urgent issues, but hopefully we'll release in the next couple of days and update here.

Hi, any update?