Hi folks, I am curious, is there any way to define my own metrics in Elementary? Let's say I have a business-type of metric that i want to write myself (not provided by Elementary), where do I define it? Is it a macro with a special prefix? Maybe this is in the documentation and I missed it, so please, feel free to point me to the right direction!

Hey @Suela Isaj, thank you for reaching out! Care to give an example?

Sure, I am also new to

elementary

, and we have had

re_data

before, so the syntax below is related to that specific framework. I have this macro that calculates the fraction of users that are found day by day, basically ensuring that the users we see today have an overlap with the ones before. What I would like is the flexibility to have a general metric that I can use in different tables.

{% macro re_data_metric_frac_requiring_user_day_by_day(context) %}
    with
        relevant_data as (
            select
                {{ context.time_filter }} as time_filter,
                {{ context.column_name }} as id,
            from {{ context.table_name }}
            where
                date({{ context.time_filter }})
                > date_sub(current_date(), interval 3 day)
        )
    select countif(old_data.id is not null) / if(count(*) = 0, 1, count(*))
    from {{ context.table_name }} new_data
    left join
        relevant_data old_data
        on new_data.{{ context.column_name }} = old_data.id
        and timestamp_diff(
            new_data.{{ context.time_filter }}, old_data.time_filter, hour
        )
        between 14 and 30
    where {{ in_time_window(context.time_filter) }}
{% endmacro %}

If you have a similar example, let me know, we don't have to go into detail about the metric above. I am basically looking for a way to define a macro or maybe a test that I can call in 6 different tables as an elementary metric

@Suela Isaj thanks for the information! Currently we don't support having custom monitor types for our tests, but there are some ways to go around it: 1. Use the

null_precent

test - if there is an anomaly in the

null_precent

(too many or too less), that means you have an anomaly at the not null precent (too less or too many). 2. Create a view over the table which replaces nulls with a value and values with null, and use the

null_precent

test over it

Thanks @Idan Shavit! I am actually monitoring a column with a high imbalance of null and non null so I actually need to

non_null_percent

to catch the real anomalies. I can try to lower the anomaly sensitivity for now for the

null_percent

. Any plans to support custom metrics in the future? I would be very interested!

Yes @Suela Isaj, it is in our plans to add a custom metric that we will run anomaly test over 🙂