Hi All, I am trying to add a custom Python test, but getting the following unexpected error:

Internal Error
  popped ['materialization'], expected macro.elementary.test_python

Any resolution for this?

Hey there @Anil! Could you share the test you're trying to add?

So in the model config yml file, I have declared the test macro like this:

- elementary.python:
          code_macro: mean_test

And in the macros folder I have added the sample test as:

{% macro mean_test(args) %}
import pandas as pd

def test(model_df, ref, session):
    original_table = model_df.toPandas()
    return True
{% endmacro %}

Anything I am doing wrong here?

@Anil What version of elementary and dbt are you using?

Core: - installed: 1.7.9 - latest: 1.7.9 - Up to date! Plugins: - bigquery: 1.7.6 - Up to date! - package: elementary-data/elementary version: 0.14.1

Is that under the tests of your module? Besides a bit of an over indent, it looks ok.

version:2
models:
  - name: orders
    description: This table has basic information about orders, as well as some derived facts based on payments
    config:
      tags: ["finance"]
      elementary:
        timestamp_column: "order_date"

    tests:
      - elementary.python:
          tags: [anil_test]
          code_macro: compare_tables
          macro_args:
            other_table: raw_orders

and:

{% macro compare_tables(args) %}
# in macros/test_for_anil.sql
def test(model_df, ref, session):
    return 0
{% endmacro %}

Then I run it with:

dbt test -s 'tag:anil_test'

Yes that is under tests of my model and under the same model other inbuilt tests like volume anomaly are working fine. Do I need to configure any other things ? Like Do I need to give Dataproc or any other compute for this test calculation?

@Anil did you try to run my simple example to verify if it works?

Yes yes, I tried same thing with my model but no luck!

Are you running in a virtual env?

Yes I am running in a virtual environment in windows in local machine

Hey @Anil, this issue happens when there is an exception in the python code, sadly, due to a limitation in the way that

dbt

allows for python code execution, the exception itself cannot be retrieved and displayed in the command output, so you would need to check it in your data warehouse.

But I have nothing in my python code:

{% macro mean_test_(args) %}
def test(model_df, ref, session):
    return True
{% endmacro %}

I think is is not able to find the macro itself. I place it in the macros of whole project, do I need to put in macros of elementary package?

Hmm interesting. i think there might be an error in our docs, i dont thing that returning a

bool

is supported, only

int

(number of failing rows) or

DataFrame

(failing rows). can you try to change it to

return 0

? also, can you share how your test is configured in the

yml

file?

Hi @Ofek Weiss, thanks for replying. Even with returning 0, getting same error. Attaching yml config below:

What dwh are you using? did you go through the necessary setup for dbt python - https://docs.getdbt.com/docs/build/python-models#specific-data-platforms