Hey everyone, I added a schema change test for my...
# community-support
a
Hey everyone, I added a schema change test for my source tables : tables: - name: account tests: - elementary.schema_changes: tags: ["data-observability"] config: severity: warn and I keep getting this error :
Database Error in test elementary_source_schema_changes_raw_salesforce_account_ (models/schema.yml)
001003 (42000): SQL compilation error:
syntax error line 2 at position 16 unexpected '('.
What could be the reason, is it even possible to test source tables for schema change ?
o
Hey @Ahmed Hamid, it is possible! i need you to share some info with me in order to verify the problem: • what are the versions of: ◦ dbt ◦ elementary dbt package • what warehouse are you using? • do you overwrite any builtin dbt macros? also it would help if you can share your
dbt.log
file (you can do so privately if you prefer)
a
Hi @Ofek Weiss thank you so much for your willingness to help me out with this one. Sure: • dbt version 1.6.9 • elementary package 0.13.2 • Snowflake • Yes I have overwritten the source macro so that i can do do some filtering :
Copy code
{% macro source(source_name, table_name, filter_deleted_records=True) %}
    --tables that don't have isdelted  column
    {% set ignore_tables = ['user', 'recordtype'] %} 
    {% set rel = builtins.source(source_name, table_name) %}

    {% if source_name == 'raw_salesforce' and table_name not in ignore_tables and filter_deleted_records %}
            {% set subquery %}
                (select * from {{ rel }} where isdeleted = false ) 
            {% endset %}       
            {% do return(subquery) %} 
     {% else %}
        {% do return(rel) %}
    {% endif %}
{% endmacro %}
o
well yeah thats it, the test tries to query the current columns of the source, but the source is a subquery and so it fails
a
Hey @Ofek Weiss, do you think there is any work arround i can do so that when the source macro is used by elementary then use the bultin source instead of a sub query? thanks in advance
o
Hey @Ahmed Hamid, cant really think of a workaround for this, maybe there is some context you can access in the macro but i'm not sure, maybe try the dbt community? this is more of a dbt thing