Has anyone seen this behavior before? Running the ...
# community-support
r
Has anyone seen this behavior before? Running the initial creation of elementary models succeeds without issue. The artifact tables are correctly updated by the on-run-end calls. However if we try to incrementally refresh the elementary models using dbt run --select elementary.edr.dbt_artifacts (as suggested in the docs if artifact upload is disabled), then we get compilation errors on all of the elementary model sql. It appears that is breaking up the id field into each character Running with dbt version: 1.7.2 python version: 3.9.13 elementary 0.13.1 Snowflake
Copy code
19:44:32 Finished running 10 incremental models, 3 view models, 2 hooks in 0 hours 0 minutes and 38.41 seconds (38.41s).
19:44:33
19:44:33 Completed with 10 errors and 0 warnings:
19:44:33
19:44:33  Database Error in model dbt_invocations (models\edr\dbt_artifacts\dbt_invocations.sql)
 001003 (42000): SQL compilation error:
 syntax error line 24 at position 36 unexpected '_'.
 syntax error line 24 at position 58 unexpected '_'.
 compiled Code at target\run\elementary\models\edr\dbt_artifacts\dbt_invocations.sql
The compiled /target/run/elementary/models/edr/dbt_artifacts/dbt_invocations.sql file)
Copy code
-- back compat for old kwarg name
merge into EDP_DEV_N.robert_elementary.dbt_invocations as DBT_INTERNAL_DEST using EDP_DEV_N.robert_elementary.dbt_invocations__dbt_tmp as DBT_INTERNAL_SOURCE on DBT_INTERNAL_SOURCE.i = DBT_INTERNAL_DEST.i
and DBT_INTERNAL_SOURCE.n = DBT_INTERNAL_DEST.n
and DBT_INTERNAL_SOURCE.v = DBT_INTERNAL_DEST.v
and DBT_INTERNAL_SOURCE.o = DBT_INTERNAL_DEST.o
and DBT_INTERNAL_SOURCE.c = DBT_INTERNAL_DEST.c
and DBT_INTERNAL_SOURCE.a = DBT_INTERNAL_DEST.a
and DBT_INTERNAL_SOURCE.t = DBT_INTERNAL_DEST.t
and DBT_INTERNAL_SOURCE.i = DBT_INTERNAL_DEST.i
and DBT_INTERNAL_SOURCE.o = DBT_INTERNAL_DEST.o
and DBT_INTERNAL_SOURCE.n = DBT_INTERNAL_DEST.n
and DBT_INTERNAL_SOURCE._ = DBT_INTERNAL_DEST._
and DBT_INTERNAL_SOURCE.i = DBT_INTERNAL_DEST.i
and DBT_INTERNAL_SOURCE.d = DBT_INTERNAL_DEST.d
when matched then
update
set INVOCATION_ID = DBT_INTERNAL_SOURCE.INVOCATION_ID,
  JOB_ID = DBT_INTERNAL_SOURCE.JOB_ID,
  JOB_NAME = DBT_INTERNAL_SOURCE.JOB_NAME,
  JOB_RUN_ID = DBT_INTERNAL_SOURCE.JOB_RUN_ID,
  RUN_STARTED_AT = DBT_INTERNAL_SOURCE.RUN_STARTED_AT,
  RUN_COMPLETED_AT = DBT_INTERNAL_SOURCE.RUN_COMPLETED_AT,
  GENERATED_AT = DBT_INTERNAL_SOURCE.GENERATED_AT,
  CREATED_AT = DBT_INTERNAL_SOURCE.CREATED_AT,
  COMMAND = DBT_INTERNAL_SOURCE.COMMAND,
  DBT_VERSION = DBT_INTERNAL_SOURCE.DBT_VERSION,
  ELEMENTARY_VERSION = DBT_INTERNAL_SOURCE.ELEMENTARY_VERSION,
  FULL_REFRESH = DBT_INTERNAL_SOURCE.FULL_REFRESH,
  INVOCATION_VARS = DBT_INTERNAL_SOURCE.INVOCATION_VARS,
  VARS = DBT_INTERNAL_SOURCE.VARS,
  TARGET_NAME = DBT_INTERNAL_SOURCE.TARGET_NAME,
  TARGET_DATABASE = DBT_INTERNAL_SOURCE.TARGET_DATABASE,
  TARGET_SCHEMA = DBT_INTERNAL_SOURCE.TARGET_SCHEMA,
  TARGET_PROFILE_NAME = DBT_INTERNAL_SOURCE.TARGET_PROFILE_NAME,
  THREADS = DBT_INTERNAL_SOURCE.THREADS,
  SELECTED = DBT_INTERNAL_SOURCE.SELECTED,
  YAML_SELECTOR = DBT_INTERNAL_SOURCE.YAML_SELECTOR,
  PROJECT_ID = DBT_INTERNAL_SOURCE.PROJECT_ID,
  PROJECT_NAME = DBT_INTERNAL_SOURCE.PROJECT_NAME,
  ENV = DBT_INTERNAL_SOURCE.ENV,
  ENV_ID = DBT_INTERNAL_SOURCE.ENV_ID,
  CAUSE_CATEGORY = DBT_INTERNAL_SOURCE.CAUSE_CATEGORY,
  CAUSE = DBT_INTERNAL_SOURCE.CAUSE,
  PULL_REQUEST_ID = DBT_INTERNAL_SOURCE.PULL_REQUEST_ID,
  GIT_SHA = DBT_INTERNAL_SOURCE.GIT_SHA,
  ORCHESTRATOR = DBT_INTERNAL_SOURCE.ORCHESTRATOR,
  DBT_USER = DBT_INTERNAL_SOURCE.DBT_USER,
  JOB_URL = DBT_INTERNAL_SOURCE.JOB_URL,
  JOB_RUN_URL = DBT_INTERNAL_SOURCE.JOB_RUN_URL,
  ACCOUNT_ID = DBT_INTERNAL_SOURCE.ACCOUNT_ID,
  TARGET_ADAPTER_SPECIFIC_FIELDS = DBT_INTERNAL_SOURCE.TARGET_ADAPTER_SPECIFIC_FIELDS
  when not matched then
insert (
    "INVOCATION_ID",
    "JOB_ID",
    "JOB_NAME",
    "JOB_RUN_ID",
    "RUN_STARTED_AT",
    "RUN_COMPLETED_AT",
    "GENERATED_AT",
    "CREATED_AT",
    "COMMAND",
    "DBT_VERSION",
    "ELEMENTARY_VERSION",
    "FULL_REFRESH",
    "INVOCATION_VARS",
    "VARS",
    "TARGET_NAME",
    "TARGET_DATABASE",
    "TARGET_SCHEMA",
    "TARGET_PROFILE_NAME",
    "THREADS",
    "SELECTED",
    "YAML_SELECTOR",
    "PROJECT_ID",
    "PROJECT_NAME",
    "ENV",
    "ENV_ID",
    "CAUSE_CATEGORY",
    "CAUSE",
    "PULL_REQUEST_ID",
    "GIT_SHA",
    "ORCHESTRATOR",
    "DBT_USER",
    "JOB_URL",
    "JOB_RUN_URL",
    "ACCOUNT_ID",
    "TARGET_ADAPTER_SPECIFIC_FIELDS"
  )
values (
    "INVOCATION_ID",
    "JOB_ID",
    "JOB_NAME",
    "JOB_RUN_ID",
    "RUN_STARTED_AT",
    "RUN_COMPLETED_AT",
    "GENERATED_AT",
    "CREATED_AT",
    "COMMAND",
    "DBT_VERSION",
    "ELEMENTARY_VERSION",
    "FULL_REFRESH",
    "INVOCATION_VARS",
    "VARS",
    "TARGET_NAME",
    "TARGET_DATABASE",
    "TARGET_SCHEMA",
    "TARGET_PROFILE_NAME",
    "THREADS",
    "SELECTED",
    "YAML_SELECTOR",
    "PROJECT_ID",
    "PROJECT_NAME",
    "ENV",
    "ENV_ID",
    "CAUSE_CATEGORY",
    "CAUSE",
    "PULL_REQUEST_ID",
    "GIT_SHA",
    "ORCHESTRATOR",
    "DBT_USER",
    "JOB_URL",
    "JOB_RUN_URL",
    "ACCOUNT_ID",
    "TARGET_ADAPTER_SPECIFIC_FIELDS"
  )
e
Hi @Robert O, looking into it now. Seems that for some reason the merge incremental strategy parses the unique_id as a list instead of a string. Trying to understand why.
Do you happen to use
incremental_predicates
?
r
We don't use incremental_predicates in our project at all
o
Hi @Robert O, can you share the contents of the dbt_invocations.sql file in your project? it should be in
dbt_packages/elementary/models/edr/dbt_artifacts/dbt_invocations.sql
r
I'm still seeing the same issue with latest release (0.14). Note that this error is occurring for all the elementary models, not just dbt_invocations here is the contents of dbt_packages\elementary\models\edr\dbt_artifacts\dbt_invocations.sql:
Copy code
{{
  config(
    materialized = 'incremental',
    transient=False,
    unique_key = 'invocation_id',
    on_schema_change = 'append_new_columns',
    full_refresh=elementary.get_config_var('elementary_full_refresh'),
    meta={
      "timestamp_column": "created_at",
      "prev_timestamp_column": "generated_at",
      },
    table_type=elementary.get_default_table_type(),
    incremental_strategy=elementary.get_default_incremental_strategy()
  )
}}

{{ elementary.get_dbt_invocations_empty_table_query() }}
For future reference, we found this issue was caused by an old copy of the merge macro in our macro folder which was overriding the default. Once we removed the old merge macro and it started using the default dbt merge macro, the elementary incremental refresh is running successfully