how to resolve the issue with dbt_run_results - rows_affected column whihc is default INT, why this is not bigint ?

dbt_run_results — rows_affected Type Issues There are actually two related but distinct problems documented around the

rows_affected

column: 1. CAST_OVERFLOW: rows_affected defined as INT instead of BIGINT On a first run against a new database (Databricks), the column type mismatch between

BIGINT

and

INT

causes an overflow error:

[CAST_OVERFLOW_IN_TABLE_INSERT] Fail to assign a value of "BIGINT" type to the "INT" type column or variable `rows_affected` due to an overflow.

This was reported as a bug and the Elementary team confirmed a fix was released for the Databricks V2 materialization in a subsequent dbt package version. [GitHub Issue] Workaround until fixed: Re-running

dbt run

after the first failure succeeds, as the tables are created correctly on the second run. 2. String "-1" inserted into an INT column (skipped models) When dbt skips a model (e.g., a view with

+view_update_via_alter: true

and no schema changes, or a BigQuery materialized view with

on_configuration_change = 'continue'

), the adapter returns

rows_affected

as the string

"-1"

instead of an integer or

NULL

. Elementary inserts this string as-is, causing failures like: • Databricks Runtime 17.3+:

[INVALID_INLINE_TABLE.INCOMPATIBLE_TYPES_IN_INLINE_TABLE]

[GitHub Issue] • BigQuery:

Value has type STRING which cannot be inserted into column rows_affected, which has type INT64

[GitHub Issue] The root cause is in

macros/edr/dbt_artifacts/upload_run_results.sql

— the

flatten_run_result

macro extracts

rows_affected

without type validation:

'rows_affected': run_result_dict.get('adapter_response', {}).get('rows_affected'),

Fix (confirmed working): The Elementary team developed a fix that converts the string

"-1"

to

NULL

before insertion. A community member confirmed: > "The

-1

is converted to a null" [GitHub Issue] Recommended Actions 1. Upgrade your Elementary dbt package to the latest version, as fixes for both issues have been released. 2. If you're on Databricks Runtime 17.3+ with

+view_update_via_alter: true

, ensure you're on a version that includes the

rows_affected

type-safety fix. 3. If you're still hitting the issue, consider using

try_cast

in the generated SQL or converting the value in the Jinja macro as a temporary workaround. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai which version should i use i am using 0.16.0

Based on the knowledge sources, if you are using dbt package v0.16.0, you should align it with edr (CLI) v0.16.1. The general guidance from the Elementary team is: > "The minor version in the dbt package and the Python package should be aligned." The compatible combinations documented are: [Slack discussion] However, regarding the

rows_affected

bug specifically, the knowledge sources do not explicitly confirm which

0.16.x

patch version includes the fix. The Elementary team recommended upgrading to

0.16.4

(the latest in the

0.16.x

line) as it was released with fixes. [Slack discussion] Recommended steps 1. Upgrade your dbt package to

0.16.4

in `packages.yml`:

packages:
  - package: elementary-data/elementary
    version: 0.16.4

2. Run:

dbt deps
dbt run --select elementary

3. Upgrade the CLI accordingly:

pip install elementary-data==0.16.2

> Note: The knowledge sources do not explicitly confirm that

0.16.4

contains the

rows_affected

fix. If the issue persists, consider checking the Elementary GitHub releases for the exact version that includes the fix. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements

@kapa.ai i have tried this with 0.19.4 version as well it still creates table ewith row_affected as int

I'm sorry, but the knowledge sources do not contain enough information to confirm whether the

rows_affected

column type (INT vs BIGINT) issue is fully resolved in version 0.19.4 specifically. What the knowledge sources do confirm is: • The

CAST_OVERFLOW

error (INT vs BIGINT) on Databricks was reported on v0.19.1 and the Elementary team stated a fix was released for the Databricks V2 materialization shortly after. [GitHub Issue] • The fix was described as being released "in the next couple of days" from when the comment was made, but the exact version number containing the fix is not explicitly stated in the knowledge sources. What I'd recommend 1. Upgrade to the latest available version (the knowledge sources reference up to

0.21.0

) as it contains the most accumulated fixes for Databricks-related issues:

packages:
  - package: elementary-data/elementary
    version: 0.21.0

2. If the table already exists with the wrong

INT

type, you may need to drop and recreate the

dbt_run_results

table so it gets rebuilt with the correct schema:

dbt run --select elementary --full-refresh

3. If the issue persists after upgrading, I'd strongly recommend opening a GitHub issue or posting in the Elementary Slack community with your exact Databricks Runtime version and dbt-databricks version, as the team can provide a targeted fix. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.