The dbt_models table is created, but the loading fails

from what I can see when it is querying for the temp table columns it can't see them.

[0m134550.470387 [debug] [Thread-1 (]: In "redshift"."tf_warehouse_elementary_zone"."dbt_models": Schema changed: True Source columns not in target: [] Target columns not in source: [<Column unique_id (character varying(4096))>, <Column alias (character varying(4096))>, <Column checksum (character varying(4096))>, <Column materialization (character varying(4096))>, <Column tags (character varying(65535))>, <Column meta (character varying(65535))>, <Column owner (character varying(4096))>, <Column database_name (character varying(4096))>, <Column schema_name (character varying(4096))>, <Column depends_on_macros (character varying(65535))>, <Column depends_on_nodes (character varying(65535))>, <Column description (character varying(65535))>, <Column name (character varying(4096))>, <Column package_name (character varying(4096))>, <Column original_path (character varying(65535))>, <Column path (character varying(4096))>, <Column patch_path (character varying(4096))>, <Column generated_at (character varying(4096))>, <Column metadata_hash (character varying(4096))>] New column types: [] [0m134550.497789 [debug] [Thread-1 (]: Using redshift connection "model.elementary.dbt_models" [0m134550.498483 [debug] [Thread-1 (]: On model.elementary.dbt_models: /* {"app": "dbt", "dbt_version": "1.8.1", "profile_name": "bremer_bank", "target_name": "SANDBOX", "node_id": "model.elementary.dbt_models"} */ alter table "redshift"."tf_warehouse_elementary_zone"."dbt_models" drop column unique_id [0m134550.535390 [debug] [Thread-1 (]: SQL status: SUCCESS in 0.0 seconds [0m134550.540385 [debug] [Thread-1 (]: Using redshift connection "model.elementary.dbt_models" [0m134550.542960 [debug] [Thread-1 (]: On model.elementary.dbt_models: /* {"app": "dbt", "dbt_version": "1.8.1", "profile_name": "bremer_bank", "target_name": "SANDBOX", "node_id": "model.elementary.dbt_models"} */

..

This is very odd since there is no error on the creation of the temp table in redshift

so then it says it succuessfully drops column on my dbt_models

..

alter table "redshift"."tf_warehouse_elementary_zone"."dbt_models" drop column unique_id [0m152333.850660 [debug] [Thread-1 (]: SQL status: SUCCESS in 0.0 seconds [0m152333.853668 [debug] [Thread-1 (]: Using redshift connection "model.elementary.dbt_models"

but when I look at the table in redshift the whole table exist

image.png

Hello o o o o o . . .

This slack channel is where questions go to collect dust. 🙂 Best of luck all.

I think I found the issue. Elementary is using temp tables and I for some reason am losing connection in between the creation of the dbt_models_tmp table and the pull of the columns from the data dictionary.

[0m152327.577076 [debug] [Thread-1 (]: Began executing node model.elementary.dbt_models [0m152327.707072 [debug] [Thread-1 (]: Using redshift connection "model.elementary.dbt_models" [0m152327.708358 [debug] [Thread-1 (]: On model.elementary.dbt_models: /* {"app": "dbt", "dbt_version": "1.8.1", "profile_name": "bremer_bank", "target_name": "SANDBOX", "node_id": "model.elementary.dbt_models"} */ create temporary table "dbt_models__dbt_tmp152327636145" ... ) as empty_table where 1 = 0 ); [0m152327.709114 [debug] [Thread-1 (]*: Opening a new connection, currently in state closed* [0m152327.709877 [debug] [Thread-1 (]: Redshift adapter: Establishing connection using ssl with

sslmode

set to 'prefer'.To connect without ssl, set

sslmode

to 'disable'. [0m152327.710376 [debug] [Thread-1 (]: Redshift adapter: Connecting to redshift with IAM based auth... [0m152327.998888 [debug] [Thread-1 (]: SQL status: SUCCESS in 0.0 seconds [0m152328.033597 [debug] [Thread-1 (]: Using redshift connection "model.elementary.dbt_models"