Hello! Why is there a reference to "da-bi-core" instead of "da-bi-dev-core" in the error message when running the "on-run-end" task with 1 thread concurrency (target='cicd')? (DBT running with --defer)

Access Denied: Table da-bi-core:bi_system_data_observability.data_monitoring_metrics: Permission bigquery.tables.updateData denied on table da-bi-core:bi_system_data_observability.data_monitoring_metrics (or it may not exist).

models:
  elementary:
    +schema: data_observability
    +database: "{{ 'da-bi-core' if target.name in ['prod'] else 'da-bi-dev-core' }}"

bi_system_profile:
  outputs:
    cicd:
      dataset: bi_system
      keyfile: /etc/git-secret/da-bi-dwh-cicd-sa.json
      method: service-account
      project: da-bi-dev-core
      threads: 18
      type: bigquery
  target: prod
elementary:
  outputs:
    cicd:
      dataset: bi_system
      keyfile: /etc/git-secret/da-bi-dwh-cicd-sa.json
      method: service-account
      project: da-bi-dev-core
      threads: 18
      type: bigquery
  target: prod

Hiya, have you tried dbt debug before? all good there? also try

dbt run-operation elementary.generate_elementary_cli_profile

I dont see your prod defined there.

Hi @Yauheni Anikhouski, I need to look into this a little, but before I do just asking- are you using elementary in your CICD? do you need the artifact upload happening in the on-run-end? because if you don't the easiest resolution would be to just disable it

Hi @Ella Katz ! We have set

disable_dbt_artifacts_autoupload: true

before, but this is not work because on-run-end hook still trying to do this

Access Denied: Table da-bi-core:bi_system_data_observability.data_monitoring_metrics: Permission bigquery.tables.updateData denied on table da-bi-core:bi_system_data_observability.data_monitoring_metrics (or it may not exist).

In general we don't need to use elementary in ci/cd

can you also add-

vars:
  disable_run_results: true
  disable_tests_results: true
  disable_dbt_invocation_autoupload: "{{ target.name != 'prod' }}"

@Ella Katz Thanks! It work)

@Yauheni Anikhouski great! Glad to hear 🙂