Dragos Cioata
03/23/2026, 11:15 AMdbt_runner-dbt-dev-1 | {"info": {"name": "RunningOperationCaughtError", "code": "Q001", "msg": "Encountered an error while running operation: Database Error\n Invalid timestamp: '2026-03-18T13:52:03.178195476Z'; while executing the filter on column 'execute_completed_at'", "level": "error", "invocationId": "ac0ba774-7c75-427e-9d4e-0cef41a060cc", "pid": 13, "thread": "MainThread", "ts": "2026-03-23T11:12:21.722963Z"}, "data": {"exc": "Database Error\n Invalid timestamp: '2026-03-18T13:52:03.178195476Z'; while executing the filter on column 'execute_completed_at'"}}
Anyone with the same problem?Noya Offer
03/24/2026, 1:40 PMDragos Cioata
03/25/2026, 7:52 AMDmitrii Drik
04/03/2026, 4:05 PMedr send-report --update-bucket-website true --profiles-dir <...> --google-service-account-path <...> --gcs-bucket-name <...> --bucket-file-path elementary_report/index.html
Elementary version:
elementary-data[bigquery]==0.23.0
LOG:
[2026-04-03, 10:45:38 EDT] {subprocess.py:106} INFO - 2026-04-03 14:45:38 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /opt/python3.11/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /home/airflow/gcs/dags/dbt
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - {"data": {"args": [], "base_msg": "<https://console.cloud.google.com/bigquery>", "exc_info": "", "name": "BigQuery"}, "info": {"category": "", "code": "E004", "extra": {}, "invocation_id": "5b59411c-cd8f-4d7e-8a5b-2ca978c6bd7d", "level": "error", "msg": "BigQuery adapter: <https://console.cloud.google.com/bigquery>", "name": "AdapterEventError", "pid": 629, "thread": "MainThread", "ts": "2026-04-03T14:45:40.698827Z"}}
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - {"data": {"exc": "Database Error\n Invalid timestamp: '2026-04-03T10:50:50.961498756Z'; while executing the filter on column 'execute_completed_at'"}, "info": {"category": "", "code": "Q001", "extra": {}, "invocation_id": "5b59411c-cd8f-4d7e-8a5b-2ca978c6bd7d", "level": "error", "msg": "Encountered an error while running operation: Database Error\n Invalid timestamp: '2026-04-03T10:50:50.961498756Z'; while executing the filter on column 'execute_completed_at'", "name": "RunningOperationCaughtError", "pid": 629, "thread": "MainThread", "ts": "2026-04-03T14:45:40.706316Z"}}
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - 2026-04-03 14:45:40 — ERROR — Could not generate the report - Error: Failed to run dbt command.
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - {"info": {"name": "RunningOperationCaughtError", "code": "Q001", "msg": "Encountered an error while running operation: Database Error\n Invalid timestamp: '2026-04-03T10:50:50.961498756Z'; while executing the filter on column 'execute_completed_at'", "level": "error", "invocationId": "5b59411c-cd8f-4d7e-8a5b-2ca978c6bd7d", "pid": 629, "thread": "MainThread", "ts": "2026-04-03T14:45:40.706316Z"}, "data": {"exc": "Database Error\n Invalid timestamp: '2026-04-03T10:50:50.961498756Z'; while executing the filter on column 'execute_completed_at'"}}
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - Please reach out to our community for help with this issue.
Error in CTE (in `cast(execute_completed_at as timestamp)`*):*
/* {"app": "dbt", "dbt_version": "1.11.2", "profile_name": "elementary", "target_name": "staging", "connection_name": "macro_log_macro_results"} */
/* --ELEMENTARY-METADATA-- {"command": "run-operation", "elementary_component": "OSS", "invocation_id": "5b59411c-cd8f-4d7e-8a5b-2ca978c6bd7d", "package_name": "elementary", "resource_name": "log_macro_results", "resource_type": "macro"} --END-ELEMENTARY-METADATA-- */
create or replace table `<elem schema>`.`ordered_test_results__tmp_20260403144539479920`
...
dbt_run_results as (
select * from `<elem schema>`.`dbt_run_results`
where
timestamp_diff(current_timestamp, cast(execute_completed_at as timestamp), day)
< 7
),
....
Reason:
The timestamp 2026-04-03T10:50:50.961498756Z has 9 decimal places (nanoseconds), while BigQuery supports a maximum of 6 decimal places (microseconds).Dmitrii Drik
04/07/2026, 4:22 PMAlex Stevens
04/07/2026, 6:02 PMedr report command in dbt-bigquery >=1.11
Environment:
- edr 0.23.1 / Elementary dbt package 0.23.0
- dbt-bigquery >=1.11.0
- BigQuery
Problem Analysis
Starting around March 25th, it seems, nanosecond-precision timestamps (9 fractional digits, e.g. 2026-04-06T23:02:36.794354259Z) began showing up in Elementary’s dbt_run_result table. The execute_completed_at, execute_started_at, compile_started_at, and compile_completed_at columns are STRING type, so the values are accepted on insert.
However, when edr report runs the elementary_cli.get_test_results macro, it filters on execute_completed_at by casting to TIMESTAMP — and BigQuery’s TIMESTAMP type only supports microsecond precision (6 fractional digits). This causes the query to fail with:
Database Error
Invalid timestamp: '2026-04-06T23:02:36.794354259Z'; while executing the filter on column 'execute_completed_at'
This completely blocks report generation. The error is not transient — it persists until the bad rows are manually fixed or aged out of the days_back window.
Workaround:
We’re adding a step to our Elementary deployment job in Github Actions which truncates the fractional seconds to 6 digits in the affected table before executing `edr report`:
UPDATE project.dataset_elementary.dbt_run_results
SET
execute_started_at = REGEXP_REPLACE(execute_started_at, r'(\.\d{6})\d+Z$', r'\1Z'),
compile_started_at = REGEXP_REPLACE(compile_started_at, r'(\.\d{6})\d+Z$', r'\1Z'),
compile_completed_at = REGEXP_REPLACE(compile_completed_at, r'(\.\d{6})\d+Z$', r'\1Z'),
execute_completed_at = REGEXP_REPLACE(execute_completed_at, r'(\.\d{6})\d+Z$', r'\1Z')
WHERE REGEXP_CONTAINS(execute_completed_at, r'\.\d{7,}Z$')
OR REGEXP_CONTAINS(execute_started_at, r'\.\d{7,}Z$')
OR REGEXP_CONTAINS(compile_started_at, r'\.\d{7,}Z$')
OR REGEXP_CONTAINS(compile_completed_at, r'\.\d{7,}Z$')
Suggested fix: Elementary’s macros that filter/cast these STRING timestamp columns should either SAFE_CAST or truncate to microsecond precision before casting.Alex Stevens
04/15/2026, 4:31 PMMika Kerman
04/16/2026, 9:18 AMDragos Cioata
04/16/2026, 9:31 AMAlex Stevens
04/20/2026, 9:55 PMdbt build -t prod ... from my local CLI / fusion runtime.Dragos Cioata
04/29/2026, 1:46 PMPylon
05/12/2026, 9:37 PMon-run-end macro.
PR: https://github.com/elementary-data/dbt-data-reliability/pull/1003
If you'd like to try it before the release, you can point your packages.yml to the branch:
packages:
- git: "<https://github.com/elementary-data/dbt-data-reliability.git>"
revision: app-1086-fix-bigquery-nanosecond-timestamp-cast-failure-in
Then run dbt deps and try edr report / edr send-report again. Any feedback would be appreciated!Alex Stevens
05/18/2026, 2:10 AMPylon
05/20/2026, 3:17 PMInvalid timestamp error on your end?
Any feedback would be really helpful before we cut the release.
Thanks!Alex Stevens
05/22/2026, 6:46 AMrevision: app-1086-fix-bigquery-nanosecond-timestamp-cast-failure-in (head SHA 6b65aa5), ran dbt deps, then exercised the new bigquery__edr_cast_as_timestamp macro four ways:
1. Synthetic edge cases (3 / 6 / 7 / 9 / 11 fractional digits, plus none) — all cast cleanly to microsecond TIMESTAMP.
2. Control — cast('2026-04-03T105050.961498756Z' as timestamp) without the macro: Invalid timestamp as expected.
3. Real production data — pulled rows from analytics_elementary.dbt_run_results where execute_started_at has 9 fractional digits, ran them through the macro: all cast successfully.
4. Elementary models that consume the macro — dbt run -s job_run_results alerts_dbt_models alerts_dbt_source_freshness on dev: all green, no compile errors.
So far so good. Then I installed elementary-data[bigquery]==0.23.4 and ran edr report against the prod Elementary dataset.
The gotcha…
edr report still failed with the original error:
Database Error: Invalid timestamp: '2026-05-22T00:51:19.698903453Z';
while executing the filter on column 'execute_completed_at'
Root cause: the edr CLI doesn’t use the project’s dbt_packages/elementary/. It ships its own bundled copy of the dbt package at site-packages/elementary/monitor/dbt_project/dbt_packages/elementary/, and that bundled copy is the pre-fix version.
The cast that fails (current_tests_run_results_query.sql) does call elementary.edr_cast_as_timestamp, but dispatch resolves to default__edr_cast_as_timestamp because the bundled package has no bigquery__ override.
To confirm, I cp’d the patched cast_column.sql over the bundled copy and re-ran edr report — generated the full HTML report (28 MB) successfully, no workaround needed.
All to say, the fix in the PR is correct, assuming you’ll also cut a new elementary-data PyPI release whose bundled dbt_project/dbt_packages/elementary/ contains the fix — pointing packages.yml at the branch alone doesn't resolve the issue in CLI.Pylon
05/24/2026, 8:00 AMelementary-data PyPI release for edr report / edr send-report to pick it up. We'll make sure the next release includes it.
Thanks again for validating that the macro itself works correctly 🙏Alex Stevens
05/26/2026, 11:53 PMPylon
06/01/2026, 7:20 AM