Hello, When i run the edr send-report i got ```dbt...
# community-support
d
Hello, When i run the edr send-report i got
Copy code
dbt_runner-dbt-dev-1  | {"info": {"name": "RunningOperationCaughtError", "code": "Q001", "msg": "Encountered an error while running operation: Database Error\n  Invalid timestamp: '2026-03-18T13:52:03.178195476Z'; while executing the filter on column 'execute_completed_at'", "level": "error", "invocationId": "ac0ba774-7c75-427e-9d4e-0cef41a060cc", "pid": 13, "thread": "MainThread", "ts": "2026-03-23T11:12:21.722963Z"}, "data": {"exc": "Database Error\n  Invalid timestamp: '2026-03-18T13:52:03.178195476Z'; while executing the filter on column 'execute_completed_at'"}}
Anyone with the same problem?
➕ 1
🎫 1
✅ 1
n
Hey @Dragos Cioata, thank you for bringing this to our attention. Which DWH and Elementary versions are you using?
d
bigquery and v0.23.0
d
Hi, @Noya Offer The same issue occurs when generating an EDR report using the following command:
Copy code
edr send-report --update-bucket-website true --profiles-dir <...> --google-service-account-path <...> --gcs-bucket-name <...> --bucket-file-path elementary_report/index.html
Elementary version:
elementary-data[bigquery]==0.23.0
LOG:
Copy code
[2026-04-03, 10:45:38 EDT] {subprocess.py:106} INFO - 2026-04-03 14:45:38 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /opt/python3.11/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /home/airflow/gcs/dags/dbt
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - {"data": {"args": [], "base_msg": "<https://console.cloud.google.com/bigquery>", "exc_info": "", "name": "BigQuery"}, "info": {"category": "", "code": "E004", "extra": {}, "invocation_id": "5b59411c-cd8f-4d7e-8a5b-2ca978c6bd7d", "level": "error", "msg": "BigQuery adapter: <https://console.cloud.google.com/bigquery>", "name": "AdapterEventError", "pid": 629, "thread": "MainThread", "ts": "2026-04-03T14:45:40.698827Z"}}
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - {"data": {"exc": "Database Error\n  Invalid timestamp: '2026-04-03T10:50:50.961498756Z'; while executing the filter on column 'execute_completed_at'"}, "info": {"category": "", "code": "Q001", "extra": {}, "invocation_id": "5b59411c-cd8f-4d7e-8a5b-2ca978c6bd7d", "level": "error", "msg": "Encountered an error while running operation: Database Error\n  Invalid timestamp: '2026-04-03T10:50:50.961498756Z'; while executing the filter on column 'execute_completed_at'", "name": "RunningOperationCaughtError", "pid": 629, "thread": "MainThread", "ts": "2026-04-03T14:45:40.706316Z"}}
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - 2026-04-03 14:45:40 — ERROR — Could not generate the report - Error: Failed to run dbt command.
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - {"info": {"name": "RunningOperationCaughtError", "code": "Q001", "msg": "Encountered an error while running operation: Database Error\n  Invalid timestamp: '2026-04-03T10:50:50.961498756Z'; while executing the filter on column 'execute_completed_at'", "level": "error", "invocationId": "5b59411c-cd8f-4d7e-8a5b-2ca978c6bd7d", "pid": 629, "thread": "MainThread", "ts": "2026-04-03T14:45:40.706316Z"}, "data": {"exc": "Database Error\n  Invalid timestamp: '2026-04-03T10:50:50.961498756Z'; while executing the filter on column 'execute_completed_at'"}}
[2026-04-03, 10:45:40 EDT] {subprocess.py:106} INFO - Please reach out to our community for help with this issue.
Error in CTE (in `cast(execute_completed_at as timestamp)`*):*
Copy code
/* {"app": "dbt", "dbt_version": "1.11.2", "profile_name": "elementary", "target_name": "staging", "connection_name": "macro_log_macro_results"} */

    
    /* --ELEMENTARY-METADATA-- {"command": "run-operation", "elementary_component": "OSS", "invocation_id": "5b59411c-cd8f-4d7e-8a5b-2ca978c6bd7d", "package_name": "elementary", "resource_name": "log_macro_results", "resource_type": "macro"} --END-ELEMENTARY-METADATA-- */
    
  create or replace table `<elem schema>`.`ordered_test_results__tmp_20260403144539479920`
...
dbt_run_results as (
        select * from `<elem schema>`.`dbt_run_results`
        
            where 
        timestamp_diff(current_timestamp, cast(execute_completed_at as timestamp), day)
 < 7
        
    ),
....
Reason: The timestamp
2026-04-03T10:50:50.961498756Z
has 9 decimal places (nanoseconds), while BigQuery supports a maximum of 6 decimal places (microseconds).
Hi <!subteam^S08HYBWCF6E> Perhaps you've already created an issue on GitHub?
a
We too are experiencing this issue, but with the
edr report
command in
dbt-bigquery >=1.11
Environment: -
edr 0.23.1
/ Elementary dbt package 0.23.0 -
dbt-bigquery >=1.11.0
- BigQuery Problem Analysis Starting around March 25th, it seems, nanosecond-precision timestamps (9 fractional digits, e.g.
2026-04-06T23:02:36.794354259Z
) began showing up in Elementary’s
dbt_run_result
table. The
execute_completed_at
,
execute_started_at
,
compile_started_at
, and
compile_completed_at
columns are STRING type, so the values are accepted on insert. However, when
edr report
runs the
elementary_cli.get_test_results
macro, it filters on
execute_completed_at
by casting to TIMESTAMP — and BigQuery’s TIMESTAMP type only supports microsecond precision (6 fractional digits). This causes the query to fail with:
Copy code
Database Error                                                                                                                                                          
    Invalid timestamp: '2026-04-06T23:02:36.794354259Z'; while executing the filter on column 'execute_completed_at'
This completely blocks report generation. The error is not transient — it persists until the bad rows are manually fixed or aged out of the
days_back
window. Workaround: We’re adding a step to our Elementary deployment job in Github Actions which truncates the fractional seconds to 6 digits in the affected table before executing `edr report`:
Copy code
UPDATE project.dataset_elementary.dbt_run_results                                                                                                                        
  SET                                                                                                                                                                        
    execute_started_at = REGEXP_REPLACE(execute_started_at, r'(\.\d{6})\d+Z$', r'\1Z'),                                                                                      
    compile_started_at = REGEXP_REPLACE(compile_started_at, r'(\.\d{6})\d+Z$', r'\1Z'),                                                                                      
    compile_completed_at = REGEXP_REPLACE(compile_completed_at, r'(\.\d{6})\d+Z$', r'\1Z'),                                                                                  
    execute_completed_at = REGEXP_REPLACE(execute_completed_at, r'(\.\d{6})\d+Z$', r'\1Z')                                                                                   
  WHERE REGEXP_CONTAINS(execute_completed_at, r'\.\d{7,}Z$')                                                                                                                 
     OR REGEXP_CONTAINS(execute_started_at, r'\.\d{7,}Z$')                                                                                                                   
     OR REGEXP_CONTAINS(compile_started_at, r'\.\d{7,}Z$')                                                                                                                   
     OR REGEXP_CONTAINS(compile_completed_at, r'\.\d{7,}Z$')
Suggested fix: Elementary’s macros that filter/cast these STRING timestamp columns should either
SAFE_CAST
or truncate to microsecond precision before casting.
cc: @Yosef Arbiv
🙌 1
m
Hi @Dragos Cioata @Dmitrii Drik @Alex Stevens, thanks for the detailed reports 🙏 Quick question to help us pinpoint the root cause: are either of you using dbt-fusion in your setup, or are you running standard dbt-core?
d
For me it was because of version of edr vs dbt elementary data and also the bigquery adapter was not intalled. Now its fine
a
@Mika Kerman hiya — so …. I guess it’s a mix? 😬 i am using dbt-fusion in my local dev environment, but standard dbt-core for production build jobs in dbt Cloud + CI/CD jobs in GitHub. I’ve configured elementary to set all the `upload`/`results` runtime.config arguments to false when the target is dev … that said, every now and then I will run
dbt build -t prod ...
from my local CLI / fusion runtime.
d
@Mika Kerman any update or sugestion on this?
p
Hey @Dragos Cioata @Alex Stevens @Dmitrii Drik - sorry for the delay on this. We identified the root cause and have a fix ready. We added a BigQuery-specific macro that truncates to microsecond precision before casting. This handles it transparently at query time - no need for the UPDATE workaround or the
on-run-end
macro. PR: https://github.com/elementary-data/dbt-data-reliability/pull/1003 If you'd like to try it before the release, you can point your
packages.yml
to the branch:
Copy code
packages:
  - git: "<https://github.com/elementary-data/dbt-data-reliability.git>"
    revision: app-1086-fix-bigquery-nanosecond-timestamp-cast-failure-in
Then run
dbt deps
and try
edr report
/
edr send-report
again. Any feedback would be appreciated!
👍 1
thankyou 1
a
Thank you!
p
Hi @Alex Stevens, Just following up - were you able to try the fix from the branch? If so, did it resolve the
Invalid timestamp
error on your end? Any feedback would be really helpful before we cut the release. Thanks!
a
So, I did test it all end to end, and long story short: The PR fix is correct, but it’s not sufficient on its own to unblock edr report for end users — they’d still get the same error. The elementary-data Python package needs a new release that bundles the patched dbt package. Pinned packages.yml →
revision: app-1086-fix-bigquery-nanosecond-timestamp-cast-failure-in
(head SHA
6b65aa5
), ran dbt deps, then exercised the new
bigquery__edr_cast_as_timestamp
macro four ways: 1. Synthetic edge cases (3 / 6 / 7 / 9 / 11 fractional digits, plus none) — all cast cleanly to microsecond TIMESTAMP. 2. Control — cast('2026-04-03T105050.961498756Z' as timestamp) without the macro: Invalid timestamp as expected. 3. Real production data — pulled rows from
analytics_elementary.dbt_run_results
where execute_started_at has 9 fractional digits, ran them through the macro: all cast successfully. 4. Elementary models that consume the macro —
dbt run -s job_run_results alerts_dbt_models alerts_dbt_source_freshness
on dev: all green, no compile errors. So far so good. Then I installed
elementary-data[bigquery]==0.23.4
and
ran edr report
against the prod Elementary dataset. The gotcha… edr report still failed with the original error:
Copy code
Database Error: Invalid timestamp: '2026-05-22T00:51:19.698903453Z';
while executing the filter on column 'execute_completed_at'
Root cause: the edr CLI doesn’t use the project’s
dbt_packages/elementary/
. It ships its own bundled copy of the dbt package at
site-packages/elementary/monitor/dbt_project/dbt_packages/elementary/
, and that bundled copy is the pre-fix version. The cast that fails (
current_tests_run_results_query.sql
) does call
elementary.edr_cast_as_timestamp
, but dispatch resolves to
default__edr_cast_as_timestamp
because the bundled package has no
bigquery__ override
. To confirm, I cp’d the patched cast_column.sql over the bundled copy and re-ran edr report — generated the full HTML report (28 MB) successfully, no workaround needed. All to say, the fix in the PR is correct, assuming you’ll also cut a new
elementary-data
PyPI release whose bundled
dbt_project/dbt_packages/elementary/
contains the fix — pointing packages.yml at the branch alone doesn't resolve the issue in CLI.
p
Thank you for the thorough testing! Really helpful. You're absolutely right that the fix needs to be bundled into a new
elementary-data
PyPI release for
edr report
/
edr send-report
to pick it up. We'll make sure the next release includes it. Thanks again for validating that the macro itself works correctly 🙏
a
my pleasure!
p
Hey everyone! Thank you for cooperating with the team in this. The fix was just released in Elementary 0.24.0, please let us know that it all works smoothly!