shobia
11/24/2025, 6:22 AMkapa.ai
11/24/2025, 6:22 AMkapa.ai
11/24/2025, 6:22 AMcolumn_anomalies (and sometimes volume_anomalies) run without a timestamp_column: the generated SQL includes an extra comma in ... as start_bucket_in_data, from ..., which causes a syntax error in several warehouses (Redshift, Databricks, Trino). [[Redshift bug](https://github.com/elementary-data/elementary/issues/1540); [Databricks bug](https://github.com/elementary-data/elementary/issues/1554)] In your error: > syntax error at or near "from" in context "as start_bucket_in_data, from" that’s exactly the same pattern as described in the open GitHub issues. [[Redshift bug](https://github.com/elementary-data/elementary/issues/1540); [Databricks bug](https://github.com/elementary-data/elementary/issues/1554)] ### Why it happens When no timestamp_column is configured, Elementary should monitor the full table over runs (no time bucketing), which is a fully supported mode. [[Column anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies); [Timestamp behavior](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/timestamp-column)] But in affected versions (e.g. package 0.15.2), the macro that builds the monitoring query has a trailing comma in the SELECT when timestamp_column is missing, producing SQL like: sql select <your_column>, {{ elementary.null_timestamp() }} as start_bucket_in_data, from monitored_table This is exactly what the GitHub issues report. [[Redshift bug](https://github.com/elementary-data/elementary/issues/1540); [Databricks bug](https://github.com/elementary-data/elementary/issues/1554)] ### How to fix / work around it You have two options, depending on whether you can use a timestamp: #### 1. Recommended: add timestamp_column (workaround and better behavior) If your model has a creation/loaded timestamp, configure it so Elementary buckets by time and avoids the broken code path: yml models: - name: item_branch__parquet_common_refined4 config: elementary: timestamp_column: itembranch_creation_date # or another appropriate timestamp column columns: - name: "itembranch.creation date" tests: - elementary.column_anomalies: column_anomalies: - null_count anomaly_sensitivity: 2 anomaly_direction: both ignore_small_changes: spike_failure_percent_threshold: 3 drop_failure_percent_threshold: 3 tags: ["elementary", "daily_elementary", "refined4", "common_refined4"] With `timestamp_column`: - Elementary splits data into time buckets using that column. - It compares null_count across buckets (e.g. yesterday vs previous days) to detect spikes/drops. [[Timestamp behavior](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/timestamp-column); [Column anomalies config](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies#test-configuration)] This is the same workaround other users used to bypass the bug. [[Databricks bug](https://slack.com/archives/C02CTC89LAX/p1716306300184349)] #### 2. If you truly cannot use a timestamp The docs say tests should work without a timestamp_column and just compare metrics between test runs. [[Timestamp behavior](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/timestamp-column); [Column anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies)] However, the current open issues show that in some versions/warehouses this path is broken by the extra comma. [[Redshift bug](https://github.com/elementary-data/elementary/issues/1540); [Databricks bug](https://github.com/elementary-data/elementary/issues/1554)] Since the knowledge sources do not state which exact version contains the fix, I cannot confidently say “upgrade to version X and it will be solved.” The maintainers mention a PR was merged “in the near future,” but no tagged version is given. [[1554 comments](https://github.com/elementary-data/elementary/issues/1554)] Given that, your realistic options are: - Temporarily add a suitable timestamp column (best and simplest workaround). - Or, if you know you’re on a version with this bug (e.g. 0.15.2), upgrade Elementary dbt package / CLI to the latest available in your environment and re-run; if the issue persists, it’s still the same open bug. ### Why your current config fails Your test (without a timestamp): yml - name: "itembranch.creation date" tests: - elementary.column_anomalies: column_anomalies: - null_count anomaly_sensitivity: 2 anomaly_direction: both ignore_small_changes: spike_failure_percent_threshold: 3 drop_failure_percent_threshold: 3 tags: ["elementary", "daily_elementary", "refined4", "common_refined4"] is conceptually correct and matches the documented pattern for running column_anomalies without a timestamp_column. [[Column anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies)] The database error is not because of your YAML, but because of the known SQL generation bug on the no-timestamp path. --- Summary - Your YAML is valid; the error is from a known bug in the generated SQL when timestamp_column is omitted. - To detect “yesterday vs today” spikes in null_count, configure a timestamp_column for the model and keep your test as is. - If you cannot use a timestamp, you’re blocked by this bug until you’re on a version where that PR is released; the knowledge sources don’t specify which version that is.