<@U088WCP3B4P> Hi ! We’re encountering a parsing ...
# ask-elementary-ai
c
@kapa.ai Hi ! We’re encountering a parsing error in Athena when Elementary drops temporary test tables after dbt test. Observed behavior After running
dbt test
with Elementary anomaly tests, Elementary (or the cleanup step triggered through Elementary) issues a DROP TABLE IF EXISTS without quoting the catalog/schema/identifier. Since our schema name contains hyphens, Athena parses “-” as subtraction and fails.
Copy code
Log excerpt 18:33:20.324465 [debug] [MainThread]: On master: -- /* {"app": "dbt", "dbt_version": "1.10.9", "profile_name": "dfn", "target_name": "sales", "connection_name": "master"} / / --ELEMENTARY-METADATA-- {"command": "test", "invocation_id": "18a4eff8-9b0f-42bc-b238-65bc3ce78d77", "package_name": "dfn", "resource_name": "dfn-on-run-end-0", "resource_type": "operation"} --END-ELEMENTARY-METADATA-- */ DROP TABLE IF EXISTS AwsDataCatalog.dfn-sales-dev-dq_elementary.test_ce9b69c0cb_elementary_column_anomalies_bookings_5__null_percent__average__max__zero_percent__price__day__1__country_id__source_id__day__1__day__30__metrics__tmp_20250829163234346922
...
Athena error botocore.errorfactory.InvalidRequestException: An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: line 3:44: mismatched input '-'. Expecting: '.',
What we expect • DROP statements to be rendered with proper quoting, e.g.: DROP TABLE IF EXISTS "AwsDataCatalog"."dfn-sales-dev-dq_elementary"."test_..._metrics__tmp_..." Notes • Project-level quoting for schema/identifier is enabled, but the statement above appears unquoted. • We cannot change the database/schema naming convention (hyphens are mandated). Environment ◦ Warehouse: Amazon Athena (Glue Data Catalog) ◦ dbt-core version: 1.10.9 ◦ elementary version: 0.19.4 ◦ Adapter: dbt-athena (dbt-athena-community wrapper) ◦ Python client: pyathena/botocore ◦ Target schema includes hyphens ◦ dbt project has quoting enabled for schema/identifier Happy to provide full debug logs or run a minimal repro if helpful. Thanks!
k
A message from kapa.ai
Thanks for the clear repro details — this is a known issue with Athena when identifiers (catalog/schema/table) include hyphens and the DROP statements aren’t quoted. What we know from the knowledge sources: - Similar Athena failures occur when Elementary (or its cleanup step) issues unquoted DROP TABLE statements; quoting with backticks or adapter-aware quoting resolves the parse error (mismatched input ‘-’) in Athena ([Slack: schema with dash](https://slack.com/archives/C02CTC89LAX/p1709655251458079)). - The team addressed related Athena cleanup bugs where temporary test tables weren’t fully dropped; the fix introduced adapter-specific cleanup via a fully_drop_relation macro and Athena-specific clean-up, acknowledging Athena’s different behavior for temp tables and file cleanup ([GH #1514](https://github.com/elementary-data/elementary/issues/1514)). - For quoting mismatches in Snowflake/Databricks, Elementary provides a CLI flag --dbt-quoting to override quoting for its internal package, but that applies when running edr commands, not dbt test hooks ([Slack: dbt-quoting flag](https://slack.com/archives/C02CTC89LAX/p1700141940747659), [Slack: on_run_end quoting](https://slack.com/archives/C02CTC89LAX/p1729840602042949)). - Users have worked around Databricks/Athena quoting issues by overriding package macros to enforce proper quoting/backticks for catalogs with special characters ([Slack: Databricks backticks](https://slack.com/archives/C02CTC89LAX/p1696522119299639)). Recommended path forward: 1) Confirm you’re on the latest Elementary dbt package version that includes the Athena cleanup fixes (the fix in GH #1514 was merged). If you’re already on 0.19.4 and still seeing unquoted DROP, it suggests a remaining edge case in the DROP path for Athena ([GH #1514](https://github.com/elementary-data/elementary/issues/1514)). 2) Short-term workaround: override the relevant Elementary macro in your dbt project to enforce adapter-aware quoting for DROP. The pattern used in prior workarounds is to build a Relation via the adapter and render it quoted, or to ensure the catalog/schema are quoted/backticked per adapter rules (similar to the Databricks override example) ([Slack: Databricks backticks](https://slack.com/archives/C02CTC89LAX/p1696522119299639), [Slack: schema with dash](https://slack.com/archives/C02CTC89LAX/p1709655251458079)). 3) If feasible, run a cleanup macro using an adapter-specific fully_drop_relation that calls both adapter.drop_relation and Athena’s clean_up_table to remove catalog metadata and S3 files for temp tables (this pattern was introduced for Athena) ([GH #1514](https://github.com/elementary-data/elementary/issues/1514)). 4) If you’re not invoking edr, the --dbt-quoting flag won’t help for dbt test hooks. The fix needs to be in the package macro path that generates DROP statements during test cleanup ([Slack: on_run_end quoting](https://slack.com/archives/C02CTC89LAX/p1729840602042949)). If you can share a minimal debug log snippet pointing to the exact macro path generating the DROP (e.g., clean_elementary_test_tables or similar), we can confirm where to place the override. If this persists on 0.19.4 with dbt-core 1.10.9, it’s likely a remaining bug — please open an issue and we’ll prioritize an adapter-specific quoting fix for Athena’s DROP path (and happy to review a PR if you have one). ([Slack: schema with dash](https://slack.com/archives/C02CTC89LAX/p1709655251458079), [GH #1514](https://github.com/elementary-data/elementary/issues/1514)).