Yes, this is a known issue when using Elementary with Spark/Databricks and columns of type

decimal(p, s)

, such as

decimal(28, 6)

. The problem arises because earlier versions of Elementary did not fully support the

decimal

type in Spark, and the type normalization logic sometimes failed to recognize or handle

decimal(p, s)

columns correctly. This could lead to attempts to cast or convert these values to types like

float

, resulting in out-of-range or precision errors, especially for large or highly precise numbers. A similar issue was reported where numeric column anomaly tests on a

decimal

column in Databricks resulted in blank graphs or errors, while the same test worked fine on an

int

column. The root cause was that the

decimal

type was not included in the list of recognized numeric types in the Elementary macros. The fix involved updating the macro to recognize

decimal

types, but it was also noted that the Spark adapter returns types in the format

decimal(p,s)

, so the macro needed to handle this pattern, not just the string

'decimal'

alone. A patch was released to address this, and you can find the details and the fix here: [GitHub PR #294](https://github.com/elementary-data/dbt-data-reliability/pull/294) (as referenced in the Slack thread) https://slack.com/archives/C02CTC89LAX/p1679047291649449. Workarounds and Recommendations: - If you are on an older version of Elementary, upgrade to the latest version where this fix is included. - If you need an immediate workaround, you can locally patch the macro that normalizes data types to ensure it recognizes

decimal(p,s)

as numeric. The macro is typically found at

dbt_packages/elementary/macros/utils/data_types/data_type_list.sql

. Make sure it matches

decimal

types with or without precision/scale. - If you still encounter issues, consider setting Spark’s

ansi_mode

to

false

as a temporary measure, but be aware this may cause out-of-range values to be returned as NULL instead of raising errors. If you are already on a recent version (such as 0.18.3) and still see this error, it’s possible that the fix has not fully addressed all edge cases, or there may be a new regression. In that case, please consider opening an issue on the Elementary GitHub or reaching out on Slack with a minimal reproducible example and your configuration details. Summary: This is a recognized issue with decimal handling in Spark/Databricks workflows in Elementary, and there is a documented fix. Upgrading to the latest version or patching the macro as described should resolve the problem for most cases. If not, further investigation may be needed based on your specific workflow and data. For more details, see the discussion and solution here: [Elementary Slack thread on decimal support in Databricks](https://slack.com/archives/C02CTC89LAX/p1679047291649449).