To run Elementary tests on Airflow using the Astronomer Cosmos library, you need to be aware of some important considerations due to Cosmos' approach of running each dbt model/test as a separate Airflow task. This causes the Elementary on-run-end hook to execute after every model/test, which can lead to performance issues or database locks, especially on platforms like Redshift or Athena. Recommended Approach: 1. Disable Automatic Artifact Uploads: In your

dbt_project.yml

, set the following variable to prevent the on-run-end hook from uploading artifacts after every task:

yaml    vars:      disable_dbt_artifacts_autoupload: true

This prevents Elementary from writing to its metadata tables after every model/test run, reducing contention and performance issues. 2. Run a Separate Job for Elementary: After your main dbt tasks have completed, run a dedicated dbt job to upload all artifacts and run Elementary models/tests in one go:

bash    dbt run -s elementary

This can be orchestrated as a separate Airflow task, ensuring all results are collected and processed together, minimizing database locking and improving performance. Example Airflow DAG Structure with Cosmos: - Use Cosmos to generate your dbt model/test tasks. - Add a final BashOperator or similar task to run

dbt run -s elementary

after all model/test tasks are complete. Note: There is currently no built-in mechanism in Elementary to aggregate artifacts from S3 and upload them to Elementary tables, but this approach is being discussed in the community for future improvements. For now, the above method is the recommended workaround for Cosmos users to avoid performance bottlenecks and ensure reliable Elementary integration. For more details and community discussion, see [this Slack thread](https://slack.com/archives/C02CTC89LAX/p1740748388054459).