<@U088WCP3B4P> I'm processing artifacts like `dbt_...
# ask-elementary-ai
r
@kapa.ai I'm processing artifacts like
dbt_run_results
and
dbt_invocations
. We have many teams running multiple dbt projects, so these datasets keep growing and there's no partitioning. Because these seem to be build incrementally, I think I can just update the tables myself. But before I start hanging around, I wanted to confirm that I'm not missing something, like some built-in config, or some other way that this would break.
k
A message from kapa.ai
You’re right that
dbt_run_results
and
dbt_invocations
are incremental history tables, and there is no built‑in lifecycle/retention mechanism from Elementary today. From the docs and Slack threads: • These tables are incremental and keep growing: they “insert data for the amount of tests runs / model runs / invocations / etc.” [Lifecycle & growth]. • There is *no automatic retention policy*; you’re expected to manage size yourself (e.g., delete old rows using
created_at
/ timestamps), with the caveat that the Elementary report can only show periods that still exist in those tables. [Lifecycle & growth]. •
dbt_invocations
is explicitly documented as an incremental model that appends one row per invocation. [dbt_invocations model]. So: • You can safely manage these tables yourself (e.g., periodic deletes of old data, or moving old data to your own partitioned tables) as long as you’re okay with losing historical coverage in Elementary reports/analyses beyond whatever window you keep. • There is no hidden config that will auto‑partition or auto‑prune these models; any partitioning or retention has to be implemented on your side (e.g., warehouse‑level partitioning, scheduled cleanup jobs). • Be careful not to change the schema or primary keys; stick to deleting or archiving rows, not altering columns or constraints, so Elementary’s incremental logic continues to work as expected. If you want to be conservative, start with a cleanup that only removes very old data (e.g., > 6–12 months) and verify that your reports still show the expected time range. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.