It'd be great if users could see table freshness s...
# feature-requests
j
It'd be great if users could see table freshness somewhere in Lightdash such as a badge on the table explore page showing "last updated 2h ago" (perhaps even with green/yellow/red against an SLA.) Two ways I could see this working: 1) Read from warehouse metadata. One cheap query per warehouse covers every table: • Snowflake:
INFORMATION_SCHEMA.TABLES.LAST_ALTERED
• BigQuery:
INFORMATION_SCHEMA.TABLE_STORAGE.last_modified_time
• Redshift:
SVV_TABLE_INFO
/
STL_INSERT
2) Let users define a
loaded_at
column for views, business-time freshness, or where metadata timestamps are:
Copy code
meta:
  lightdash:
    freshness:
      loaded_at_field: updated_at
      warn_after: { count: 6, period: hour }
      error_after: { count: 24, period: hour }
p
šŸ‘‹ Thanks for your message - someone from the Lightdash support team will get back to you asap. Feel free to add any additional context to the thread here in the meantime (screenshots, app version if you're self-hosting etc.).
j
Hey John šŸ‘‹ both the warehouse metadata approach and the loaded_at config idea make a lot of sense. This doesn't exist in Lightdash today, but we do have an open feature request tracking it here: https://github.com/lightdash/lightdash/issues/621 I'll upvote so you'll get updated in this thread when we work on it! In the meantime the closest thing would be running dbt source freshness in your orchestrator to monitor staleness, but that obviously doesn't get you the in-app badge experience you're describing. Let me know if you have any questions!
šŸ™Œ 1