hi airbyte community. my startup is trying to build it’s first data pipeline, and there are 2 usecases which have come up: 1. ELT: postgres/mixpanel/salesforce/etc => snowflake, dbt for analytics transforms 2. audit trails so we can ask “what was the state of this object (and related objects) at a specific point in time” (for use by operations, ML, etc) The first one seems like a pretty clear implementation of airbyte, where snowflake has the latest snapshot of the postgres data. 🙂 🎉 For the second one, it sounds like a problem in the event sourcing space: an append only log of object changes, so i can replay the changes at anytime to get the state of the object at a specific time (not just right now). Is there anyone using airbyte to solve this problem (ie: postgres CDC => kafka => snowflake => query to calculate aggregates on the fly)? or do I need to tackle this in the application tier (app => eventstore/kafka => projection application)?

it sounds like snapshot in DBT could help

@Jean-Baptiste Delafosse very cool, didn’t know about that feature yet - I’ll definitely check it out further. (I’m an application engineer, so very quickly diving into the data engineering world). :) for full context, my current goal for a data platform is: • airbyte + dbt for ELT • snowflake as datalake • dagster as orchestrator • dagster + dbt + great expecations for dbt “unit tests” (similar to shopify’s python unit tests w/ Seamster) • airbyte for snowflake to kafka for updates to rails app • airbyte for reverse etl to saas apps • metabase for BI (then looker as we grow)

airbyte for reverse etl to saas apps

this is on your plans @Tom Hallett 🔥