Hello, We're new to airbyte so forgive me if this is stupid 🙂 We started using the marketing connectors to sync data from the ads platform and found it quite surprising that there was no easy way to configure some extra metadata columns on the destination table and a way to add an extra little lambda function/ sql/ whatever to populate them. Working around this is actually quite painful so just wanted to put it on your radar and check with you if we're thinking about this the wrong way. Great experience other than that overall thanks a lot for all your efforts building the product!!

that's exactly what we ended up doing, Airbyte writes raw data per-workspace into its own schema, then on the sync-completed webhook we run a post-sync step that copies it into shared tables with tenant columns injected as SQL literals (not via a JOIN at query time). Curious if that's also something you see in the wild, or if the join-at-read pattern is more common. Thanks!

I've seen both, but more often than not most of us seem to be kicking off a full modeling job (often dbt or Dataform) and handling any joins there as part of the model itself. I've also seen simple scheduled queries, Spark pipelines, you name it. Another emerging pattern are things like incrementally updated Materialized Views in platforms like BigQuery that sometimes allow you to efficiently handle those joins without having to actually duplicate the full data. But most of that gets very warehouse-specific fast. My general rule of thumb is to try and strike the balance between speed, maintainability, and reliability. We're already doing complex modeling, so it makes more sense for me to do it there 9 times out of 10. But if I was just pulling over a simple join-like enrichment, I'd be fine with your method.

Thanks Justin, super helpful! Makes sense, we don’t have dbt in the stack and our enrichment is just adding tenant cols (no complex modeling), so INSERT-INTO-SELECT post-webhook keeps things simple. Will keep an eye on the incremental MV pattern for our dashboard rollups down the line, ClickHouse has Refreshable MVs that look similar. Appreciate the time! 🙏