Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Is there any way to use custom transformations to add a custom column to each source extract? We'd like to do a datetimediff between max(_airbyte_normalized_at) and now() so that we can calculate the hours since the last successful ingestion for some monitoring and alerting purposes

Hi <@U02SS1ZSU8G>, yes I think it's something you can definitely achieve with DBT. But I don't really get why you would do this in a custom transformation for monitoring, because the datetimediff (and now() ) will be computed at transformation time and won't be updated afterward. I'd rather suggest you to create a view and your datawarehouse for this use case such as:
```SELECT
 DATETIME_DIFF(CURRENT_TIMESTAMP(), _airbyte_normalized_at, MINUTE) as normalization_delay
FROM
my_normalized_table```

..."datetimediff (and now() ) will be computed at transformation time"... you are absolutely right :blush: