Hello everyone ! Upgraded airbyte yesterday and most of my migrations stopped working because of the new check on nullable values in cursor: • This release • This PR I'm migrating data from a

VIEW

in Postgres, and when building views, there are apparently no way to tell PG to consider some column

NOT NULL

. So when airbyte performs the query (here is code):

SELECT (EXISTS (SELECT FROM information_schema.columns WHERE table_schema = 'public' AND table_name = 'payment_metrics' AND is_nullable = 'YES' AND column_name = 'trace_start')) AND (EXISTS (SELECT from public."payment_metrics" where "trace_start" IS NULL LIMIT 1)) AS nullValue

(where

payment_metrics

is my view and

trace_start

is the column used as a cursor) It takes forever... I have no way of: • forcing PG to consider

trace_start

as

is_nullable = 'NO'

• creating index so that the

SELECT ... IS NULL LIMIT 1

is fast ◦ although none of the values are null... Is there a way that I can tell airbyte to skip that check?

@Rodi (Airbyte) maybe a edge case to consider in your PR, what is it your opinion? Some users use views as a way to do column selection or filter data in database syncs

In our case we use it to transform the data beforehand (pre-calculate aggregate values, format stuff etc etc)