Hello everyone! To kick-start this channel I would love to know what is the biggest challenge you are facing today when ingesting data into the data warehouse?

With Airbyte or in general?

In general and/or with Airbyte.

More out of the box connectors

In general, schema drift is a pain to deal with.. Also, when there are constraints on data, and the data itself doesn't adhere to it. For example, if we define a column to be no more than 10 characters, but the app we're ingesting from is free-form and somebody enters 20 characters, we get errors when the data and data type mismatch, with no easy way to fix it

Deciding between Apache Kafka or Pulsar with a view point to ensuring schema enforcement.

hi @Ari Bajo (Airbyte) . very good question: • add a new table in an existing sync. I need to note down the existing tables, refresh , then re-add all again. a bit time consuming. ◦ also , we have 2 airbyte env. dev and prd. we need to repeat the above mentioned process on prd also. • In some specific cases the sync takes a bit longer.Being able to dynamically change the fetch size would be a big win. from the source definition. • also, when having multiple sync between same source and dest, i need to manually enter each of them so i can see the one i need. A connection name would be great I think you are working on improving that