interesting article by Rudderstack on the future of data engineering https://rudderstack.medium.com/the-future-of-data-engineering-115f0772c9b6 a few points: • coming rise of chief data officers • data becomes important in every single development team (goes hand in hand with the data mesh concept) • data integration will become commoditized (that's us!!)

@John (Airbyte) what are your thoughts on, "Real-Time (and Near Real-Time) Infrastructure will Become Standard"?

I feel this shows Rudderstack’s bias. From what we’ve seen, 95% of use cases don’t need real-time. Communication workflows need it, and some real-time applications (map for instance need it, but otherwise, real time doesn’t add much value. What do you think?

I definitely agree that it isn't required most of the time. But the cost of delivering realtime is getting lower. Technologies like Flink and Materialize are reducing the effort dramatically. You can start to see companies moving from the "lambda" architecture into the "kappa" architecture: • https://eng.uber.com/kappa-architecture-data-stream-processing/ • https://www.alibabacloud.com/blog/implementating-a-real-time-data-warehouse-with-flink_595681 So while I don't think that it is something that very many people actually need, I do think that it's going to become easy enough to implement that it will become a lot more common.