🎉 Ververica Announces Public Availability of Bring Your Own Cloud (BYOC) Deployment Option on AWS Marketplace Enabling Ultra-High Performance and Scalable Real-Time Data Streaming Solutions on Organizations' Existing Cloud Infrastructure Ververica, creators of Apache Flink® and a leader in real-time data streaming, today announced that its Bring Your Own Cloud (BYOC) deployment option for the Unified Streaming Data Platform is now publicly available on the AWS Marketplace. This milestone provides organizations with the ultimate solution to balance flexibility, efficiency, and security in their cloud deployments. Building on Ververica’s commitment to innovation, BYOC offers a hybrid approach to cloud-native data processing. Unlike traditional fully-managed services or self-managed software deployments, BYOC allows organizations to retain full control over their data and cloud footprint while leveraging Ververica’s Unified Streaming Data Platform; by deploying it on a zero-trust cloud environment. “Organizations face increasing pressure to adapt their cloud strategies to meet operational, cost, and compliance requirements,” said Alex Walden, CEO of Ververica. “BYOC offers the best of both worlds: complete data sovereignty for customers and the operational simplicity of a managed service. With its Zero Trust principles and seamless integration into existing infrastructures, BYOC empowers organizations to take full control of their cloud environments.” Key Benefits of BYOC Include: • Flexibility: BYOC integrates seamlessly with a customer’s existing cloud footprint and invoicing, creating a complete plug-and-play solution for enterprises’ data processing needs. • Efficiency: By leveraging customers’ existing cloud resources, BYOC maximizes cost-effectiveness. Organizations can leverage their negotiated pricing agreements and discounts; all while avoiding unnecessary networking costs. • Security: BYOC’s design is built on Zero Trust principles, ensuring the customer maintains data governance within the hosted environment. • BYOC further embodies Ververica’s “Available Anywhere” value, which emphasizes enabling customers to deploy and scale streaming data applications in whichever environment is most advantageous to them. By extending the Unified Streaming Data Platform’s capabilities, BYOC equips organizations with the tools to simplify operations, optimize costs, and safeguard sensitive data. For more information about Ververica’s BYOC deployment option, visit the AWS Marketplace listing or learn more through Ververica’s website.

Datorios Announces New Search Bar for Apache Flink Datorios' new search bar for Apache Flink makes navigating and pinpointing data across multiple screens effortless. Whether you're analyzing job performance, investigating logs, or tracing records in lineage, the search bar empowers you with: Auto-complete suggestions: Build queries step-by-step with real-time guidance. Advanced filtering: Filter by data types (#TIME, #STRING, #NUMBER, etc.) and use operators like #BETWEEN, #CONTAINS, and #REGEX. Logical operators: Combine filters with #AND, #OR, and parentheses for complex queries. Query management: Easily clear or expand queries for improved readability. Available across all investigation tools: tracer, state insights, job performance, logs, and lineage. Try it out now and experience faster, more efficient investigations: https://datorios.com/product/

Hello! I'd like to start contributing with the project, starting with the rabbitmq connector upgrade, is there any current plan or thread related to it? and some guidelines to participate as dev?

Hello folk, I wanted to discuss our current system setup and see if Apache Flink would be a good fit to replace one of our consumer groups. Right now, we have two Kafka consumer groups: the raw-consumer group, which is scalable and handles the input from vehicles, and the parsed-consumer group, which handles the post-processing tasks like decoding and filtering. The issue we’re facing is that while the raw-consumer group can scale effectively with the number of vehicles, the parsed-consumer group starts to slow down when we increase the load, especially as the tasks it performs are quite CPU-intensive. Currently, we’re running the system with 20 vehicles, and here’s the resource breakdown: We have 20 pods, each allocated 150 millicores (0.15 vCPU) and 400 MB of memory. That totals 3 vCPUs and 8 GB of memory. However, when scaling to 10,000 vehicles, we’re expecting a significant increase in resource usage. Specifically, we’ll need 250 pods, which translates to 37.5 vCPUs and 100 GB of memory. Given this, we’re wondering if replacing the parsed-consumer group with Apache Flink would be a good solution. Flink is known for handling heavy processing tasks like decoding and filtering efficiently, and we believe it could scale better to meet the needs of our growing vehicle fleet. Would Flink be able to handle this load more effectively than our current parsed-consumer group? And if we were to make this transition, would it provide the scalability we need without running into performance issues or excessive resource consumption? I just wnat some good suggestions? As no other company having similar use case i couldn't find any article or so.

Hi all, I'm working on FLINK-36429*: Enhancing Flink History Server File Storage and Retrieval with RocksDB*. I've created the Jira ticket and submitted a PR. Since a FLIP is required, could anyone guide me on how to get an account for submission? Thanks!

Oh since I'm here going to share my only article I have ever written https://dev.to/calificio/my-goofy-attempt-on-building-a-flink-bigquery-source-connector-336c

Hey all, Just dropping a quick note that Data Council is back with our best speaker lineup to date on Apr 22-24 in Oakland. This year’s conference features heavy hitters like Lloyd Tabb (Looker founder), Hannes Mühleisen (DuckDB co-creator), Ryan Blue (Apache Iceberg creator), plus experts from Dagster, Clickhouse, Modal Labs, Hex, and more! We’ll have over 100+ speakers across 10 tracks covering data eng, ML Ops, GenAI, databases, etc. Technical deep dives only - no marketing fluff. For the full lowdown, visit us here: https://www.datacouncil.ai/BAY-2025 Early Bird ENDS TODAY at midnight so don’t miss out!

Hey everyone, Excited to announce that Datorios now fully supports all join types in Flink SQL/Table API for streaming mode! What’s new? • Full support for inner, left, right, full, lookup, window, interval, temporal, semi, and anti joins • Enhanced SQL observability—detect bottlenecks, monitor state growth, and debug real-time execution • Improved query tracing & performance insights for streaming SQL With this, you can enrich data in real time, correlate events across sources, and optimize Flink SQL queries with deeper visibility. Release note: https://datorios.com/blog/flink-sql-joins-streaming-mode/ Try it out and let us know what you think!

📣 Hello Flink community! 👋 I wanted to share an opportunity that might interest those looking to deepen their Flink expertise. The Ververica Academy is hosting successful Bootcamp in several cities over the coming months: • Warsaw, Poland: 6-7 May 2025 🇵🇱 • Lima, Peru: 27-28 May 2025 🇵🇪 • New York City: 3-4 June 2025 🇺🇸 • San Francisco: 24-25 June 2025 🇺🇸 This is a 2-day intensive program specifically designed for those with 1-2+ years of Flink experience. The curriculum covers practical skills many of us work with daily - advanced windowing, state management optimization, exactly-once processing, and building complex real-time pipelines. Participants will get hands-on experience with real-world scenarios using Ververica technology. If you've been looking to level up your Flink skills, this might be worth exploring. For all the details click here! We have group discounts for teams and organizations too! As always if you have any questions, please reach out to me directly. 😎

✍️ Blogged: Joining two streams of data with Flink SQL https://rmoff.net/2025/03/06/joining-two-streams-of-data-with-flink-sql/

Created a live kafka-like traffic simulation, that can even detect out or order events depending on what you do with your producers/partitions. https://evoura.com/kafka-traffic-visualizer/

✍️ Blogged: Data Wrangling with Flink SQL https://rmoff.net/2025/03/10/data-wrangling-with-flink-sql/

📣 Flink Forward Barcelona 2025 Announcement! Ververica is excited to share details about the upcoming Flink Forward Barcelona 2025! 📅 Dates: 13-16 October 2025 📍 Location: Fira de Barcelona Montjuïc The event will follow our successful our 2+2 day format: • Days 1-2: Ververica Academy Learning Sessions • Days 3-4: Conference days with keynotes and parallel breakout tracks 🎟️ Special Promotion: We're offering a limited number of early bird tickets! Sign up for pre-registration to be the first to know when they become available here. 📣 Call for Presentations will open in April - please share with anyone in your network who might be interested in speaking! Feel free to spread the word and let me know if you have any questions. Looking forward to seeing you in Barcelona! 🇪🇸 🔽 🔽 🔽 🔽 Don't forget, Ververica Academy is hosting four intensive, expert-led Bootcamp sessions. • Warsaw, Poland: 6-7 May 2025 • Lima, Peru: 27-28 May 2025 • New York City: 3-4 June 2025 • San Francisco: 24-25 June 2025 This 2-day program is specifically designed for Apache Flink users with 1-2 years of experience, focusing on advanced concepts like state management, exactly-once processing, and workflow optimization. ➡️ Click here for information on tickets, group discounts, and more!

🎉 📣 Join @Giannis Polyzos Ververica's Staff Streaming Product Architect, as he introduces , the next evolution of streaming storage built for real-time analytics. 🌊 ▶️ Discover how Apache Flink®, the industry-leading stream processing engine, paired with Fluss, a high-performance transport and storage layer, creates a powerful, cost-effective, and scalable solution for modern data streaming. 🔍 In this session, you'll explore: • Fluss: The Next Evolution of Streaming Analytics • Value of Data Over Time & Why It Matters • Traditional Streaming Analytics Challenges • Event Consolidation & Stream/Table Duality • Tables vs. Topics: Storage Layers & Querying Data • Changelog Generation & Streaming Joins: FLIP-486 • Delta Joins & Lakehouse Integration • Streaming & Lakehouse Unification 📌 Learn why streaming analytics require columnar streams, and how Fluss and Flink provides sub-second read/write latency that offers 10x read throughput improvement over row-based analytics. ✍🏻 Subscribe to stay updated on real-time analytics & innovations! 🔗 Join the Fluss community on GitHub 👉🏻 Don't forget about Flink Forward 2025 in Barcelona and the Ververica Academy Live Bootcamps in Warsaw, Lima, NYC and San Francisco.

🤩 A ton of interesting links in the data space this month, covering DuckDB, Kafka, Iceberg, Flink, Postgres, and much more! 🔗 https://rmoff.net/2025/03/24/interesting-links-march-2025/

Hi, there are lot of blogs on flink from

decodable

which can help in getting started. Just wanted to share here https://www.decodable.co/blog

Hi there , looking for blogs on flink 2.0 AI Capabilities , specially using the flink processors with Knowledge Graphs and RAGs in real time. Any pointers would be great. Thanks

This message was deleted.

Hello Flink Team, I hope you're doing well. I am currently working with Flink 2.0 (latest version) and would appreciate your guidance on which Kafka connector version (Maven library) is recommended for optimal compatibility and performance with this release. Thank you for your help!

📣 Call for Presentations is OPEN for Flink Forward 2025 in Barcelona Do you have a data streaming story to share? We want to hear all about it! The stage could be yours! 🔥 Hot topics this year include: 🔹 Real-time AI & ML applications 🔹 Streaming architectures & event-driven applications 🔹 Deep dives into Apache Flink & real-world use cases 🔹 Observability, operations, & managing mission-critical Flink deployments 🔹 Innovative customer success stories 📅 Flink Forward Barcelona 2025 is set to be our biggest event yet! Join us in shaping the future of real-time data streaming. ⚡ Submit your talk here. ▶️ Check out Flink Forward 2024 highlights on YouTube and all the sessions for 2023 and 2024 can be found on Ververica Academy. Ticket sales will open soon. Stay tuned.

📣 Current London Happy Hour 2025 🇬🇧 • Monday, May 19, 2025 • 5:30pm — 7:30pm Engel Bar Royal Exchange, City of London, London EC3V 3LL, UK Start Current London 2025 off in style with Redpanda, Conduktor and Ververica! Join us for a happy hour at Engel Bar located on the north mezzanine inside The Royal Exchange. Connect with a diverse group of thought leaders, innovators, analysts, and top practitioners across the entire data landscape. Whether you're into data streaming, analytics, or anything in between, we’ve got you covered. ‍ RSVP here. Cheerio and we all hope to see you there mate 😉

🌟 Flink Forward Asia 2025 is coming to Singapore! Call for Presentations! Mark your calendars—the Apache Flink community’s premier tech conference comes to Singapore on July 3, 2025! Calling all developers, data engineers, and real-time computing innovators! Flink Forward Asia 2025 is now accepting talk proposals for its flagship event in Singapore this July! 🌐 ✨ Why Submit? Showcase your work to Apache Flink’s global community of core contributors, industry leaders, and academic pioneers. Dive into cutting-edge topics: real-time AI, real-time lakehouse (with Apache Paimon!), real-time analytics, and more. Gain global exposure and shape the future of streaming technologies. 📢 We Want Your Insights! Whether you’re pushing boundaries with Apache Flink, Flink CDC, pioneering Real-time AI, or building next-gen analytics pipelines, this is your stage! Share actionable tech deep dives, battle-tested use cases, or bold visions for real-time computing. 🔑 Key Dates Submission Deadline: May 30, 2025 Speaker Notification: June 9, 2025 Event Day: July 3, 2025 🎤 Submit Your Talk Today! https://asia.flink-forward.org/singapore-2025/call-for-presentations

Hello there folks! Just posted about 🪟 and flink in

The case for a Custom Window in Flink

https://pedromazala.substack.com/p/the-case-for-a-custom-window-in-flink

Collection of interesting stuff about Flink and beyond from the last month: https://rmoff.net/2025/04/22/interesting-links-april-2025/

Hi Flink Team, Is anyone interested in a native accelerator for Apache Flink? I am Hongze from the Apache Gluten (incubating) community. We are currently working on a native Flink accelerator backed by Meta Velox. The work is in its earliest stage and we are aiming to get a PoC delivered in the next phase. Please let me know if anyone is interested with the topic. Thanks! Could view the project page to check what we have achieved on Apache Spark: https://github.com/apache/incubator-gluten

And this is why OLake (Open Source) is fast! Here's something for your weekend read: Exploring OLake's Architecture. If you're diving into real-time data replication or building modern data lakehouse architectures with Apache Iceberg, we've just shared an in-depth look at how OLake actually works behind the scenes. Whether your stack includes MongoDB, PostgreSQL, or MySQL, and you're targeting formats like Apache Iceberg or Parquet, this article has practical insights on designing scalable, efficient data pipelines. OLake is an open-source tool specifically built for high-speed data ingestion. Key Highlights: ⚡ Speed: Load data 4x to 10x faster compared to traditional ETL tools. 🕒 Real-Time CDC: Minimal-lag Change Data Capture from MongoDB, PostgreSQL, and MySQL. 🧩 Plug-and-Play Architecture: Cleanly separated core, drivers, and writers make extending OLake straightforward. 📊 Schema Flexibility: Seamlessly handles schema evolution and type changes compatible with Apache Iceberg. 🔄 Reliable Syncs: Built-in state management means your sync operations can resume effortlessly if interrupted. https://olake.io/blog/olake-architecture-deep-dive

_With apologies for writing a long blog, as I didn't have time to write a shorter one_—here are my notes as I tried to learn about watermarks in Apache Flink SQL: https://rmoff.net/2025/04/25/its-time-we-talked-about-time-exploring-watermarks-and-more-in-flink-sql/ Please let me know if you spot any errors 🙂

📣 Early Bird tickets for Flink Forward are now on sale! Ververica is thrilled to announce that Early Bird ticket sales are open for Flink Forward 2025, taking place October 13–16, 2025 in Barcelona. Secure your spot today and save 30% on conference and training passes! That means that you could get a conference-only ticket for €699 or a combined conference + training ticket for €1399! 👉🏻 ❗ Early Bird tickets will only be sold until May 31. Grab your discounted ticket before it's too late! Why Attend Flink Forward Barcelona? • Cutting‑edge talks: Learn from top engineers and data architects about the latest Apache Flink® features, best practices, and real‑world use cases. • Hands-on learning: Dive deep into streaming analytics, stateful processing, and Flink’s ecosystem with interactive, instructor‑led sessions. • Community connections: Network with hundreds of Flink developers, contributors, PMC members and users from around the globe. Forge partnerships, share experiences, and grow your professional network. • Barcelona experience: Enjoy one of Europe’s most vibrant cities—sunny beaches, world‑class cuisine, and rich cultural heritage—all just steps from the conference venue. 📌*Grab your Flink Forward Insider ticket today and see you in Barcelona!*