Hiya. Some call FLINK-7129 the most watched issue of all time. The FLIP discussion seems to have stalled a couple of years back. I came to this issue through searching for a tool that allows pattern processing in a setup consisting of a control plane (adding rules to a table) and a data plane (processing those rules), i.e. Dynamic CEP. It should be a pretty solid case for Flink, but somehow developing the functionality has stalled. There’s some existing work; a relatively newer one that I don’t think is open-source: https://www.alibabacloud.com/blog/application-of-dynamic-flink-cep-at-bank-of-hangzhou_602041 They’re building on FLIP-200 and Ververica’s Dynamic Flink CEP demo code.

I think I have found a bug (or at least oddity) in some Flink-managed k8s pods where the

ownerReference

on Pods are being set to an unrelated

Deployment

(as opposed to the

FlinkDeployment

who is the real owner controlling the pods' creation). This ends up creating unexpected results in downstream metrics (investigation here). I think ArgoCD does a good job of illistrating the

ownerReference

relationships for visual learners.

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: apache-feed-job-notifications
    <http://app.kubernetes.io/instance|app.kubernetes.io/instance>: nis-apache-feed-app
    <http://app.kubernetes.io/name|app.kubernetes.io/name>: apache-flink-feed-job
    component: taskmanager
    <http://helm.sh/chart|helm.sh/chart>: apache-flink-feed-job-0.0.3
    type: flink-native-kubernetes
  name: apache-feed-job-notifications-taskmanager-5-39
  namespace: nis
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: Deployment
    name: apache-feed-job-notifications
spec:
  {...}

I am waiting on a JIRA account to file an issue but thought I would at least start a discussion here to get the community's appetite for aligning these pods'

ownerRefernce

to be more consistent with k8s best practices(?). I am a bit new to all the intrecicies of

ownerReference

so maybe there is some part I am missing here but at minimum the current behavior seems strange that a 3rd party controller is setting an

ownerReference

to a different k8s kind outside of it's domain, no?

Hi devs, The vote thread for flink-connector-aws v5.1.0 RC1 is open at https://lists.apache.org/thread/5zpjy2s1gxjlm9q8n5w52lg3gxsgfgdp A few devs and I have tested it and shared the results on the thread, and currently looking for more reviewers, testers and voters. If you are interested to review, vote or simply to find out more about the connector, please do not hesitate to reach out to me on Slack or email! Thank you!

Hey All, why is there no

java21

docker image for flink

1.19.X or 1.20.X

?

Hi, masters, While conducting preliminary POC validation for FLIP-495, I noticed several spelling issues. Therefore, I’ve submitted a PR https://github.com/apache/flink/pull/26820 attempting to fix it. If anyone has time, could you please help review it? Many thanks!

Does anyone know why many connectors got abandoned after 1.18? Looks like even when there are PRs open to update them, there is no one to review them and release updates.

Hello devs, As a part of this initiative : SourceFunction API removal i am working on this task https://issues.apache.org/jira/browse/FLINK-32695 . Since many test cases still use the legacy Source API, I’ve compiled a spreadsheet that lists the SourceFunctions involved and their corresponding PRs updating them to Source V2. The first set of PRs is already up, and I plan to complete most of the remaining ones over the next 1-2 weeks. (Code is almost done, just need to push the PR's) To keep things manageable and reviewable, I’ve kept the changes isolated per test case. I’d really appreciate the support of committee members for reviewing and merging these PRs. If you're available to help, please let me know — I’ll be happy to tag you on the current and upcoming PRs next week. Thank you so much for your support!

👋 Hello, looking for Flink Runtime maintainers to look at this change https://github.com/apache/flink/pull/26811 It corrects the hostname resolution between the BlobServer and BlobClient in Flink clusters when a Jobmanager bind-host is set.

👋 Hello, Devs. Kindly Requesting votes/discussion on FLIP-495. I have done the POC[1] for FLIP-495[2] to provide you with a concrete sample for discussion[3], voting[4] review. Currently, - FLIP-495 [2] is waiting for a final review and is lacking a required binding vote at least (get 2 binding votes and 2 non-binding votes now). - FLIP-487 [5] is pending the completion of FLIP-495[1].(The UI design is completed.) It would be greatly appreciated if you could share some feedback on discussion[3] or help with the voting[4]. [1]https://github.com/RocMarshal/flink/pull/7 [2]https://cwiki.apache.org/confluence/display/FLINK/FLIP-495%3A+Support+AdaptiveScheduler+record+and+query+the+rescale+history [3]https://lists.apache.org/thread/t3r9wdd5gpbqnvzw35kb3wb3d9brpnon [4]https://lists.apache.org/thread/7v10k44kgkfmn3dftmb7loslom4hndy5 [5]https://cwiki.apache.org/confluence/display/FLINK/FLIP-487%3A+Show+history+of+rescales+in+Web+UI+for+AdaptiveScheduler CC @Matthias @rmetzger

Hi All, I want to consume messages from RabbitMQ (super streams) using Flink connectors. I explored RabbitMQ connectors for Flink at https://nightlies.apache.org/flink/flink-docs-release-2.1/docs/connectors/datastream/rabbitmq/ but this page only explains on consuming messages from RabbitMQ queue and not from the RabbitMQ super streams. More details on RabbitMQ streams can be found here https://www.rabbitmq.com/docs/streams I'm looking out for an out of the box Flink connector which provides fault tolerance and scalability to connect to RabbitMQ streams and consume messages using Flink?

Kindly ping~ @Sergey Nuyanzin @Leonard Xu https://issues.apache.org/jira/browse/FLINK-33463 https://github.com/apache/flink-connector-jdbc/pull/161

hello everyone, could someone take a look on the following PRs for bumping commons-lang3 version please? Thanks! https://issues.apache.org/jira/browse/FLINK-38193 - https://github.com/apache/flink/pull/26869 https://issues.apache.org/jira/browse/FLINK-38159 - https://github.com/apache/flink-kubernetes-operator/pull/1002

Hello everybody, I’d like to start a discussion about FLIP-490: Enhanced Job History Retention Policies for HistoryServer[1][2] Any input and feedback would be appreciated !🌹 [1] https://lists.apache.org/thread/vo6obg2yhhrt6g3nzf77g9xwtxo0j3wg [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=332499857

Kindly ping~ @Sergey Nuyanzin @Leonard Xu https://issues.apache.org/jira/browse/FLINK-34467 https://github.com/apache/flink-connector-jdbc/pull/149

Hello devs, is it in the roadmap to introduce a HyperLogLog Operator for DataStream API? I have seen that it has just been introduced for SQL API

hello folks, just a kindly ping @Gyula Fóra @Máté Czagány about https://issues.apache.org/jira/browse/FLINK-38253 https://github.com/apache/flink/pull/26908 , could you please take a look once you will have some spare time? Thanks a lot

👋 We have a community approved PR, how do we merge this? https://github.com/apache/flink/pull/26811

Hi Folks, I'm trying to find out how widespread the use of Session Clusters are. Any of you doing that, or know of any companies that are?

Hey folks, I was hoping we could get a new release (5.0?) of the Flink Connector Kafka?

Hey guys does anybody know when flink-connector-jdbc for Flink 2.x is planned to be released? I noticed v4.0.0 tag was created 5 months ago, it seems to be compatible with 2.0. https://github.com/apache/flink-connector-jdbc/blob/v4.0.0/pom.xml#L59 Yet on maven repo it is up to 3.3.0-1.20, https://mvnrepository.com/artifact/org.apache.flink/flink-connector-jdbc Thanks

Hey everyone, would love some 👀 on this as AWS SDK V1 is EOL in December 2025

Hello folks! I opened a PR to contribute to this Jira issue I saw that the CI Github actions have to be approved by a maintainer to run. Can anyone help me with it? Thanks in advance

Hello, everyone. Flink CI test results are currently unstable, with more than 10 flaky tests. We need developers' help to resolve them. I’ve assigned some, but many failures need community support. Below is the detailed list: • https://issues.apache.org/jira/browse/FLINK-38404: JobMasterTest consistently fails. • https://issues.apache.org/jira/browse/FLINK-38402: EventTimeWindowCheckpointingITCase fails frequently. • https://issues.apache.org/jira/browse/FLINK-38439: SqlGatewayServiceITCase fails for the first time. • https://issues.apache.org/jira/browse/FLINK-38441: ExecutionGraphCoLocationRestartTest is not frequently reproducible. • https://issues.apache.org/jira/browse/FLINK-38408: MapStateNullValueCheckpointingITCase is not frequently reproducible. • https://issues.apache.org/jira/browse/FLINK-38407: GroupAggregateRestoreTest is not frequently reproducible. • https://issues.apache.org/jira/browse/FLINK-38406: DefaultSchedulerCheckpointCoordinatorTest is not frequently reproducible. • https://issues.apache.org/jira/browse/FLINK-38405: ChangelogNormalizeRestoreTest is not frequently reproducible. • https://issues.apache.org/jira/browse/FLINK-38403: UnalignedCheckpointITCase is not frequently reproducible. • https://issues.apache.org/jira/browse/FLINK-38369: RestClientTest is not frequently reproducible. • https://issues.apache.org/jira/browse/FLINK-38360: HadoopRecoverableWriterTest is not frequently reproducible. Thanks all ~ 😃

Hi there, looking for reviewers for this implementation for enabling FlinkSQL gateway "read-only" mode. https://issues.apache.org/jira/browse/FLINK-38516 https://github.com/apache/flink/pull/27118

hi, I opened this bug "Silent record dropping when exceeding Kafka max.request.size": https://issues.apache.org/jira/browse/FLINK-38529 Can someone take a look at it?

Hi all, I have logged a bug https://issues.apache.org/jira/browse/FLINK-38522 and opened a PR to address it, looking for anyone who might be able to review? Thanks.

Hey folks! I'm working on migrating the Delta Lake connector from Flink 1.16 to 2.0 and hit an interesting blocker that I'd love some guidance on. Quick context: I've made substantial progress (15 commits, 105+ tests migrated, everything compiles), but there's one missing piece that's blocking integration tests. The Problem: My custom

DeltaSinkInternal

implements

Sink<IN>

and has these methods:

public Optional<Committer<T>> createCommitter() throws IOException
public Optional<SimpleVersionedSerializer<T>> getCommittableSerializer()

The framework discovers and invokes

createWriter()

perfectly, but

createCommitter()

is never called. When I tried adding

@Override

, I got a compilation error saying these methods aren't part of the

Sink<IN>

interface. The Question: How does a Sink expose its Committer in Flink 2.0? Is there a marker interface I'm missing? How does

FileSink

do it? I suspect there's a specific pattern or interface (maybe something that replaced

TwoPhaseCommittingSink

from 1.x) that I need to implement, but I can't find documentation about it. Related: • Delta Lake issue: https://github.com/delta-io/delta/issues/5228 • My implementation branch: https://github.com/kdsnunes/delta/tree/flink/add-flink-2.0-support • (I'm also trying to get access to Flink's Jira to create an issue there, but it's pending approval) Any pointers would be super appreciated! Has anyone migrated a connector with Committer support to 2.0? Thanks in advance! 🙏

👋 There is a bug with GCS precondition check, which makes job fail under heavy load with GCS backend: FLINK-38225 As suggested by @Xi Zhang I have made a precondition check in [FLINK-38225] Sets proper precondition to ensure that GCS operations are retryable

Extremely sorry for off the topic message. Hi All, I’ve recently been impacted by an organizational layoff(45%) at Chegg, and am now exploring new opportunities as a Backend Engineer / Distributed Systems Engineer. Over the past 7 years, I’ve worked on high-throughput, large-scale backend systems, architected subscription and payment platforms serving millions of users, and contributed to open-source projects at Debezium optimizing Change Data Capture (CDC) performance in PostgreSQL and MariaDB. Apart from writing APIs and Queries that solves complex business problem, I have good knowledge of Flink as well. I built subscription churn predictions using Kafka and Flink while I was learning Flink. I’m immediately available and fully committed to bringing dedication in solving business problems and sincerity to my work and word. Please refer me to a role in your company or anyother company that you know is hiring. Your small step will help me to get a job🙏 Thank you Pranav

Hi, dev The feature freeze of 2.2 has started now. That means that no new features or improvements should now be merged into the master branch unless you ask the release managers first, which has already been done for PRs, or pending on CI to pass. Bug fixes and documentation PRs can still be merged. - Cutting release branch Currently we have no blocker issues(beside tickets that are used for release-testing). We are planning to cut the release branch on next Wednesday (November 5) if no new test instabilities, and we'll make another announcement in the dev mailing list then. - Cross-team testing The release testing will start right after we cut the release branch, which is expected to come in the next week. As a prerequisite of it, please complete the documentation of your new feature and mark the related JIRA issue in the 2.2 release wiki page [1] before we start testing, which would be quite helpful for other developers to validate your features. [1] https://cwiki.apache.org/confluence/display/FLINK/2.2+Release Best, Hang