👋 hi made this ticket, just wanted to verify there actually isn't a way to do this already. i'm happy to make the pr for it as well if we think its worth adding.

Hey folks, I've created this ticket and a looking for feedback on it, it's related to the DynamoDB Sink, I'm wondering if this is the correct place to ask? or is there a relevant mailing list?

What is the current status / position on stateful functions. It seems like there hasn't been much forward movement with that project or ecosystem in about 4 years? is this still considered a 1st class citizen within flink?

Hey, I'm trying to create a

KinesisStreamsSource

and then just print out the results from the stream to console (working locally with vscode). Code with the deprecated

FlinkKinesisConsumer

works fine, but when running with

KinesisStreamsSource

I am getting an error saying:

Unable to load credentials from any of the providers in the chain ...

I have added the new dependency for SDKv2

<dependency>
  <groupId>software.amazon.awssdk</groupId>
  <artifactId>sso</artifactId>
  <version>2.30.33</version>
</dependency>

Additionally I've run

aws sso login

and verified that the connection works. Any ideas what could be the problem?

Hi, we had issues with bad performance of loading lot of files from AWS S3. The problem we found is that

NonSplittingRecursiveEnumerator

does file operations in sequence, which in case of remote storages mainly just sit idle on IO operations. This could be improved (at least 10x in execution speed) by using ForkJoinPool to parallelized the work and would like to contribute it to Flink code base. https://issues.apache.org/jira/browse/FLINK-35704 https://github.com/apache/flink/pull/24986

Hi, is anyone of you if there is some major issue in handling savepoints/checkpoints in 2.1-SNAPSHOT? I have build the dist from

main

branch and run some simple job with stateful function. The problem is both checkpoints and savepoints directories only containing

_metadata

file. There are no blobs file even thou

_metadata

refers to them.

👋 hey, i made a pr for adding labels to services via the flinkconfig. issue is linked in the pr

Hello there! I remember seeing a discussion about cleaning the current state of open PRs on Flink repo. Does anyone have a link for it? Is it something we're still looking forward to do?

Hi Team, any tentative date for Flink 2 production ready version released, thanks.

Is there a way to trigger new HA files be written? (to avoid life_cycle configurations in S3 that age off old files)

Hi Team, wanted to take your suggestion on Flink 1.20.1 with Jdk 17, is it production ready version available or still in experimental. Is any one using it now. Thanks.

Is it planned to support efficient serialization for java 17 records?

The release notes for 2.0 linked to from the blog at

<https://nightlies.apache.org/flink/flink-docs-release-2.0/release-notes/flink-2.0/>

are 404 - is this because they are still being published? /cc @weijie guo

Hi, would like to raise to your attention that with 1.20.1 there was a major bug introduced by https://issues.apache.org/jira/browse/FLINK-35886 which prevents from using custom metrics in watermark generator due to its leading to StackOverflowError. This is due a change in code from:

watermarkGenerator =
                emitProgressiveWatermarks
                        ? watermarkStrategy.createWatermarkGenerator(this::getMetricGroup)
                        : new NoWatermarksGenerator<>();

to

watermarkGenerator =
                emitProgressiveWatermarks
                        ? watermarkStrategy.createWatermarkGenerator(
                                new WatermarkGeneratorSupplier.Context() {
                                    @Override
                                    public MetricGroup getMetricGroup() {
                                        return this.getMetricGroup();
                                    }

                                    @Override
                                    public RelativeClock getInputActivityClock() {
                                        return inputActivityClock;
                                    }
                                })
                        : new NoWatermarksGenerator<>();

Where

this.getMetricGroup

refereed to parent class instance before and now refers to anonymous class instance which leads to infinite recursion. https://issues.apache.org/jira/browse/FLINK-37545

Hello Everyone 👋 , Have anyone used two different Kafka Stream as a soruce in a single Flink application? If so could someone please share any cons that you found? In my use case, I have I want to process events from two different region Kafka cluster, the application itself is critical in terms of latency, so I found three solutions, 1. Use Flink's union stream to use two different Kafka source and process after union-ing it 2. Replicate the events from one region (say X) to other region (say Y) and use Y as source in Flink application 3. Deploy two Flink application so one Flink application can process one region data and the other Flink application can process the other region data - This is hard to maintain for sure For #2 and #3, I can work out pros/cons, but I am curious to know are there any cons/constraints/limitations that I need to be aware of if I choose #1. The doc didn't mention anything explicitly as a drawback, so wondering can some one can tell their experience with #1. Obviously I want to process the events without any latency, time to time both the source might ingest events at various rates.

Hai, When does flink-operator 1.10 submits the flink sessionjob resource? I'm using flink version 1.20 and all batch session job resources are not cleaned up and keep on resubmitting to the Flink Deployments and running multiple times.

Kafka Updates in Flink Connector Kafka Thanks for merging my Kafka 3.9 PR @Arvid Heise. I was wondering about the release versions. The main branch is currently on version

4.0-SNAPSHOT

, but I think if we are moving to the next major release for the Kafka connector it makes sense to also move to the Kafka 4.0 client libraries as they drop support for Kafka <2.1 (a breaking change). I have a PR open for that update. I which case do we want to do a Flink Connector Kafka version 3.5 release which maintains the support for Kafka 2.0 and lower and also supports flink 2.0.0?

Hi everyone. I am new to Flink and open source contributions. I came across a minor issue in the Flink documentation for which I want to open a Jira ticket and find a committer who can review the possible fixes and apply the patch. Is it the right way to go about this? Also, it has been a few days since I requested for the Jira account on the ASF self-serve portal. Who can I reach out to review my Jira account request? Thank you!

Hi, does the flink-kubernetes-operator guarantee support for a minimum number of flink versions? When does it deem a flink version not supported anymore?

Hello, I recently created this issue regarding thread safety issues during jobs submission via REST API (usually happens during job submission by Flink K8S Operator). Would appreciate feedback

Is there a timeline for Apache Pulsar Connector for Flink 1.20 and 2.0 ?

Hello everyone, I've noticed that there is no support for datastream window join in pyflink. I've opened a discussion on the dev mailing list but sadly there has been no response yet from the community. More generally, I am very keen to become more involved in the flink community and more specifically pyflink by doing some tickets and contributing to the project. Let me know if and how I can help Thank you very much

Hi, I'm waiting for Flink Kubernetes Operator v1.12. Does anyone know when it will be available for use?

Hi, I am looking into building Volcano support for Flink natively vs the K8s operator I see this flip got approved and is still open https://cwiki.apache.org/confluence/display/FLINK/FLIP-250%3A+Support+Customized+Kubernetes+Schedulers+Proposal Any idea on the status of this work?

Hi all, I've raised a PR to address an issue in KinesisSource which causes high CPU utilisation and throttling when reading from idle shards. Would appreciate review from committer. Thank you! https://github.com/apache/flink-connector-aws/pull/195

👋 hey all was hoping to get a review on this pr. and is it possible to backport it to 1.20? https://github.com/apache/flink/pull/26301

Hi all, I am interested in the current setup and infrastructure used to build and test PRs, nightly and release candidates as well as performance tests. Specifically, what is the best resource for me to start looking into this? I am looking into possibility of contributing additional coverage and as well as infrastructures to run these. Thank you!

Hi, any tentative date to release flink-connector-kafka 4.0.0 lib which supports flink 2.0, thanks

Hello I am new here and having committed to other Apache projects in the past, I plan to contribute with good first issues (and probably with the flink-iceberg interaction given my experience with the latter)

What is the process of adding test cases for a class in Flink? Do I need to create a JIRA ticket for it??