Hi Flink community! I have a couple of questions r...
# troubleshooting
c
Hi Flink community! I have a couple of questions regarding pyarrow/beam use in flink: • I saw this PR which allows us to use beam > 2.54 (which then allows a newer version of pyarrow), but as far as I can tell this is not a part of the 1.20.1 release of flink. I was wondering if there will potentially be a 1.x.x release which might include this change so that we can avoid using the Flink 2.0 pre-release and benefit from these changes? This would be very helpful to us and allow us to avoid doing a Flink major version upgrade as soon as the 2.0 release happens. If this won't be happening, may I ask when the Flink 2.0 release is planned? • My understanding is that while we are not explicitly using apache beam, pyflink internals makes use of the beam framework, which is why there's beam listed in the python requirements. Is that correct? • How risky would it be to use flink 1.20.1 and force the beam dependency to 2.54 so we can use pyarrow 14.0.2, which satisfies requirement of another python library we are using? I understand this is not ideal, but the only alternatives I see are using a pre release 2.0 or forking 1.20.1 to pull in that change we need. Thanks in advance for any help/insight here!