Hi All,
Is there any limitation on converting pyfl...
# troubleshootingj
Jagan Nalla
08/03/2023, 1:23 PMHi All,
Is there any limitation on converting pyflink table to pandas? https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/dev/python/table/conversion_of_pandas/
Jagan Nalla
08/03/2023, 1:43 PMI am consuming the events from kaka topic and doing the window function on the data to capture and aggregate it.
After that I can see my results by printing it, you can find it on below screenshot.
Jagan Nalla
08/03/2023, 1:46 PMHowever when I convert pyflink table to pandas dataframe and print the dataframe I don't see any results.
I wonder where it is going wrong and how to overcome this problem.
please see below screenshot of my code.
e
Elizaveta Batanina
08/03/2023, 1:59 PMAs far as I understand, pandas works only in batch mode and if you connect to kafka, you probably use streaming mode
j
Jagan Nalla
08/03/2023, 2:14 PMOkay Thank you for your clarifying.
Jagan Nalla
08/03/2023, 2:15 PMBut flink documentation they didn't specify anywhere like that.
Jagan Nalla
08/03/2023, 2:19 PMAs per this pyflink example, they took streaming mode. https://github.com/apache/flink/blob/release-1.17/flink-python/pyflink/examples/table/pandas/conversion_from_dataframe.py
Jagan Nalla
08/03/2023, 2:47 PMI referred same example and able to print and convert to pandas. However when I read the data from kafka topic and transforming the data by storing the results to pyflink table and print it works. But it doesn't convert to pandas dataframe.
Eugenio Gastelum
08/04/2023, 7:08 PM@Jagan Nalla Did that worked?
j
Jagan Nalla
08/04/2023, 7:08 PMNo @Eugenio Gastelum my bad I forgot to reply here.
2 Views