Hi Team In a class called `ResultProcessor` I m using Table Apache Flink #troubleshooting

Hi Team, In a class called `ResultProcessor`, I’m...

Raghunadh Nittala

03/31/2023, 11:03 AM

Hi Team, In a class called

ResultProcessor

, I’m using Table API and executing few SQL queries on a Kafka stream using

executeSql()

method. In another class called

ResultExecutor

, I’m using DataStream to process another Kafka stream, using

env.execute()

method. Now, in the Main class, I’m calling

ResultExecutor

first and then

ResultProcessor

. While submitting the job, I’m getting an exception

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Cannot have more than one execute() or executeAsync() call in a single environment

. When I reverse the order, I don’t see the exception, but only one stream is getting executed. When I comment out the first one, the second one is working fine and I’m able to see the operators in the job overview (and viceversa). Is there any restriction on the environment to execute TableAPI/DataStream alone? I’m using Flink 1.16 version. 🙏

sap1ens

03/31/2023, 5:19 PM

https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/table/data_stream_api/#execution-behavior might be helpful here

Raghunadh Nittala

04/01/2023, 7:17 AM

Thanks @sap1ens for your reply, I’ll add

.createStatement

and

execute()

on top of the Table API and verify this.

Raghunadh Nittala

04/02/2023, 10:42 AM

Hi @sap1ens - I’ve added

.createStatement

and

execute()

on top of it. If I keep DataStream implementation in the beginning, I see only the DataStream operators in the UI, seems TableAPI implementation is not working. If I keep TableAPI first and then the DataStream implementation, I see the errors -

Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Cannot have more than one execute() or executeAsync() call in a single environment.

at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372) ~[flink-dist-1.16.1.jar:1.16.1]

at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist-1.16.1.jar:1.16.1]

at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98) ~[flink-dist-1.16.1.jar:1.16.1]

at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301) ~[flink-dist-1.16.1.jar:1.16.1]

... 13 more

Caused by: org.apache.flink.util.FlinkRuntimeException: Cannot have more than one execute() or executeAsync() call in a single environment.

Raghunadh Nittala

04/02/2023, 10:43 AM

I’m not getting what mistake I’m doing here.

Raghunadh Nittala

04/05/2023, 5:31 AM

@sap1ens - Do you think I need to create 2 environments? one for the Table API and one for the DataStream one

Raghunadh Nittala

04/06/2023, 10:57 AM

I converted both to Table API and able to see the operators from both pipelines now. Thanks for your time @sap1ens 🙏

Adrian Chang

06/14/2023, 3:21 PM

This solved my issue. https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/table/data_stream_api/#adding-table-api-pipelines-to-datastream-api

Copy code

// attach both pipelines to StreamExecutionEnvironment
// (the statement set will be cleared after calling this method)
statementSet.attachAsDataStream();

5 Views

Open in Slack

Previous Next