I'm using Flink 1.6 with Kinesis Data Analytics an...
# troubleshooting
d
I'm using Flink 1.6 with Kinesis Data Analytics and I'm seeing the following error message quite often; is anyone aware of how to resolve this?:
Copy code
{
  "locationInformation": "org.slf4j.helpers.MarkerIgnoringBase.warn(MarkerIgnoringBase.java:131)",
  "logger": "akka.remote.transport.netty.NettyTransport",
  "message": "Remote connection failed with javax.net.ssl.SSLException: Cannot kickstart, the connection is broken or closed",
  "threadName": "flink-akka.actor.default-dispatcher-567",
  "applicationARN": "arn:aws:kinesisanalytics:us-east-1:xxxxxx:application/xxxxx",
  "applicationVersionId": "55",
  "messageSchemaVersion": "1",
  "messageType": "WARN"
}
j
what is this app doing? it appears to be a networking issue but could be related to state or another error causing this.
d
It's consuming from a kinesis stream and transforming records and writing them into a database
When I enable debug logging, I see the following:
Copy code
{
    "applicationARN": "arn:aws:kinesisanalytics:us-east-1:xxxxx:application/xxxx",
    "applicationVersionId": "11",
    "locationInformation": "org.slf4j.helpers.MarkerIgnoringBase.debug(MarkerIgnoringBase.java:83)",
    "logger": "akka.remote.transport.netty.NettyTransport",
    "message": "Remote connection to [/10.161.198.253:32858] was disconnected because of [id: 0x66dbbabe, /10.161.198.253:32858 :> /10.161.197.95:6123] DISCONNECTED",
    "messageSchemaVersion": "1",
    "messageType": "DEBUG",
    "threadName": "flink-akka.actor.default-dispatcher-31"
}

{
    "applicationARN": "arn:aws:kinesisanalytics:us-east-1:xxxxx:application/xxxxx",
    "applicationVersionId": "11",
    "locationInformation": "org.slf4j.helpers.MarkerIgnoringBase.debug(MarkerIgnoringBase.java:83)",
    "logger": "akka.remote.transport.ProtocolStateActor",
    "message": "Association between local [<ssl.tcp://flink@flink-jobmanager:6123>] and remote [<ssl.tcp://flink@10.161.198.253:32858>] was disassociated because the ProtocolStateActor failed: Unknown",
    "messageSchemaVersion": "1",
    "messageType": "DEBUG",
    "threadName": "flink-akka.actor.default-dispatcher-31"
}
j
any exceptions in the flink dashboard?
also curious about your checkpointed state size / duration?
d
No exceptions in the flink dashboard
j
instead of writing to the database can you use a discardingSink (or print) and see if the error still occurs?
d
Oh that's a good idea
j
ooh you weren’t lying about using flink 1.6 🙂 That is quite an old version. KDA now supports up to Flink 1.15. Would you be interested in upgrading? you may see better errors there too
d
Yeah it's a tech debt item that hasn't garnered much attention yet on my team
How smooth is the upgrade path?
j
flink claims to be backwards compatible but there definitely are some API changes. It is a worthwhile upgrade and there may be some changes but i wouldn’t think it would be significant if you are simply using the datastream API
hi Daniel, following up, how is this project going?
d
I am revisiting this today, in the past when I have tried deploying our flink app on flink 1.13 I get the following error:
Copy code
2022-08-10T15:56:20.213-05:00
{
    "applicationARN": "arn:aws:kinesisanalytics:us-east-1:xxxx:application/daniel-test-flink-1-13",
    "applicationVersionId": 4,
    "message": "org.apache.flink.runtime.rest.handler.RestHandlerException: No jobs included in application.\n\tat org.apache.flink.runtime.webmonitor.handlers.JarRunOverrideHandler.lambda$handleRequest$4(JarRunOverrideHandler.java:253)\n\tat java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)\n\tat java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)\n\tat java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1705)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\n",
    "messageType": "ERROR",
    "messageSchemaVersion": "1",
    "errorCode": "CodeError.InvalidApplicationCode"
}
j
Is this Python code by any chance?
d
No, it's Java 8
j
oh wait i see this: No jobs included in application.
can you share your pom? were you specifying a main class?
d
We have gradle in this project and the project doesn't have a pom, but in my manifest.mf I have the following:
Copy code
Manifest-Version: 1.0
Built-By: xxxxxx
Build-Jdk: 1.8.0_171
Main-Class: com.xxxx.schedule.ScheduleUpdatesFlinkApplicat
 ion
it's not clear to me why the name of the main class is wrapped to the second line
j
Hmm could that be causing the issue?
d
I think it could be, I'll try pursuing this again