Hi all, Is there anybody using GCP PubSub connecto...
# troubleshooting
h
Hi all, Is there anybody using GCP PubSub connector with Flink 1.17.1? I'm facing an issue when initializing a new pubsub source. More specifically It's unable to find the following class :
java.lang.NoClassDefFoundError: io/grpc/CallCredentials2
m
I'm wondering if that's related to https://issues.apache.org/jira/browse/FLINK-32031
h
Thanks I'm looking into it. By the way do you have a sample
build.gradle
when using this connector ? I want to confirm It matches with mine
m
I don't, I only have Maven.
h
Ok Maven would be fine as well 🙇‍♂️
p
Did you find out anything about this? I am not using the GCP pubsub connector, but attempting to use grpc embedded in a job and getting weird not class found exceptions too: such as
Copy code
java.lang.NoClassDefFoundError: io/grpc/netty/shaded/io/netty/util/concurrent/GlobalEventExecutor$2
h
Ok our problem is similar then. For our application, we're using protobuf (and so we have a dependency on grpc). When we use the following in our `build.gradle`:
Copy code
implementation group: 'com.google.cloud', name: 'libraries-bom', version: '8.1.0', ext: 'pom'
implementation group: 'org.apache.flink', name: 'flink-connector-gcp-pubsub', version: '3.0.1-1.17'
PubSub will work fine. However adding
Copy code
implementation('com.twitter:chill-protobuf:0.10.0') {
        exclude group: 'com.esotericsoftware.kryo', module: 'kryo'
    }
implementation group: 'io.grpc', name: 'grpc-netty-shaded', version: '1.58.0'
Will make the following error come out :
Copy code
java.lang.NoClassDefFoundError: io/grpc/CallCredentials2
Which version of grpc are you using ?
m
This kind of sounds like a classpath/dependency versions problem
h
Yes I think so
p
I solved it using the same grpc version that I saw in one of the pom.xml in flink repo (1.17), so no conflicts now: grpcVersion=
1.25.0
protobufVersion=
3.21.7
the
gradle.build.kts
here:
Copy code
runtimeOnly("io.grpc:grpc-netty-shaded:$grpcVersion")
    implementation("io.grpc:grpc-protobuf:$grpcVersion")
    implementation("io.grpc:grpc-stub:$grpcVersion")
    implementation("com.google.protobuf:protobuf-java:$protobufVersion")
h
Oh thanks ! I'll check if It's compatible with PubSub connector
p
I had similar exception as described here: https://github.com/pravega/pravega/issues/4401 maybe it helps.