https://flink.apache.org/ logo
Join Slack
Powered by
I’m having trouble with running my flink test job ...
# troubleshooting
c
I’m having trouble with running my flink test job in my local IDEA (Intellij). When use rocksdb as state.backend, this error is occur,
Copy code
java.lang.NoSuchFieldError: NONE
	at org.apache.flink.contrib.streaming.state.RocksDBConfigurableOptions.<clinit>(RocksDBConfigurableOptions.java:138)
	at org.apache.flink.contrib.streaming.state.PredefinedOptions$1.<init>(PredefinedOptions.java:84)
	at org.apache.flink.contrib.streaming.state.PredefinedOptions.<clinit>(PredefinedOptions.java:79)
	at org.apache.flink.contrib.streaming.state.RocksDBOptions.<clinit>(RocksDBOptions.java:80)
	at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.<init>(EmbeddedRocksDBStateBackend.java:223)
    ...
Because my project’s flink-statebackend-rocksdb:1.16.0 points the other packages. (Hudi and Spark also have rocksdb dependencies..) I notice Hudi & Spark rocksdb dependency is 
org.rocksdb:rocksdbjni:6.20.3
, but flink-statebackend-rocksdb:1.16.0 has 
com.ververica:frocksdbjni:6.20.3-ververica-1.0
under that. And there is no enum field “NONE” in 
org.rocksdb:rocksdbjni:6.20.3
! How can i fix it?
z
hey @chankyeong won! The error message you're seeing is caused by a conflict between different versions of the RocksDB library that are being used in your project. It seems that your project has a dependency on both the 
org.rocksdb:rocksdbjni
library (which is used by Hudi and Spark) and the 
com.ververica:frocksdbjni
library (which is used by the Flink RocksDB state backend).
As you also higlighted, it appears that the version of the 
org.rocksdb:rocksdbjni
library being used in your project does not have an enum field called "NONE", which is causing the error
To fix the issue, you need to make sure that only one version of the RocksDB library is being used in your project. One way to do this is by excluding the 
org.rocksdb:rocksdbjni
library from the dependencies of Hudi and Spark, and instead using the 
com.ververica:frocksdbjni
library that is provided by the Flink RocksDB state backend. You can do this by excluding the library in the Hudi and Spark dependency in your maven or gradle.
Another alternative is to use the same version of rocksdb in your project that is being used in flink-statebackend-rocksdb:1.16.0.
You could also try to use the RocksDB version that is compatible with both Flink and Hudi/Spark, but this may not be feasible if the version of RocksDB used in Flink is not compatible with the version used in Hudi and Spark
c
Thank you reply! It would have been nice if I saw your comment before I fixed my issue. I'm using Gradle for package manager. I had needed runtime flink dependencies to run my flink test app on my local IDE, so I changed spark-sql & hudi-common dependency from "implementation" to "compileOnly"! And I moved the flink dependencies to top of the list! My solution is not perfectly because when spark & hudi is required in runtimeClasspath, but it's nice for my use case now.
3 Views
Open in Slack
PreviousNext
Powered by