Hi all. Having an error about jdbc driver not foun...
# troubleshooting
t
Hi all. Having an error about jdbc driver not found. The driver is included in the user application jar. Running Flink in session mode and the Flink cluster have no jdbc driver. Wondering why the driver cannot be found. Thanks in advance.🙏 Error log, code & pom in thread ->
Exception in jobmanager
Copy code
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not execute application.
	at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(JarRunHandler.java:113)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1705)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.concurrent.CompletionException: org.apache.flink.util.FlinkRuntimeException: Could not execute application.
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1702)
	... 1 more
Caused by: org.apache.flink.util.FlinkRuntimeException: Could not execute application.
	at org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:88)
	at org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70)
	at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:107)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
	... 1 more
Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Failed in getting MySQL driver version by jdbc:<mysql://xxxx.com:3306/xxxx>.
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98)
	at org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84)
	... 4 more
Caused by: org.apache.flink.table.catalog.exceptions.CatalogException: Failed in getting MySQL driver version by jdbc:<mysql://xxxx.com:3306/xxxx>.
	at org.apache.flink.connector.jdbc.catalog.MySqlCatalog.getDriverVersion(MySqlCatalog.java:144)
	at org.apache.flink.connector.jdbc.catalog.MySqlCatalog.(MySqlCatalog.java:73)
	at org.apache.flink.connector.jdbc.catalog.JdbcCatalogUtils.createCatalog(JdbcCatalogUtils.java:54)
	at org.apache.flink.connector.jdbc.catalog.JdbcCatalog.(JdbcCatalog.java:79)
	at org.example.StreamingJob.main(StreamingJob.java:60)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
	... 7 more
Caused by: java.sql.SQLException: No suitable driver found for jdbc:<mysql://xxxx.com:3306/xxxx>
	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:702)
	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
	at org.apache.flink.connector.jdbc.catalog.MySqlCatalog.getDriverVersion(MySqlCatalog.java:137)
Snip of code & pom
Copy code
public class StreamingJob {

	private static final Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());

	public static void main(String[] args) throws Exception {
		// set up the streaming execution environment
		final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

		StreamTableEnvironmentImpl tableEnv = (StreamTableEnvironmentImpl) StreamTableEnvironment
				.create(env);

		String jdbcCatalogName = "jdbc_catalog";
		String defaultDatabase = "xxxx";
		String username = "xxxx";
		String password = "xxxx";
		String baseUrl = "jdbc:<mysql://xxxx.com:3306/>";

		JdbcCatalog jdbcCatalog = new JdbcCatalog(Thread.currentThread().getContextClassLoader(),
				jdbcCatalogName, defaultDatabase, username, password, baseUrl);

		tableEnv.registerCatalog(jdbcCatalogName, jdbcCatalog);

		<http://LOG.info|LOG.info>("All Catalog: {}", tableEnv.listCatalogs());

		// execute program
		env.execute("Flink Streaming Java API Skeleton");
	}
}
Copy code
<dependencies>
		<dependency>
			<groupId>org.apache.flink</groupId>
			<artifactId>flink-table-api-java-bridge</artifactId>
			<version>${flink.version}</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>org.apache.flink</groupId>
			<artifactId>flink-connector-jdbc</artifactId>
			<version>${flink.version}</version>
			<scope>compile</scope>
		</dependency>
		<dependency>
			<groupId>com.mysql</groupId>
			<artifactId>mysql-connector-j</artifactId>
			<version>8.0.32</version>
			<exclusions>
				<exclusion>
					<groupId>com.google.protobuf</groupId>
					<artifactId>protobuf-java</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
Seem sth related to classloader & SPI Default parent-first for org.apache.flink (
flink-connector-jdbc
)
mysql-connector-j
is the available SPI service to it Found 2 circumstances which would works: • Both
flink-connector-jdbc
&
mysql-connector-j
package in flink/lib but not included in user app jar • Both
flink-connector-jdbc
&
mysql-connector-j
package in user app jar but not included in flink/lib
Shading with maven plugin might be a possible alternative. But not find the correct config yet