George Leonard
07/10/2024, 1:41 PMhttps://www.decodable.co/blog/catalogs-in-flink-sql-hands-on
I've tried copying the hive-site.xml and jar's into the sql-client and into the jobmanager,
when I then execute :
CREATE CATALOG c_hive WITH (
'type' = 'hive',
'hive-conf-dir' = './conf/');
with the updated/fully qualified hive-conf-dir specified I get either of the below 2 error's.
this is all in docker containers spun up via a docker-compose file.
i copied my jars into a subdir './hive/' where I found the native supplied jar's
Flink SQL> CREATE CATALOG c_hive WITH (
> 'type' = 'hive',
> 'hive-conf-dir' = '/opt/flink/conf/');
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.table.catalog.exceptions.CatalogException: Failed to create Hive Metastore client
or
Flink SQL> CREATE CATALOG c_hive WITH (
> 'type' = 'hive',
> 'hive-conf-dir' = '/opt/sql-client/conf/');
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'hive' that implements 'org.apache.flink.table.factories.CatalogFactory' in the classpath.
Available factory identifiers are:
generic_in_memory
jdbcrmoff
07/10/2024, 1:44 PMrmoff
07/10/2024, 1:44 PMrmoff
07/10/2024, 1:44 PMGeorge Leonard
07/10/2024, 1:47 PMGeorge Leonard
07/10/2024, 1:47 PMGeorge Leonard
07/10/2024, 1:56 PMGeorge Leonard
07/10/2024, 2:16 PMrmoff
07/10/2024, 2:31 PMGeorge Leonard
07/10/2024, 2:51 PMGeorge Leonard
07/10/2024, 3:02 PMfailed to solve: hms-standalone: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
I'm def logged into dockerGeorge Leonard
07/10/2024, 3:34 PMGeorge Leonard
07/10/2024, 3:34 PMGeorge Leonard
07/10/2024, 3:47 PM=> [flink 5/12] RUN sed -i "s/jobmanager.rpc.address: localhost/jobmanager.rpc.address: flink-jobmanager/g" ./conf/flink-conf.yaml 0.1s
=> ERROR [flink 6/12] RUN apt-get update && apt-get install -y vim lnav 3.5s
------
> [flink 6/12] RUN apt-get update && apt-get install -y vim lnav:
0.819 Get:1 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
0.819 Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
2.115 Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]
2.130 Err:2 http://security.ubuntu.com/ubuntu jammy-security InRelease
2.130 At least one invalid signature was encountered.
2.430 Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]
2.553 Err:1 http://archive.ubuntu.com/ubuntu jammy InRelease
2.553 At least one invalid signature was encountered.
2.968 Err:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
2.968 At least one invalid signature was encountered.
3.382 Err:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
3.382 At least one invalid signature was encountered.
3.422 Reading package lists...
3.482 W: GPG error: http://security.ubuntu.com/ubuntu jammy-security InRelease: At least one invalid signature was encountered.
3.482 E: The repository 'http://security.ubuntu.com/ubuntu jammy-security InRelease' is not signed.
3.482 W: GPG error: http://archive.ubuntu.com/ubuntu jammy InRelease: At least one invalid signature was encountered.
3.482 E: The repository 'http://archive.ubuntu.com/ubuntu jammy InRelease' is not signed.
3.482 W: GPG error: http://archive.ubuntu.com/ubuntu jammy-updates InRelease: At least one invalid signature was encountered.
3.482 E: The repository 'http://archive.ubuntu.com/ubuntu jammy-updates InRelease' is not signed.
3.482 W: GPG error: http://archive.ubuntu.com/ubuntu jammy-backports InRelease: At least one invalid signature was encountered.
3.482 E: The repository 'http://archive.ubuntu.com/ubuntu jammy-backports InRelease' is not signed.
------
failed to solve: process "/bin/sh -c apt-get update && apt-get install -y vim lnav" did not complete successfully: exit code: 100