Hi.
Quick question: it seems catalogs entries are ...
# randomc
Christophe Bornet
03/23/2023, 1:55 PMHi.
Quick question: it seems catalogs entries are flushed when the session is closed (the entries as shown in SHOW CATALOGS, not the catalog tables which can be stored in DB/HDFS/etc..). Is that always the case ? Or can the catalog configs be stored somewhere and shared between sessions ?
m
Martijn Visser
03/23/2023, 4:53 PMYou can store them. Out of the box, there is integration with the Hive Metastore. You can also build your own. See https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/catalogs/
c
Christophe Bornet
03/23/2023, 4:54 PMIIUC this stores the content of the catalog but not the info that is put in the
CREATE CATALOGChristophe Bornet
03/23/2023, 4:57 PMIf in session 1, I do:
Copy code
CREATE CATALOG foo {...};
SHOW catalogs;
+-----------------+
| catalog name |
+-----------------+
| default_catalog |
| foo |
+-----------------+
2 rows in set Copy code
SHOW CATALOGS;
+-----------------+
| catalog name |
+-----------------+
| default_catalog |
+-----------------+
1 row in setm
Martijn Visser
03/23/2023, 4:58 PMDo you mean something like how can I load multiple catalogs during the start of your SQL client session?
c
Christophe Bornet
03/23/2023, 4:58 PMYes. And R/W them to a storage
m
Martijn Visser
03/23/2023, 5:01 PMI think right now you can either initialize your session with an initialization SQL statement, like described at https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/#initialize-session-using-sql-files and I think with the SQL Gateway you can also configure your session (not sure 100% on the latter, but basing this off https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql-gateway/rest/#sessions-session_handle-configure-session)
Martijn Visser
03/23/2023, 5:01 PMBut I don't think the SQL Gateway yet allows you to store this in a database or something
Martijn Visser
03/23/2023, 5:02 PMI think you're looking for https://issues.apache.org/jira/browse/FLINK-31259
Martijn Visser
03/23/2023, 5:02 PMc
Christophe Bornet
03/23/2023, 5:03 PMOK. So this is something that has to be configured. Actually that's a good thing for my use case π. I want session2 to have absolutely no possibility of using catalog foo !
Christophe Bornet
03/23/2023, 5:03 PMI think you're looking for https://issues.apache.org/jira/browse/FLINK-31259Thanks. That clarifies things a lot.
Christophe Bornet
03/23/2023, 5:05 PMCurrently eachThis is perfect.in gateway has its ownSessionCatalogManager