a curious question, previously I used HMS as my ca...
# random
g
a curious question, previously I used HMS as my catalog. so all my inbound sources, be that from kafka or mongo/CDC or postgres/CDC I always created them into my hive_catalog. I would then create the outbound destination table in say my iceberg created catalog or my paimon catalog. below... i create my outbound paimon catalog, with the catalog now stored in jdbc into postgres.
Copy code
CREATE CATALOG c_paimon WITH (
     'type'                          = 'paimon'
    ,'metastore'                     = 'jdbc'
    ,'catalog-key'                   = 'jdbc'
    -- PostgreSQL JDBC connection
    ,'uri'                           = 'jdbc:<postgresql://postgrescat:5432/flink_catalog?currentSchema=paimon>'
    ,'jdbc.user'                     = 'dbadmin'
    ,'jdbc.password'                 = 'dbpassword'
    ,'jdbc.driver'                   = 'org.postgresql.Driver'
    -- MinIO S3 configuration with SSL/TLS (if needed)
    ,'warehouse'                     = '<s3://warehouse/paimon>'  -- bucket / datastore
    ,'s3.endpoint'                   = '<http://minio:9000>'      -- MinIO endpoint
    ,'s3.access-key'                 = 'mnadmin'
    ,'s3.secret-key'                 = 'mnpassword'
    ,'s3.path-style-access'          = 'true'                   -- Required for MinIO
    -- Default table properties
    ,'table-default.file.format'     = 'parquet'
);
CREATE DATABASE IF NOT EXISTS c_paimon.outbound;
Curious now though, at the moment i'm creating my inbound objects using :
Copy code
CREATE CATALOG kafka_catalog WITH 
    ('type'='generic_in_memory'); 

CREATE DATABASE IF NOT EXISTS kafka_catalog.inbound;  

CREATE CATALOG postgres_catalog WITH 
    ('type'='generic_in_memory'); 

CREATE DATABASE IF NOT EXISTS postgres_catalog.inbound;
I then create my inbound table into as per above either into my kafka catalog or my postgres catalog. But I'd like to rather change this so that my tables and catalog data is actually persisted. I'd like to store my catalog into postgres / aka not use HMS. need some advise/guidance.