This message was deleted.

that's not really a 1 to 1 migration

I didn't understand, actually what I want to do is to send the data from posgtreSQL to Apache Druid but I have some databases and I want to load them one by one

Hi Hircham, you should be able to load your postgesSQL tables into Druid in the same way as MySQL tables are loaded in this blog post You might even be able to do it using the new MSQ ingestion like:

INSERT INTO some_datasource
SELECT * FROM TABLE(
  EXTERN(
    '{
      "type":"sql",
      "database":{
        "type":"postgresql",
        "connectorConfig":{
          "connectURI":"jdbc:<postgresql://some-rds-host.us-west-1.rds.amazonaws.com:5432/druid>",
          "user": "admin",
          "password": "password"
        }
      },
      "sqls": [
        "SELECT * FROM some_table"
        ]
    }',
    '{"type":"json"}',
    '[
        {"name":"column1","type":"string"},
        {"name":"column2","type":"string"}
     ]'
  )
)

Thank you, But I have a lot of databases, each database has a lot of tables which means it's gonna take much time to migrate them all, I want an approach that is able to load all databases at once

Hi @Hicham Laqrafi, this will probably not be the case, there will be multiple uploads

Hi @Mathieu Andrei I hope you're doing well, Thanks for the answer, Now I see that I have to do multiple uploads so right now I'm reffering to upload one database from postgreSQL to Aapache Druid if I do this task the rest will be easy But I didn't find any way to do it Do you have any suggestions ?

You are going to want to extract your tables from Postgress in files, and then ingest them into Druid.

But in my case I have a lot of data each table has hundreds of columns and I have a lot of databases which means it's gonna take much effort to do it, is there any other approach, for example a script in python that can load the whole database to druid ?

not to my knowledge, what do you think @Kyle Hoondert

I don’t know of any tools that would help with this. My thoughts are: better get scripting…