hey, I am trying to make a ETL pipeline from postg...
# ask-community-for-troubleshooting
p
hey, I am trying to make a ETL pipeline from postgres to postgres, I want to use log based replication from the slave of my master db, currently postgres slave DBs do not support replication slots, is thereany wat I can do this?
1
j
It sounds like you might want cascading replication: https://www.postgresql.org/docs/9.3/warm-standby.html#CASCADING-REPLICATION
👍 1
octavia loves 1
p
and does airbyte support cascading replication?
u
I never tried before @Prateek Gupta, but probably you can connect to the slave-node using the direct IP. Could you validate it? and share what you got?
j
It would likely work with full refreshes, but if the incremental mode is using something like debezium, I don't think it would work. As you mentioned, I don't think replication slots are currently supported by Postgres on standby servers. If you are looking at doing incremental, is there a specific reason that you can't do it on the primary? Sorry, I think I misunderstood what you were trying to do earlier with this.
p
well, the master db is already low on memory and the WAL logs take up memory till the moment they are read and merged in the target db. hence it would be very helpful if we could do it from a slave
j
Low on memory or disk? The WAL logs should be stored on disk and will continue to take up disk space until they are read.