how to increase sync speed in postgres to postgres...
# ask-community-for-troubleshooting
y
how to increase sync speed in postgres to postgres replication
k
A message from kapa.ai
To increase the sync speed in Postgres to Postgres replication using Airbyte, you can consider the following recommendations: 1. Set
hot_standby_feedback
to
true
on the replica server. This prevents the primary server from deleting write-ahead logs when the replica is busy serving user queries. However, the downside is that the write-ahead log will increase in size. [Source](https://docs.airbyte.com/integrations/sources/postgres)
Copy code
sql # In postgresql.conf on the replica server hot_standby_feedback = true
2. Sync data when there is no update running in the primary server, or sync data from the primary server. [Source](https://docs.airbyte.com/integrations/sources/postgres) 3. Choose an appropriate sync mode for each table within the same database. Airbyte supports several sync modes for Postgres, like full, incremental, and logical replication (also called CDC). Selecting the right sync mode for each table can help optimize the sync speed. [Source](https://airbyte.com/tutorials/postgres-replication) Please note that increasing the
max_standby_archive_delay
and
max_standby_streaming_delay
is not recommended, as it is usually hard to tell how much time it will take to sync all the data, and this approach is not very practical. [Source](https://docs.airbyte.com/integrations/sources/postgres)
y
@kapa.ai no, i am talking in terms of batch size or increase memory in which container
k
A message from kapa.ai
To increase the memory for the Airbyte instance, you can set the
JOB_MAIN_CONTAINER_MEMORY_REQUEST
and
JOB_MAIN_CONTAINER_MEMORY_LIMIT
environment variables to custom values. This will help you allocate more memory to the source and destination containers, which can improve performance. For example, you can modify these environment variables in your
.env
file or in your Kubernetes deployment configuration. Here's an example of how a user set them in a discussion thread: [link](https://discuss.airbyte.io/t/increase-sync-worker-resources/1714/3). For more information on configuring Airbyte, you can refer to the [shared configuration documentation](https://docs.airbyte.com/operator-guides/configuring-airbyte/#shared).