Hi I'm new to Airbyte, and have a few question. Do...
# ask-ai
j
Hi I'm new to Airbyte, and have a few question. Does it support CDC from PostgreSQL to ClickHouse database? Please provide and do I get start on this?
k
Airbyte does support Change Data Capture (CDC) from PostgreSQL, but based on the provided knowledge sources, there is no specific mention of support for ClickHouse as a destination for CDC. However, you can still set up CDC from PostgreSQL and then configure your destination as needed. ### Getting Started with CDC from PostgreSQL To get started with CDC from PostgreSQL using Airbyte, follow these steps: 1. Ensure Prerequisites: - Make sure you have a compatible logical decoding plugin and a replication slot in your PostgreSQL instance. - If you’re using a PostgreSQL instance in the cloud, such as Amazon RDS, refer to the [Debezium connector documentation](https://debezium.io/documentation/reference/1.9/connectors/postgresql.html) for specific requirements. 2. Configure PostgreSQL for CDC: - Provide additional
REPLICATION
permissions to the read-only user. - Enable logical replication on your PostgreSQL database. - Create a replication slot on your PostgreSQL database. - Create publication and replication identities for each PostgreSQL table. 3. Set Up Airbyte: - Follow the [quick start guide](https://docs.airbyte.com/integrations/sources/postgres#quick-start) to confirm that Airbyte can connect to your PostgreSQL database. - Enable CDC replication in the Airbyte UI. 4. Use Docker to Start a PostgreSQL Instance: - You can use Docker to start a PostgreSQL instance for testing purposes. The tutorial [here](https://airbyte.com/tutorials/postgresql-change-data-capture-elt) provides detailed steps on how to do this. 5. Watch the Tutorial: - For a visual guide, you can watch the YouTube tutorial on [Real-Time Change Data Capture with PostgreSQL](

https://www.youtube.com/watch?v=CQ7JKwKIIg8

). This video walks through the steps of setting up a PostgreSQL Docker container, configuring the database schema and user privileges, creating a table and populating it with data, creating a logical replication slot, and setting up a publication for subscription to events. ### Additional Resources - [Airbyte's CDC Documentation](https://docs.airbyte.com/understanding-airbyte/cdc) - [PostgreSQL Source Setup](https://docs.airbyte.com/integrations/sources/postgres#setup-using-cdc) - [Debezium Connector Documentation](https://debezium.io/documentation/reference/1.9/connectors/postgresql.html) If you need more specific instructions or run into any issues, feel free to refer to the detailed documentation or tutorials provided above.
j
Hi Has anyone done CDC replication from PostgreSQL (source) to Clickhouse(destination) running on Kubernetes using Airbyte?
How do I ensure that your ClickHouse instance can be accessed by Airbyte? I have both Airbtye and ClickHouse installed on the same Kubernetes , and have connectivity issue from Airbyte to Clickhouse using Airbyte UI. What tools that I can use to test connectivity to Clickhouse?