Why in a CDC connection when a record is deleted the deletion is not propagated into the destination? We can still see in the destination records that have been deleted in source

In a standard incremental replication, the system periodically executes queries on a source system for new or updated records, and then transmits the results of these queries to the destination. However, because a query cannot return deleted records, standard incremental replication does not correctly transmit deletions from the source to the destination. This is because the records that are replicated to the destination are selected by a query on the source system for records that have been inserted or modified since the last synchronization. Any records that have been deleted are not returned by such a query, and therefore deleted records are not propagated to the destination [source](https://airbyte.com/tutorials/incremental-data-synchronization). On the other hand, Change Data Capture (CDC) incremental replication reads a log of the changes that have been made to the source database and transmits these changes to the destination. Because changes are read from a transaction log when using CDC, it is not necessary for the source data to have a suitable cursor field. Additionally, intermediate changes and deletions on the source are correctly transmitted to the destination because they are logged just like any other modification to the source data [source](https://airbyte.com/tutorials/incremental-change-data-capture-cdc-replication). So, if you are using CDC replication and deletions are not being propagated to the destination, it might be due to some misconfiguration or issue with the CDC setup. You should check your CDC settings and ensure that your source database is correctly configured for CDC. For example, in MySQL, you must enable binary logging for MySQL replication [source](https://docs.airbyte.com/integrations/sources/mysql).

@kapa.ai which would be the correct config in airbyte?

The correct configuration in Airbyte depends on the specific source and destination you are working with. However, here are some general steps you can follow: 1. Create a Connection: After creating a connection, you can modify how your data syncs to the destination by changing the configuration settings and the stream settings. You can do this from the Airbyte Cloud dashboard by clicking on Connections and then the connection you want to change ([source](https://docs.airbyte.com/cloud/managing-airbyte-cloud/configuring-connections)). 2. Configure Connection Settings: You can manage various aspects of the sync, such as how often data syncs and where data is written. These settings apply to all streams in the connection. Some of the settings you can configure include Replication frequency, Destination namespace, Destination stream prefix, and how Airbyte handles syncs when it detects schema changes in the source ([source](https://docs.airbyte.com/cloud/managing-airbyte-cloud/configuring-connections)). 3. Configure the Airbyte MongoDB Source: If you are using MongoDB as a source, you can configure it using the database credentials and connection string ([source](https://docs.airbyte.com/integrations/sources/mongodb-v2)). 4. Set up a Connection: After deploying Airbyte locally and setting up your first source and destination, you can create your first connection. You can configure the connection, set the Replication frequency, and choose which streams you want to replicate ([source](https://docs.airbyte.com/quickstart/set-up-a-connection)). 5. Add a Source: You can set up a new source in Airbyte by clicking the Sources tab in the Airbyte UI. Each connector in Airbyte will have its own set of authentication methods and configurable parameters ([source](https://docs.airbyte.com/quickstart/add-a-source)). Remember, the specific configuration settings will depend on the source and destination you are using. Always refer to the specific documentation for your source and destination for the most accurate information.

@kapa.ai I'm using MySQL to BigQuery and I need to know the specific settings in airbyte sync so that the CDC ingestion deletes records when they are deleted in the source

A message from kapa.ai

Isn't it incremental append + deduped? @kapa.ai

A message from kapa.ai