gunu
02/07/2022, 7:20 AM2 connections:
- MySQL --> S3 (Incremental Append)
- S3 --> Snowflake (Incremental Dedupe)
Once the data is written to S3, it now contains the additional metadata
{
"_airbyte_ab_id": "b00c41e6-8a2f-4ed7-a10f-123",
"_airbyte_emitted_at": 1644216617782,
"_ab_cdc_log_pos": 123,
"_ab_cdc_log_file": "mysql-bin-changelog.123",
"_ab_cdc_updated_at": "2022-02-07T06:04:14Z"
}
when configuring the S3 --> Snowflake connection: the cursor field is source-defined but for the primary key, can I now use one of the metadata columns e.g. _airbyte_ab_id
or do i still need to use the primary keys that were defined in the original MySQL table?Marcos Marx (Airbyte)
02/09/2022, 10:36 PM