Hello, I have coded an airbyte custom destination ...
# help-connector-development
k
Hello, I have coded an airbyte custom destination using python. I have implemented an incremental sync dedup operation using this query,
Copy code
MERGE INTO {self.schema_name}.{table_name} AS target
USING {self.schema_name}.{table_name}_temp AS source
ON target.data_{primary_keys[0][0]}=source.data_{primary_keys[0][0]}
WHEN MATCHED THEN
        {query_placeholder_refined}
WHEN NOT MATCHED THEN
        INSERT *
I am taking github as source and my custom python destination, but some streams cause sync failure for instance this error,
pyspark.sql.utils.AnalysisException: Updates are in conflict for these columns: data_user_id