Kazim Raza
04/20/2023, 3:50 PMMERGE INTO {self.schema_name}.{table_name} AS target
USING {self.schema_name}.{table_name}_temp AS source
ON target.data_{primary_keys[0][0]}=source.data_{primary_keys[0][0]}
WHEN MATCHED THEN
{query_placeholder_refined}
WHEN NOT MATCHED THEN
INSERT *
I am taking github as source and my custom python destination, but some streams cause sync failure for instance this error,
pyspark.sql.utils.AnalysisException: Updates are in conflict for these columns: data_user_id