it would also be helpful to have compound cursors for increm

Question

it would also be helpful to have compound cursors for incremental loading select multiple fields

Kyle Mok · Answer

~You can have multiple cursor fields~ My bad but why do you need this Is this a requirement for custom connector

Kyle Mok · Answer

< Harshith Airbyte > This is interesting how can we do it with mysql connector

Kyle Mok · Answer

< Raj> I think my bad I got confused between primary keys and cursors rolling on the floor laughing just today thing Sorry to get you excited

Kyle Mok · Answer

But now I am excited how will this feature help

Kyle Mok · Answer

< Harshith Airbyte > In many systems timestamps are not maintained properly sometimes we might need a custom logic like createdAt and updateAt as the cursor This is just an example There can be other cases aswell

Kyle Mok · Answer

Got it Then it would makes sense if we can create an issue and so that team can track this

Rachel RIZK · Answer

my specific use case is that we have a table with two date fields `created at` and `discarded at` setting the cursor on `created at` will capture records created but if it ends up becoming `discarded` we won t know On the other hand if we track `discarded at` we won t see when the record is `created` An option on our engineering side is to have an `updated at` field but that may or may not be prioritized so if we could `coalesce discarded at created at ` as a cursor that could help solve the problem

gunu · Answer

Got it Right now the option is to handle this in Custom DBT but it would also makes sense if you can create a issue around it add this info

Tyler Buth · Answer

Is there Custom DBT executed prior to loading

Dave Lindley · Answer

No custom dbt are executed after the loading process

Arvi · Answer

got it that means that custom dbt won t help here since the data won t be loaded

Arvi · Answer

You re correct