Hey all, love the product and is proving useful for dragging large sets of Google Ads and Bing data into big query. My main sticking point and something I think would be really useful would be the ability to set dynamic start dates on sources. For example I use the Google Ads connector to pull data for this year and last year, this is then used to compare yesterdays spend and this months spend vs last years (in data studio). I would use append, but this just seems to duplicate data so I am forced to use full refresh, 2 years data can take 30 mins to sync and I have to go back and change the start date every now and again to keep it as manageable as possible. So yeah an option to select like 2 years worth from today would be useful (this would be a mute point if the append worked by just adding on the next days data and not duplicating anything, but I cant get that to work). On that note, something similar would be the ability to set a specific time for the sync to run. So at say 2am every day, regardless of if a manual pull has been done.
Those are just my initial ramblings, hopefully they can be of some use.
But yeah other than that and the horror that is 0 auth, this is by far the best thing I have seen for this kind of data movement especially for someone like me who is a marketeer first so thanks!
Hey why not try increment dedup mode for your google ads sync ?
I struggled to get it to work, though just worked out its due to the product key.....will give it another shot now
The best way probably is to solve the incremental + dedup sync mode; This way you can keep the historical data to future analysis
02/18/2022, 7:18 AM
This is the way to go and it works well. Except for the Amazon Ads connector unfortunately. And as a feedback: we are lost in that regard, came up with solutions to mitigate a major flaw (tmp tables not being deleted after failed syncs), shared our solution with the community and asked for further support. It seems though, that attention levels from the Airbyte team have dropped recently regarding the deduplication malfunctioning of that connector. Something that we do not understand.
Hey jens if there is an issue created around this can you share it here