Hey. Evaluating Airbyte after Fivetran quoted us 1...
# ask-community-for-troubleshootinge
Espen Espelund
03/02/2022, 4:35 PMHey. Evaluating Airbyte after Fivetran quoted us 10-15k usd a month that seems like a giant waste of money to extract and load some data automatically. Can someone help me out with these questions?
• Does Airbyte support < 5 min syncing?
• How stable are the Facebook, AdWords and other major ad connectors? Are they production ready at scale, or can we expect a lot of troubleshooting or possible downtime over multiple days if things change?
• How stable are the MySQL CDC?
• What is the performance like compared to Fivetran?
• Biggest difference from Meltano and Rudderstack that seems like the other open-source alternatives?
• Which of the open-source platforms are most pleasant to integrate a custom rest api source on?
• Any real-time / streaming data functionality?
c
Christian Persson
03/02/2022, 4:44 PMInteresting. What data volumes are we talking about? Or is it anything specific that stands out in your setup/quote?
e
Espen Espelund
03/02/2022, 4:46 PMA few million e-commerce orders. Thousands per day. Significant ad spend on multiple stores
Espen Espelund
03/02/2022, 4:46 PMProbably 100m+ active rows. The mysql CDC is a big part tho
Espen Espelund
03/02/2022, 4:47 PMWe just want our data in Snowflake. As frequently as possible. No particular needs apart from that 😛
m
Marcos Marx (Airbyte)
03/02/2022, 4:47 PM• Does Airbyte support < 5 min syncing?
You can configure your connection to manual scheduled and trigger using the API with less than 5 minutes. We’re working to implement a cron-syntax allowing you choose the frequency you want.
Marcos Marx (Airbyte)
03/02/2022, 4:50 PM• How stable are the Facebook, AdWords and other major ad connectors? Are they production ready at scale, or can we expect a lot of troubleshooting or possible downtime over multiple days if things change?
Facebook, AdWords, Google Analytics and Hubspot, I consider them stable because most PR’s I reviewed are adding new features (end_date, new streams etc). And because a lot of users use them when a problem impact most of them the team look into the problem quicker
Marcos Marx (Airbyte)
03/02/2022, 4:51 PM• How stable are the MySQL CDC?
I saw a few people having problems with it, depends on how much data you have right now and if your data works fine with Debezium, some issues where because conflict about MySQL data types and Debezium. If possible to give a test and your feedback would be really great for us
c
Christian Persson
03/02/2022, 4:52 PMAh okej! That pricing definitely sounds quite aggressive in my opinion.. (~order of magnitude higher than I would have naively guessed), but I am not familiar with their guarantees and agreements on the other hand, but it should definitely be possible to set it up with a simler and cheaper solution if you don’t have any extreme preferences.
Airbyte is offering their cloud product, but I don’t think it’s available in your region yet, but it could be interesting to compare the quotes at least if possible. But yeah, that is only relevant if all connectors are ready.
m
Marcos Marx (Airbyte)
03/02/2022, 4:53 PM• What is the performance like compared to Fivetran?
• Biggest difference from Meltano and Rudderstack that seems like the other open-source alternatives?
We don’t have any benchmark at the moment, maybe something you can do with the connector you’re charged in Fivetran.
And for comparison you can read our pages: https://airbyte.com/etl-tools-comparison
Marcos Marx (Airbyte)
03/02/2022, 4:54 PM• Which of the open-source platforms are most pleasant to integrate a custom rest api source on?
Are you trying to create a custom connector? If yes, Airbyte has the Airbyte CDK where you can easily integrate a Rest API with Python. There are communtiy versions using C# and TypeScript too.
Marcos Marx (Airbyte)
03/02/2022, 4:55 PM• Any real-time / streaming data functionality?
Not yet, Airbyte has Kafka and other events connectors but they work in a batch mode.
e
Espen Espelund
03/03/2022, 6:42 AMThank you Marcos. This helps. We'll keep investigating Airbyte. Sure seems like the development is going fast here 💪
d
Donk
10/19/2022, 11:12 AMHi Espen. Did you go with Airbyte? If so, how is the performance. Any issues with reloading a few million records?
l
Louis Monier
05/29/2023, 5:02 PMHi @Espen Espelund,
I'd have the same question here ! How is the performance ? Are you having trouble ingesting your data ?
4 Views