Hey Airbyte! I'm looking to test Airbyte as a replacement / companion to an ETL framework we've built in-house. First question I have is how does Airbyte handle bad input data (e.g. csvs with malformed rows)?

Quick question -- I am planning to pull data from google analytics to Sql Server on AWS

Another Q... Am I limited to GUI interaction or is there a way to set up / run / interact with Airbyte programmatically? An example of doing this would be to use a dynamic url in an http-request source (e.g. updating query param each day/hour) rather than it having a static value. Or creating a new http-request source each run with the dynamically updated url and remove the old one. Is this sort of thing possible or is use limited to frontend capability?

Hey everyone. Today i was testing a mysql connector with a slightly larger number of tables (15) to see performances. Unfortunately Airbyte seems to be idling after completing the creation of temp tables in snowflake. Anyone has any experience with this?

<!channel> Hey everyone 👋! airbyte rocket We are currently looking for a SENIOR SOFTWARE ENGINEER. If you know anyone that would be interested, we would love to talk to then. airbyte rocket Here are the updates for the past 2 weeks: • We got above 1k deployments of Airbyte this week! • Here are the 2 winners for our contest in January: @Murilo Nigris and @Samuel Gordalina 🎉!! Every contributor should have received or soon receive something too :).  • Here’s the very first version of our API documentation. It’s in alpha, so expect changes. You won’t lose any functionality, but you may need to update your code to catch up to any backwards incompatible changes in the API. • 1 new destination connector: MeiliSearch • 2 new sources that support incremental append: Freshdesk and Sendgrid • Other fixes / features: ◦ Thanks to @Nicolas Stalder for contributing an upgrade to the Shopify source connector which now provides the landing_site field containing UTM parameters in the Orders table. ◦ Sendgrid source connector supports most available endpoints available in the API ◦ Facebook Source connector now supports syncing Ad Insights data  ◦ Freshdesk source connector now supports syncing satisfaction ratings and conversations ◦ Microsoft Teams source connector now gracefully handles rate limiting  ◦ Bug fix in Slack source where the last few records in a sync were sporadically dropped  ◦ Bug fix in Google Analytics source where the last few records in sync were sporadically dropped ◦ In Redshift source, support non alpha-numeric table names  ◦ Bug fix in Github Source to fix instances where syncs didn’t always fail if there was an error while reading data from the API

Hi guys! would you be interested to have Airbyte available as one click app on https://flashdrive.io ? Do you have a docker version with less containers required (7 now)?

Hi guys, It's possible to keep the same table name in the destination as in the source? for instance if we've a table "abcd" in the source DB, that become "public_abcd" in the destination. • Source: Postgres • Destination: Postgres

Hi Team, I am using sendgrid as source and snowflake as target. But every time I run get 0 records from source. Any specific version needs to use for sendgrid? Thanks, Nithin

hey team, ive been sharing airbyte to a couple of people, and ive noticed that the site loads very slowly. i ran a lighthouse score and the time to first paint is pretty slow. wanted to flag it just in case if people were clicking on airbyte and dropping off because they thought the site was broken!

👋 Hi everyone!

is there a documentation for version upgrades. My installation is one day old. One of the bug got fixed today. Using latest master branch from github is

docker-compose down

and

docker-compose up

from root directory. Or there are different steps.

Hey everyone, How to expand Hubspot schema on Airbyte? I would like to add: Tickets, Ticket_contact objects

by the way, Michel and I were recently interviewed on the Data Engineering Podcast. We shared about our vision, our understanding of the ecosystem, our learnings, etc. Check it out here: https://www.dataengineeringpodcast.com/airbyte-open-source-data-integration-episode-173/

👋 Hi everyone!

Hi all, here is our March Roadmap. We'd love any feedback you might have on our goals or other things that are important to you. These priorities are governed by all of the great feedback that you have given us so far! Please keep it coming.

Hello World! I would like to know whether is it possible to setup connections (sources and destinations too) without UI? I'm asking because it would be nice to do a deployment to different stages (DEV, QA and PROD) with hiding credentials. The same for schedules. Can they be maintained by configuration file deployment via i.e. Octopus?

👋 Hi everyone!, I’m trying to migrate from postgres to postgres, It’s possible to remove the

public_

prefix for the table destination?

@John (Airbyte) what is the timing on getting a proper login page for Airbyte? @Raphael Burnes and I are wondering.

Hey, about idempotent job I looked in Github issues and didn't find anything about it. Generally DE jobs try to be idempotent, that is, if they are turned again with the same parameters they should have the same result. Today there is a simple trigger in the UI and the API just receives the

connectionId

to trigger the job ... is there anything about it under discussion?

airbyte rocket We're being featured on Techcrunch again 🙂. This time, to announce our Seed round with Accel!airbyte rocket https://techcrunch.com/2021/03/02/airbyte-raises-5-2m-for-its-open-source-data-integration-platform/ Here's @Michel’s article about the seed round, explaining our vision: https://airbyte.io/articles/our-story/we-raised-a-5m-seed-round-with-accel-to-commoditize-data-integration/

Hello everyone, i have one question: when a sync fails and a new attempt is started, will the new attempt start syncing from where the previous one left? (in case of an incremental load for example)

👋 Bonjour à tous ! Congrats Airbyte !

Hey everyone! Apologies if I’m posting this in the wrong place, but I’m looking to implement AirByte for a client (local SQL Server to BigQuery) and have a couple of questions: 1. As the client is in a different country, is it normal to only have remote access initially (I’ll be using team viewer) to set AirByte on their local machine, integrate the source and destination and schedule the job to run every month? Meaning, I wouldn’t need remote access after the initial set up assuming it all works smoothly? 2. Is the scheduling completely automated? Or will there have to be some sort of manual intervention to “trigger” the replication to run?

Sorry one last question! Can the sync frequency be changed to be longer than 24 hours? The use-case I'm working on only needs a monthly sync, so I feel like syncing 24 hours is overkill

had a quick question! (more out of curiosity) any reason why airbyte decided to go with the “deploy using docker” model rather than “deploy on your VPC” model?

I wanted to ask - is there a way to fully sync without cleaned up table names? My use case is I want my local web dev env to have a replicated database to play with, but it seems Airbyte renames table names each sync. I can't see a way to change this

Quick question! Is there a way to specify the source for MS SQL on the table level, or will it always be on the database level?

Any idea on how to solve

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

error? I think this is being thrown by the Snowflake destination

Hi

I started a pipeline its synced data but how it will sync insert and update real time