what do you think about the layout of our new blog? https://dataline.io/blog/

shouldn't we have a channel #changelogs ?

guys, have you heard of Hacktoberfest? apparently, this is a good opportunity if we want to find new contributors. You can host your own event - details here: https://hacktoberfest.digitalocean.com/eventkit

team, what do you think about doing weekly office hours (after we launch) for the community: inspiration = https://hasura.io/events/office-hours/

this would be different from community calls where we feature the product progress and community highlights, such as https://hasura.io/community/community-call/

Welcome @Iain Donaldson! 👋 let us know if you have any questions. also very happy to discuss about your data integration needs, and how soon we can address them 🚀

👋 Hi everyone! Love where this project is going. Is there a way to use Airbyte to pull in data from cloud accounts of our customers using OAuth? (similar to Fivetran's connect cards)

👋 Hi everyone!

👋 Hi everyone! Just heard about you all on Twitter, and love everything about this.

Glad to be here. How quickly do you think we may be able to spin up a Facebook Singer connector on Airbyte if we can contribute to the source?

About us - we're a marketing automation company focused on non-profits and progressive political campaigns -- we're about to have $2-3M in Facebook ads run through the platform in the next 40 days, would love to automate the data pipeline, and have found Fivetran / Stitch to be restrictive for all the reasons you write about.

Hello , I just tried to install airbyte on a GCP instance that is running Centos 7 , everything seems to normal from docker logs perspective, Yet the UI is not showing anything except a blank screen only the blue loading then a blank screen,

root@airbyte-sa ~]# docker ps

CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS              PORTS                              NAMES

ae120c1f8776        airbyte/webapp:0.1.0-alpha      "/docker-entrypoint.…"   14 minutes ago      Up 14 minutes       0.0.0.0:8000->80/tcp               airbyte-webapp

ef54d71cd0fa        airbyte/scheduler:0.1.0-alpha   "/bin/bash -c './wai…"   15 minutes ago      Up 14 minutes                                          airbyte-scheduler

5fd047d9e0c0        airbyte/server:0.1.0-alpha      "/bin/bash -c './wai…"   15 minutes ago      Up 14 minutes       8000/tcp, 0.0.0.0:8001->8001/tcp   airbyte-server

217c72742810        airbyte/db:0.1.0-alpha          "docker-entrypoint.s…"   15 minutes ago      Up 15 minutes       5432/tcp                           airbyte-db

[root@airbyte-sa ~]#

I'm running airbyte locally, and when attempting to connect a PostgreSQL source, I'm given this error:

The connection tests failed.
The received form did not pass validation

The database is publicly accessible and the host/port/username/password/database are correct.

hey everyone, airbyte looks amazing! is there going to a paid hosted version in the future?

hey all! we'll hold our 2nd office hour in 15 minutes here: https://us02web.zoom.us/j/84317290190?pwd=S0E5RzZRV2NrTGw1VDM5cFZoV2JCUT09 the team will hang out and answer any questions you have. we'll also be able to show what we're working on.

hey all! we wanted to share about what we're currently working on (the 0.2.0). our goal with this version is the following: • a new Admin section to enable users to add their own connectors, in addition to upgrading the ones they currently use • improve the developer experience (DX) for contributing new connectors with additional documentation and a connector protocol • as a bonus if we can, update the onboarding experience with pre-filled demo data for the users who just want to see how Airbyte works with the least effort. Here is what 0.2.0 is going to look like: https://www.figma.com/file/pO3Ob5W0yKUFOQzvg8TR3z/Airbyte?node-id=0%3A1 We will continue to improve this Admin section, so think of it as a 1st version for it. don't hesitate to add your own comments. our goal in sharing with you is to get your feedback and see if we missed anything! thanks!

Hi All!

Let me know how to resolve this issue

sharing your email address is optional. we take privacy super seriously at Airbyte. for now, we were not sending email notifications when any of your syncs failed. would you want to have that notification option? and if you were not sharing your email address until now, would you change your mind? (not trying to convert to sharing, just trying to understand the motivations)

Hi everyone, A big focus of our upcoming releases this year will be adding a lot of new data sources to Airbyte. So we’d love to hear from you: which data sources and destinations would you like to see? Fill out this very short survey: https://marketing824805.typeform.com/to/ZxTgRuXR to let us know which sources or destinations you’d like us to support 🙂

Hi all, great work on Airbyte so far. We're trying it out at Iteratively and set up a simple configuration with sync from PgSQL to BigQuery on an EC2 instance every 15 minutes. The job ran fine 9 times but on 10th try failed with:

2020-10-19 18:36:33 INFO (/tmp/workspace/15/4) WorkerRun(call):58 - Executing worker wrapper...
2020-10-19 18:36:33 ERROR (/tmp/workspace/15/4) SingerSyncWorker(run):86 - Sync worker failed.
java.lang.RuntimeException: java.nio.file.FileSystemException: /tmp/workspace/15/4/target_config.json: Too many open files
at io.airbyte.commons.io.IOs.writeFile(IOs.java:50) ~[airbyte-commons-0.1.0-alpha.jar:?]
at io.airbyte.workers.protocols.singer.DefaultSingerTarget.start(DefaultSingerTarget.java:69) ~[airbyte-workers-0.1.0-alpha.jar:?]
at io.airbyte.workers.protocols.singer.SingerSyncWorker.run(SingerSyncWorker.java:71) [airbyte-workers-0.1.0-alpha.jar:?]
...

Have you seen this before?

Hi there, very promising tool 🙏 . From the screenshots, it seemed like I can use any singer taps? But from what I can tell it only has 3 integrations. How does one go about adding more from singer?

Hi there, After reading the documentation of the architecture of AirByte, I am so interested in AirByte and can’t wait to try with it. But I was just blocked at the beginning when I want to try sync data between the PostgreSQL database. It was just hanging at the first step as the pic shows below. And I can’t see any error in the log. Do anyone have any suggestions for me to do next? Sincerely thanks.

Hey all! I raised this briefly as a 'fun thing to share' at work. The #1 question I received was:

What scale is this intended for?

For context, we often move ~100GB sets, occasionally up to ~10TB, generally between databases, sometimes through file stores (Azure Storage, SFTP, GCS) but sometimes from our events system too.

Airbyte looks so great! At AgriDigital we are literally in the middle of looking for an alternative ETL setup. We are currently utilising a poorly managed AWS Glue setup, which while it does the job, is very cumbersome and costly. Came across you guys today and it looks so great. Would love to give this a shot, but was wondering if there is a redshift connector (even alpha) available to play with?

<!channel> just created a #C01DSEKTTHB channel where we will announce the new releases and new connectors we support. I invite you to join it!

I got a question here. Today I want to recommend AirByte to our team as a data integration tool. But I was challenged that why not we use Sqoop or Flume as an option. But I only can say that AirByte can be used for many connectors not only for database but also for ReatAPI, CSV file, or even cloud service, such as google sheet, etc. Besides that, Is there anything I can say to convince our team to try AriByte?

I know Airbyte will integrate with DBT and Airflow. However it seems like I need to use those right now. What is the best practice for me to manage DBT and Airflow projects? So that in the future I will integrate to Airbyte when it is ready

How can I save Airbyte data, so I can control version or deploy to server?

Will there be any encryption between source and destination?