First time checking Airbyte out in a while. I followed these steps: https://github.com/airbytehq/airbyte/blob/master/docs/deploying-airbyte/on-your-workstation.md And the Source Type select at http://localhost:8000/onboarding is empty. Anything I need to do to see the Sources?

Is it possible to use Airbyte in a SaaS product where we need to pull data from multiple customers and multiple sources

Heya, Ive setup Salesforce as a Source and Bigquery As a destination with normalization enabled. But it seems that all Date fields are omitted

Hi, everyone! I'm new with Airbyte. why this happen to me. im using centos 7

hi, someone using Airbyte in production? How stable has it been?

Hi everyone, my name is Khoi from Melbourne Australia. I just tried out Airbyte to sync postgres from remote to local, which works great. The tables are prefix with public. though. Not sure how to get rid of it though, does anyone has the same issue?

<!channel> Hey everyone 👋 !! here are the updates for the past week. • 2 new connectors: Plaid and Looker. A huge thanks to our contributor Thomas (tgiardina) who built the Plaid connector! • 2 new source connectors that support incremental append: Mixpanel and Hubpsot • Fixes on connectors: - Fixed a bug in the github connector where the connector didn’t verify the provided API token was granted the correct permissions - Fixed a bug in the Google sheets connector where rate limits were not always respected - Alpha version of Facebook marketing API v9. This connector is a native Airbyte connector (current is Singer based). • Most of the team was off. Still, we leveraged that time to write some articles about our learnings the past few weeks. Here's the 1st one: *Our OKRs for Q1 2021 - i*t's a good read if you're interested to know more about our focus and next milestones.

Hi, I was trying Getting Started with Airbyte able to run the example but how can I see the output, I'm on a Windows 10 machine

Can you skip the onboarding page when you have installed airbyte for the first time?

Hi Guys, I'm a new Airbyte user. I did the setup using the docker-compose using a r5.xlarge EC2 Instance. In my first test trying to move a small table (1000 rows) from postgres to snowflake, airbyte works fine. When I try to move a bigger table (2M Rows) from postgres to snowflake, the process never end with the log:

2021-01-09 19:25:59 INFO (/tmp/workspace/16/1) LineGobbler(voidCall):69 - 2021-01-09 19:25:59 [32mINFO[m i.a.i.d.s.SnowflakeDestination$DestinationImpl(insertBufferedRecords):164 - {} - max size of batch: 500

The error message is:

Process is taking too long to finish. Killing it
Sync worker failed.
io.airbyte.workers.WorkerException: target process wasn't successful

There's any way to increase the BATCH_SIZE param? and there's any best practices to process a table with more than 1M rows?

is there any method of stopping current jobs?

Hi Guys, When Airyite creates a table into snowflake, the table name has the format :`dest_schema_name.source_schema_name`.

source_name

. For example source table in Postgres :

public.student

destination table name into Snowflake:

db_name.airbyte_raw.public.student

What causes some annoying issues, because the table name has a dot inside name. A simple way to fix it should be use the

Cleaned name

that already replace the

.

for

_

Hi all! I’m not terribly familiar with docker - but how straightforward would it be to deploy my airbyte config/instance I have set up on my local workstation to a server install? Mainly thinking about replicating the sources & destinations I have set up (side note: I set up airbyte locally and very impressed - really looking forward to its continued development!)

👋 Hi everyone!

Hi Guys! I have a question, how is going the integrations with S3(Parquet) - Destination and Airflow? I would like to check which topic it's interesting to start to contribute.

I searched for

sync postgres

and Airbyte came up first! The SEO is going great! 😎

I'm curious if someone can help me point out to a resource that explains how to setup a sync that syncs the actual table content, not the

.raw

ones? Thanks in advance!

Hi everyone 👋 - I'm just joining in here after using singer.io. I was able to successfuly set up a locally hosted postgres target using singer, but am running into an error on Airbyte

Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

I was wondering if anyone knew why this may be? I'm new to using Docker (and development in general) so my knowledge of what it's doing is limited and I thought my problem may be related to it.

Hello Everyone 👋, Thanks to the Airbyte's team for their product. I'm currently building an ELT pipeline from scratch to have a unified Postgres datawarehouse. For me, even if Airbyte is young, it seems to be the good candidate to avoid proprietary cloud softwares (Stitch, Firvetran, ...) without complex setup/maintenance (Meltano, Singer, ...). My targetted BI stack will be Airbyte, DBT, Postgres and Metabase. So I can't wait to use DBT in Airbyte. 😉

Hi everyone! Just trying out Airbyte for the first time. I looks really promising, but I got stuck on the onboarding and can't find any way out of it. I added a postgres source, json destination, and then get an error in the final step. From there I'm unable to edit any settings, abort or skip the onboarding. I've tried deleting cookies, deleting docker images etc but not success. Any ideas?

A couple of points also that I have observed • Sync now is not showing me anything when I click it, it is doing nothing • Reset your data is showing me dialogue asking to Reset or No Need , When I click on Reset , it is doing nothing as if it forcing me to answer with No Need 😄

Hi everyone! I’m having some issues trying to set up a dev environment. as suggested by @Chris (deprecated profile) i am following this guide: https://docs.airbyte.io/contributing-to-airbyte/developing-locally When i do

./gradlew clean build

the build fails with error:

> Task :spotlessStyling FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':spotlessStyling'.
> Running npm command 'npm install --no-audit --no-package-lock' failed with exit code: 254

Hello guys 🙂 I am a Software Engineer/Devops at Fevo Inc. (The Social Cart). We were battling with ETL issues as we were migrating from DOMO an extremely complex ETL process, and were battling data syncs between postgres and a couple of more data source with the data warehousing table. Recently, we stumbled upon this tool, and it looked really promising, we are interested in trying it out. Who can I contact from Airbyte side about some additional info regarding security and performance aspects, as we are dealing with really sensitive data? Nice work guys, seems like you have a really nice tool there

This message was deleted.

Hi , Whenever I tried to add any source in Airbyte. I get below error in airbyte server logs. Really appreciate if someone helps to resolve.

Hi , I made a successful connection google sheet as source and snowflake as destination. I am able to launch successfully. But problem is not reading and populating data in snowflake DB. Could anyone please help me why its not reading data the googlesheet data?

<!channel> Hey everyone 👋! here are the updates for the past week: • We now have a connector health page where we show how the connectors are performing: https://docs.airbyte.io/integrations/connector-health On our journey to become ”production ready”, we’ve focused on strengthening our current set of connectors. We ensure that they follow a set of best practices. If they do, then we “certify” them (last week, we certified Stripe, Hubspot and Google Sheets).  • A new connector from the community: the App Store from @Murilo Nigris (we're working on its docs)! • Other fixes / features: - Bug fix writing boolean columns to Redshift - Bug fix false timeouts on long-running operations (https://github.com/airbytehq/airbyte/issues/1669) - Bug fix where getting a connector’s input configuration hung indefinitely (https://github.com/airbytehq/airbyte/issues/1462) - Stripe connector now gracefully handles rate limiting from the Stripe API

Hi Team, When I process a huge data in google sheets and get below error in api call. Your inputs are really appreciated here

2021-01-20 05:54:56 ERROR (/tmp/workspace/42/1) LineGobbler(voidCall):69 - googleapiclient.errors.HttpError: <HttpError 400 when requesting <https://sheets.googleapis.com/v4/spreadsheets/1IixkLcgI78cbBBpVcAwFP6BRI5C4sXZ6P2CRPzbs7t0/values:batchGet?ranges=Mass%21151154%3A151354&majorDimension=ROWS&alt=json> returned "Range (Mass!151154:151354) exceeds grid limits. Max rows: 151000, max columns: 26". Details: "Range (Mass!151154:151354) exceeds grid limits. Max rows: 151000, max columns: 26">
2021-01-20 05:54:56 DEBUG (/tmp/workspace/42/1) DefaultAirbyteSource(close):109 - Closing tap process

Hi everyone, I'm testing Postgres to Postgres syncing with Incremental - Append using docker-compose on my computer: https://docs.airbyte.io/deploying-airbyte/on-your-workstation. I'm getting the following error on every sync attempt (full log attached):

2021-01-20 19:55:26 ERROR (/tmp/workspace/7/0) LineGobbler(voidCall):69 - Exception in thread "main" java.lang.NullPointerException

Any thoughts?. Am I missing something?

Hi, Airbyte says successful sync (incremental refresh - based on

id

), but I am checking the table in the destination db, it has exactly the same number of rows as before, even though Airbyte says it synced ex. 17,009 records, and I see it still has the old count 16,539 Any chance I can browse logs more deeply?