Hi @Rakhi Modi, we have an open PR which is doing a bit of refacto on the helm chart for the secret management. Feel free to checkout this branch and try the helm chart again: https://github.com/airbytehq/airbyte/pull/11020

Hey can you elaborate more on this?

Is this your first time deploying Airbyte: no OS Version / Instance: Linux EC2 m5.4xlarge Deployment: Docker Airbyte Version: 0.35.50-alpha Source: Survey Monkey 0.1.7 Destination: Snowflake 0.4.17 Description: Incremental Setting does not fetch all the records even for the first sync (“full refresh”)

Logs

G’day friends. I’ve submitted a GH issue over a month ago and it hasn’t received any feedback (yet). Are you able to make any headway on this? It’s blocking me from using Airbyte for our startup…and I would really love to because it’s the only solution that allows me to connect to a local MySQL DB with an SSH tunnel! Currently spending 10+ hours per week in data engineering manually so it would be a HUGE lifesaver. Thank you ❤️ cc: @Justin Chau (Airbyte) (sorry for tagging you 😂 )

Hi! Receiving the same error, has there been any progress on this? When I check my token with the Graph Explorer (https://developers.facebook.com/tools/explorer/) it is valid and I can query e.g. /published_posts but that only allows to test for API v13.0 and Airbyte seems to use v12.0 , Thanks!

Today it’s not possible @Mariusz Zaręba

• @here : Airbyte version : 0.35.37-alpha When using postgres sql connection , connection is success but dont show schema objects... Any special permission we should give to schema or tables in the schema to visible in airbyte connection .....or is this issue with the airbyte product specific to postgre sql source . Any help would be appreciated

Is this your first time deploying Airbyte: No OS Version / Instance: Amazon Linux 2, AWS Memory / Disk: 16GB / 30GB Deployment: Docker Airbyte Version: 0.35.15-alpha Source name: Google Analytics 0.1.16 Destination: Snowflake 0.4.22 Normalization: Basic Normalization Sync Mode: Incremental Append I have loaded the data initially without incremental append. The data load went through without an error. Now I am getting an error, If I am scheduling a daily sync. Is the issue with the incremental append?

Hi Team, I am building a new connector when i am running the main file it is working fine but when i build docker images and pull in the source airbyte.

Is this your first time deploying Airbyte: No OS Version / Instance: Debian GNU/Linux, 10 (buster), GCP ,n2-standard-2 Memory / Disk: 2 vCPU, 8Gb RAM / 50GB Balanced persistent disk Deployment: Docker Airbyte Version: 0.35.59-alpha Source name/version: Hubspot 0.1.50 Destination name/version: BigQuery 1.0.0 Description: When starting the first incremental sync for the company stream, it stops at 1.7M/6.6M records read. For some reason, the state is modified during the first full sync. (“Advancing bookmark for companies stream from None to 2022-03-23T205211.671000+00:00”). After that, the incremental sync seems okay but I have missing company due to a incomplete first full sync. Thanks for your help 🙏

Hello everyone, I am facing one issue with airbyte. I have submitted around 14 connections in airbyte. And none of them is making progress. All of them are kind of stuck.

loading...

Is this your first time deploying Airbyte: Yes OS Version / Instance: Ubuntu 20.04.4 LTS on Azure Memory / Disk: 4vcpu, 16GB ram, 30GB disk Deployment: Docker-compose Airbyte Version: 0.35.35-alpha Source name/version: MSSQL / 0.3.17 Destination name/version: csv on blob / 0.1.3 Description: When the incremental sync seems to be complete, the source seems to hang for 5 minutes. Then the database pool is closed, and the sync process fails. The record count keeps increasing on every sync, so it’s not a single record causing the problem, and it seems like the sync is actually complete - the source just doesn’t realise it yet.

Hello! Is this your first time deploying Airbyte: Yes OS Version / Instance: Ubuntu 21.10 Memory / Disk: 16Gb / 512GB Deployment: Docker Airbyte Version: 0.35.54-alpha Source name/version: Salesforce (sandbox) 1.0.2 Destination name/version: Snowflake 0.1.10 Step: On sync / normalization Description: I've set up my source and connexion successfully. When trying to sync (or refresh data, just in case) I get normalization errors (

2022-03-24 09:49:49 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. errors: $.method: does not have a value in the enumeration [Standard]

). I even tried sending raw JSON but I get the same error. Anyone encountered this error? I always assumed salesforce to snowflake wouldn't quite be out-of-the-box but I'm a little lost as to how to fix this. Happy to provide logs if it can help.

hi

Is this your first time deploying Airbyte: Yes Memory / Disk: 8Gb / 16GB Deployment: EC2 Airbyte Version: 0.35.42-alpha Source name/version: JIRA Destination name/version: Snowflake Step: Source creation Description: Trying to set a new source connection to jira, and getting the following error: HTTPError('401 Client Error: Unauthorized for url: https://zoomin-software.atlassian.net/rest/api/3/resolution'). Im able to log in into Jira with the user and the API token is correct. in addition, under manage API tokens in atlassian admin, we can see that the token was used to access the platform. any ideas what should be done?

Is this your first time deploying Airbyte: Yes OS Version / Instance: MacOS Monterey (M1) Memory / Disk: 16Gb / 1Tb Deployment: Docker Airbyte Version: 0.35.59-alpha Source name/version: Zendesk Support 0.2.2 Destination name/version: Redshift/CSV/JSON (tried all) Step: On sync Description: When extracting data from Zendesk “tickets” the worker just stops, tried multiple times, even using an Intel Mac, have the same problem… It always stops in the same part [Records read: 54000 (208 MB)]. I’ve tested other tables from Zendesk (organizations, users, etc…) and they apparently are working fine. I was able to full extract the tickets table using an old version of Airbyte (0.29.11).

Hello, I'm on Airbyte 0.35.59-alpha, created a Kafka source that connects to vanilla Kafka installation (kafka_2.13-2.8.1). I have an Elasticsearch destination as well. When I try to create a connection between Kafka source and Elasticsearch destination, I got this screen multiple times. Basically it says, "Failed to fetch schema, please try again later" I'm stuck at this point. Any help would be appreciated. OS: macOS Monteray Memory: 16GB Deployment: Docker

is this your first time deploying Airbyte: Yes OS Version / Instance: Ubuntu 18.04.5 LTS / EC2 CPU / Memory : 2 / 8Gb Deployment: EKS Airbyte Version: 0.32.0-alpha Source name/version: PostgreSQL - 0.3.7 Destination name/version: S3-Parquet - 0.1.1 Step: On sync ( Full refresh - overwrite) Issue - When i am submitting multiple connections ( having postgresql as source and s3-parquet as destination) on airbyte then for sometime connection works fine but after sometime connection doesn’t get finished. It just hangs in running state forever and never finish also no new connection starts on the airbyte. Also I checked the source and destination pods in EKS, they also remain in running state and never gets completed. Any help would be appreciated.

which slack channel is the right one for airbyte with dbt issues ?

Is this your first time deploying Airbyte: yes OS Version / Instance: Amazon Linux 2 / AWS EC2 t2.large Deployment: Docker Airbyte Version: 0.35.56-alpha Source/version: Marketo Description: Trying to set up Marketo Connection. The connection setup seems to be working but the first syncs keep failing. Getting this error "errors: $.method: does not have a value in the enumeration [Standard]"

Is this your first time deploying Airbyte: yes OS Version / Instance: Mac Monterey Deployment: Docker Airbyte Version: 0.35.59-alpha Source/version: PostgreSQL 12.8 ==> Airbyte source connector version 0.4.4 (downgraded from 0.4.9 to show schemas) Description: Trying to copy table to

Is this your first time deploying Airbyte: yes OS Version / Instance: Mac Monterey Deployment: Docker Airbyte Version: 0.35.59-alpha Source/version: PostgreSQL 12.8 ==> Airbyte source connector version 0.4.4 (downgraded from 0.4.9 to show schemas) Description: Trying to copy table to Snowflake but no records showing up in sync. The process runs fine without any errors. Log is attached

Is this your first time deploying Airbyte: yes OS Version / Instance: Mac M1 Deployment: Kubernetes Airbyte Version: 0.35.30-alpha Source/version: PostgreSQL 12.8 ==> Airbyte source connector version 0.4.4 (downgraded from 0.4.9 to show schemas) Description: Hello guys, I'm testing airbyte sync from postgres to s3 and is taking too long to finish and we notice that has an issue on github to discuss about modify fetchSize that is hard coded to 1000. We want to change this value but the only way that we find to do that is changing from source code and build our image but we don't want to maintain an exclusive image, and the github issue has not received any updates since 11/08/2021. Do you guys have any update about this feature? Or any alternative to make this sync faster?

Hello , quick question because i’m not sure to understand well 🙂 I updated a connector version to get a new available stream I click on refresh source schema on replication settings to get the new stream I want to sync is it really mandatory to reset the data and resync all data just for one new stream to add ?

Is this your first time deploying Airbyte: Yes OS Version / Instance: Windows Memory / Disk: 32Gb / 475GB SSD Deployment: Docker / Windows Local Airbyte Version: 0.35.39-alpha Source name/version: MySQL version 0.5.6 Destination name/version: Snowflake 0.4.22 Step: Initial Sync after creating connection Description: I'm trying to sync a single table from mysql with around 13m rows, failed 3 times. Logs appear to be showing that normalization failed.

Hi! I don't have much budget to run a production instance of Airbyte. I have a baremetal server (32 gb + 500gb SSD). I would like to know if anybody is running something similar in production? Any suggestion/docker compose config or similar? thanks

Hi everyone. I am using the Google Ads Connector with

Incremental | Append

Sync mode, however it won't incremental sync the data. It just appends that data every time it refreshes. Can someone help me with this? I read the logs and I couldnt see anything wrong. Google Ads version: 0.1.28 S3 version: 0.1.10

Is this your first time deploying Airbyte: No OS Version / Instance: Amazon Ubuntu 20.04 Memory / Disk: 64Gb / 250GB Deployment: Docker compose Version : 0.35.60-alpha The DBT clickhouse connector already comes with has bugs and is failing. I am not able to see the option of adding my own DBT, where to add it?