Hello everyone, we're trying to install airbyte using portainer and its docker swarm. We've seen this issue from a week ago https://github.com/airbytehq/airbyte/issues/19787 but the containers have trouble seeing each other. Has somebody already deployed successfully in portainer?

Hi everyone 🙂 just installed airbyte on local RHEL with docker compose. When trying to create connection from

Oracle DB

to

postgres

(greenplum to be more precise), we can see the UI table "Activate the streams you want to sync" with database tables in

Oracle DB

but when selecting one and set up connection we get

The connection tests failed.
non-json response

We managed to setup

Microsoft SQL Server

source to the same destination

postgres/oracle

and its working fine. Any help?

What is the method for adding changes after deploying airbyte in GCP? I need to make changes to the nginx config template and rebuild the image, but I can't access the airbyte-proxy folder from gcloud terminal like i would in my local machine

Hello! I'm currently working with a client to replicate data into Snowflake and we're using Stitch. One of the limitations of Stitch is that it can't/won't migrate records greater than 20 MB in size. Is this something that Airbyte suffers from too? Or does it handle records of any size?

Hey team, is it possible to change the default file size exported on S3 using the values.yaml? Basically I’m running Airbyte using a docker image out-of-the-box at Kubernetes, so I don’t have access to the source-code itself.

Hi Team, we are trying to connect the dev airbyte setup with temporal cloud , we have configured TEMPORAL_CLOUD_* varaiables in server and worker variables. however we are seeing below errors and server and worker components are not working as expected. please provide any inputs to resolve the issue . Errors : airbyte-server | 2022-12-02 155010 WARN i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):235 - Waiting for namespace <temporal_cloud_ns> to be initialized in temporal.

So I installed airbyte on a linux server. when I try to create a source on airbyte, I get this error Caused by: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-mongodb-v2:0.1.19

Let's say I have Facebook Ads source and Big Query Destination. Using the Campaign data stream, how could I query last month spend for all campaigns? Having a heck of a time finding all the necessary components so any help would be much appreciated.

I have been trying unsuccessfully to create a s source connection. I keep getting a connection tests failed.

Sync Failed *Last attempt:*0 Bytesno recordsno records24s Failure Origin: source, Message: Something went wrong in the connector. See the logs for more details. 12:33PM 12/03 3 attempts 2022-12-03 070417 - Additional Failure Information: java.lang.IllegalStateException: Source does not support nested cursor fields.

Hello, Please, I need guidance on how to effectively manage airbyte slot connection to Postgres. I only have 1 airbyte slot on production Postgres server setup in CDC mode with 5 minutes sync intervals, target goals is to have near real-time sync. I was told by the Devops team that it is slowing production dB down. I will be glad if I can be point to where this issue is coming from

Hey everyone, I am trying to learn connector development in Python. I have completed the step 3 of the document. But stuck at step 4 when checking connection. This is the document I am following: https://airbyte.gitbook.io/airbyte/connector-development/tutorials/cdk-tutorial-python-http/4-connection-checking When I run the command:

python main.py check --config sample_files/config.json

I get the error saying:

{
  "type": "CONNECTION_STATUS",
  "connectionStatus": {
    "status": "FAILED",
    "message": "Config validation error: 'api_key' is a required property"
  }
}

In the document it is mentioned to add api key in `secrets/config.json`:

While developing, we recommend storing configs which contain secrets in

secrets/config.json

because the

secrets

directory is gitignored by default.

I generated the api key from the exchange rates api website, added it in the

secrets/config.json

file, and build the image using this command:

docker build . -t airbyte/source-python-http-example:dev

. But the error still persists. I am contributing towards documentation. It is important for me to understand how connectors work. Here is the Github link of the partially completed example I was trying: https://github.com/Jagrutiti/airbyte/tree/source-python-http-example-connector/airbyte-integrations/connectors/source-python-http-example

Hi team, am using airbyte deployed in aws ec2 instance. My disk space getting fulled by airbyte now. Need to clear the logs manually now. I installed airbyte using docker compose file. How can I delete the airbyte logs manually, please help me on this

Hey everyone 👋, I’ve opened a PR to fix a bug in the Pivotal Tracker source here. It seems that I don’t have permission to add reviewers for this one. What should I do next for the reviewing process?

Hi there, I wanna to ask a question. I have build a new destination connector and deploy the open source airbyte, how can I add my connector to the airbyte so that I can use it in web.

When I click

add

I got the error:

Hey All, I am looking support to install Airbyte on gcp kubernetis and learn some basics from you masters. Please DM if any one can support, I will pay for your hours

Hello there, I have wrote a new destination with normalization support in this PR, https://github.com/airbytehq/airbyte/pull/19815 , would you like to help me review it? Thanks very very much!

Hi all 👋 I run airbyte locally in docker on mac m1. I’m facing with following issue: all containers are running, except

init

and

airbyte-bootloader

. What can be a reason? Adding logs in comments below.

Hi all, I am using the MS SQL Server destination in a connection to load some JSON data from a custom source connector into an Azure SQL Server. I have set up an Incremental | Append stream in the replication settings and I have enabled the Normalized tabular table under Normalization in the Transformation Settings. After the sync succeeds I can see the raw table in my destination database and it has the JSON data I expected in the column __airbyte_data_, but there is no normalized table containing the JSON properties as columns anywhere in the database. Do I have to change something to see the normalized table? Can you help me solve this? I have attached the logs in a comment below.

Hello All, We are syncing one table which has 48 crore rows from gcp cloud postgres to google biqquery, after syncing 42 crore rows I am getting error, attaching error in screenshot, not understand the reason for error as the table is static no crud operation is happening on table.

Hi Team octavia wave I have a problem with the Snapchat Marketing connector and I hope you’ll be able to help me pinpoint the cause. There’s a repetitive pattern of actions. What is happening: • I reset the stream and resync the whole thing - it runs perfectly ✅ • It’s scheduled every day at midnight and it runs with a success the next day, syncing all streams. ✅ • The following day it fails with:

"Additional Failure Information: expected string or bytes-like object"

. The problem is coming from the source. This means that every 2 days I’d have to reset and resync… octavia shock I’m attaching the exemplary log and the setup of the connector in the coment below 👇 Any ideas will be so, so much appreciated 🙏 😢

Hi Team, I'm new to Airbyte. I'm trying to set up the destination as RabbitMQ in Airbyte, RabbitMQ is running in my local in port number 31673, but I'm getting below error "Could not create connection: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)" and I tried with default port (5672), but I'm getting below error "Could not create connection: AMQPConnectionError: (AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused'),)" Please find the screenshots below for your reference.

helloo, what's the best way to add a new sourceDefinition to my instance without using the UI?

How can I set up CDC for Managed Postgres Instance on GCP?

Hi team! Despite my best efforts, I'm still unable to update a custom connector in my prod instance of Airbyte. We're on 0.40.23, so https://github.com/airbytehq/airbyte/issues/19637 should be fixed by now. But when trying to update a connector that works locally, I get the Internal Server Error: Get Spec job failed. The connector works though, and the image docker appears to be found... Is this an issue on Airbyte's side, or am I missing something?

Hi team, I saw this env variable was added to the env file recently. I’m wondering does this mean source schema change can be detected automatically now?

Hi all, is there a good way to run multiple airbyte instances on the same local machine using docker compose? I would like to not have any shared configuration, volumes, db, etc. I found an old question https://airbytehq.slack.com/archives/C021JANJ6TY/p1638783666080900 but unfortunately no answers. Thanks!

Can anyone tell me what this means? I am trying to use google analytics 4 as source using client ID, client secret, refresh token and access token

looking for some help/ inputs with error exception while setting up zuora