Hi, trying to connect to a local clickhouse, which responses to

echo 'SELECT version()' | curl '<http://localhost:8123/>'

but airbyte throws this error

Hi 🙂 Could you please help me with a "howto" question. My problem is that one of our sources is an API of our internal service. It does not seem mindful to add that connector in the Airbyte repository, nor to build another our service as an intermediate storage. Is there a way to have a plugable connector outside of the Airbyte repository? Or is there some customizable connector available? Maybe there some stories of success or examples?

Hi everyone, I’m using S3 connector to sync data from S3 into Postgresql, but in case some csv file doesn’t correct format ( delimiter & quote character doesn’t not match with S3 source config) so the sync is failed. Do we have any option to skip the file error and continue to process the rest of file in S3 ?

Hello Everyone, has anyone succeeded with connecting Airbyte to GCP Cloud SQL database? I can’t connect to it

│ 2023-01-10T15:16:35.983Z    ERROR    Unable to create SQL database.    {"error": "unable to connect to DB, tried default DB names: postgres,defaultdb, errors: [pq: pg_hba.conf rejects connection for host \"85.128.86.2\", user \"airbyte\", database \"postgres\", no encryption pq: pg_hba.conf rejects connection for host \"85.128.86.2\", user \"airbyte\", database \"defaultdb\", no encryption]", "logging-call-at": "handler.go:97"}2023/01/10 15:16:36 Loading config; env=docker,zone=,configDir=config                                                                                                                                                     
2023/01/10 15:16:36 Loading config files=[config/docker.yaml]

Hello Everyone, has anyone succeeded with connecting Airbyte to GCP Cloud SQL database? I can’t connect to it. I’m using Kustomize to apply custom connection credentials

│ 2023-01-10T15:16:35.983Z    ERROR    Unable to create SQL database.    {"error": "unable to connect to DB, tried default DB names: postgres,defaultdb, errors: [pq: pg_hba.conf rejects connection for host \"**\", user \"airbyte\", database \"postgres\", no encryption pq: pg_hba.conf rejects connection for host \"**\", user \"airbyte\", database \"defaultdb\", no encryption]", "logging-call-at": "handler.go:97"}
2023/01/10 15:16:36 Loading config; env=docker,zone=,configDir=config                                                                                                                                                     
2023/01/10 15:16:36 Loading config files=[config/docker.yaml]

hello, setting up airbyte and getting the below error. can anyone provide guidance?

Could not connect with provided configuration. Error: Failed to get driver instance for jdbcUrl=jdbc:<snowflake://https>://yea.us-east-1.snowflakecomputing.com//?role=AIRBYTE_ROLE&warehouse=AIRBYTE_WAREHOUSE&database=AIRBYTE_DATABASE&schema=AIRBYTE_SCHEMA&JDBC_QUERY_RESULT_FORMAT=JSON&CLIENT_SESSION_KEEP_ALIVE=true&application=airbyte_oss

Curious why my Helm deployment creates these two containers:

airbyte-oss-postgresql-0                                         1/1     Running     0          
airbyte-postgresql-0                                             1/1     Running     0

This is specified in my Helm values:

postgresql:
  enabled: false

externalDatabase:
  host: ${db_host}
  user: ${db_user}
  password: ${db_password}
  database: ${db_name}
  port: ${db_port}

Hi all! I´m struggling to set the OAuth authentication to a new source connector that I need to have. My problem is that the API requests secret_id and secret_key in the body for asking for a new refresh and access token (I´m asking for a new one and not refreshing the access token with the refresh token, as the refresh token expires every 3 days). In the Airbyte documentation the client_id and client_secret properties are required.. but for the api call to work for my api, it should be secret_id and secret_key. I tried to modify it and it gives me error the client id is required field. And when I run it with the Airbyte names the api call fails

hi, I am trying to understand the failure and recovery modes in airbyte. • I have a source with 100 tables, and need to sync them all to one distination. • The sync can take many hours. • From time to time a single table will fail e.g. due to schema issues etc. • Desired behavior: ◦ Single connection for the full 100 tables ◦ Job attempt to sync all tables ◦ Failure in one table doesn’t stop the job - but the result of the job will contain the tables that failed ◦ Ability to “retry” only the failed tables - including manually Is that possible? Or is it that a single connection succeeds, fails and retries monolitically?

Hey guys 👋 I encountered a weird behavior accessing the webapp and wonder if you've ever seen it I installed Airbyte on K8s using the official Helm chart, port-forwarded the webapp port (80) to my localhost:8080. When trying to access localhost:8080 I'm not even sure what happens, it seems like a 30 seconds timeout to me but I ain't no expert (See attached screenshot) Weird thing is that it works fine for other folks so I guess it is something with my machine osx 12.6.1 chrome 108.0.5359.124 (Official Build) (x86_64)

Hi all, I'm strugling to adjust the api call I successfully make in postman to an actual source connector. The thing is that I need to first make api call to get the access token (I have secret_id and secret_key and NOT client_id and client_secret). Therefore I wrote the process, but I'm stuck in passing the config to the stream. def streams(self, config: Mapping[str, Any]) -> List[Stream]: auth = NoAuth() return [Transactions(authenticator=auth)] Here I need to pass the config object. Attaching the source.py file, If anyone can help me. I'm following the cdk-tutorial-python-http tutorial. Thanks!

Hello, I am working as a Partner Engineer with Vertica. We are in the process to develop a Vertica connector for Airbyte and would really appreciate it if you can help us answer our questions. Could you also let me know how much is the estimated time taken to develop a database connector with source and target? Thanks!

Hey! I’m running a destination connector I am building locally, and I’m connected to a postgres source with CDC enabled. The sync looks fine on the destination end and I see a log message that the sync completed, but it hangs for minutes with this log (in thread) and then fails. anyone seen this before? I am yielding all state messages in the destination connector.

Hi team , can anyone let me know current stable version of airbyte. I was using 0.40.22 , but in UI it wasn't lloading

@Dheeraj Pranav I had the same issue this morning after running a git pull to update, my UI resolved after I let the server run for 5-10mins after docker-compose up. Not sure if this will work for yourself, but worth a try.

Hello, anybody had experience with webhooks using Airbyte?

image.png

Hello! Can anyone help with this? I'm having a weird error where records emitted and committed by a connector are inconsistent if I use the postgres destination with ssh tunneling https://github.com/airbytehq/airbyte/issues/21251

Hi team, I am very new to airbyte connector development. Is there a way of pulling data from source for preview, without creating an actual connection?

Hey Airbyte team! Is it possible to use different schema for Airbyte Postgres database?

Clickhouse connection is not working anymore. Two days ago it was working fine but today it does not connect at all with same configs. I tried to mess with the configs, using native port number, with or without ssl etc, no luck

Anyone facing issues on Airtable Source connector after upgrading to 1.0.1? https://github.com/airbytehq/airbyte/issues/21281

I am attempting to upload a custom destination connector to a cluster on which I am hosting an AirByte open source instance. The upload appears to happen, and I see the

spec

Docker image be invoked and apparently finish successfully. Running

spec

locally from a Docker image works. After a bit of time, I am automatically redirected to the Account tab and my new destination connector is nowhere to be found. I’m running AirByte 0.40.25.

Can anyone help with this. Saw some solutions related to same issue that on windows we should use WSL. Did that but the issue persist

Hi all! I need to get data from a csv file that is in my google Drive.Any ideas how to do that?

Hi all, for the MySQL source connector, why JSON is emitted as a String and got committed in my Snowflake destination as VARCHAR(). If this is how it works, how can I convert it to JSON? and I think there is also a limit of characters for VARCHAR in case of a long JSON, right?

for the MySQL source connector, why JSON is emitted as String and got committed in my Snowflake destination as VARCHAR().

Any if this is how it works, how can i convert it to json?

and I think there is also limit of characters for VARCHAR in case of a long json, right?

Hi all, Maybe a noob question but I am not able to load Airbyte on my local. It just keeps on loading.