I'm running into issues using Hasura to track database tables transferred from Salesforce to Postgres using Airbyte. Airbyte creates index names like in my screenshot and Hasura complains when trying to track the table that the name of the index isn't valid GraphQL syntax. Therefore, is there a way to prevent naming indexes that Airbyte creates from being starting with a number?

Hello Team, I have started using airbyte since last week and I have found it very useful to send data from different data source to my data warehouse. Currently I'm facing a problem wondering anyone here can help me. Source : MySQL (5.7) Destination: Redshift Problem: 1. When my datetime data type values in mysql are sent to redshift, they are being converted to datetime strings with random (or it seems) time zones in redshift. Sometimes it is subtracting five and half hours from mysql datetime and sometimes its not. Mysql table name Mysql datetime -----> Redshift datetime orders '2021-01-10 120000' ----> '2021-01-10T120000:00Z' deposits '2021-01-10 120000' ----> '2021-01-10T063000:00Z' Ask: 1. Understand why this is happening 2. How to prevent this from happening

Hi Team, I streaming data from Github to Snowflake using github and snowflake connector. When I ran the process, it takes 40 mins to start syncing stream process from the source, it seems idle/waiting something. I tried multiple times and in different environment also showing the same result. why does it take 40 mins to start syncing stream process ?

Hi Team, We can't get data to S3 since two days ago using Airbyte 0.35.9 and upgrading to 0.35.30 didn't help either. I'm attaching sync log Debug info details as follows: Airbyte Version: 0.35.30-alpha Source: Salesforce (0.1.23) Destination: S3 (0.2.7)

Hi all, I'm wondering how I can change the domain in the body of the webhook notification. In the message below, I would like to change localhost:8000 to the public address where airbyte is being hosted. It doesn't appear that this is an option in the interface, i there a configuration file somewhere where this can be changed?

Your connection from Okta version 0.1.4 to BigQuery version 0.6.7 succeeded

This was for sync started on Friday, February 18, 2022 at 10308 PM Coordinated Universal Time, running for 8 minutes 29 seconds.

You can access its logs here: http://localhost:8000/connections/792cdbca-8f3f-446f-ac8c-bd4d94e5285d

Hey there and happy Friday! I'm searching for the Bearer token of Airbytes API; Where is this set or how can I retrieve it? I'm using a docker deployment.

Is this your first time deploying Airbyte:  No Airbyte Version: 0.35.30-alpha Source: Salesforce (0.1.23) Destination: S3 (0.2.7) Deployment: Kubernetes Hi team, When using s3 destination with format type 

Parquet

 it seems not to support instanceprofile. Do I get it right that S3ParquetWriter is using SimpleAWSCredentialsProvider in any case?

Hello guys! Me again! I got a new problem! I made a connection source Postgres to destination Bigquery using CDC follow all documentation steps. Made a test for my staging database and works fine, when I move to production i got a error

ERROR i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null

There any thing I could do? OS Version/Instance: ec2.t2.large Memory/Disk: 8Gb/ 50Gb Deployment: Docker EC2 Source name/Version: Postgres version 0.4.7 Destination name/Version: Bigquery 0.6.8 Airbytes Version: 0.35.4-alpha

Hello, I've got AirByte set-up to pull data from a MS Sql Server and push the data to Postgres server on AWS, I'm able to run the job but no data is being sent. Is there a configuration setting that I might be missing.

Is this your first time deploying Airbyte: No Deployment: Airbyte Cloud Source name/version: Bigquery Destination name/version: Postgres Description: "Something went wrong with the source connector" I followed the same instructions for the same connection and it worked so don't know whats different here and couldn't find a useful error message in the logs

Hello, Is this your first time deploying Airbyte: Yes OS Version / Instance: Digital Ocean Droplet - Ubuntu Memory / Disk: 8Gb / 80Gb Airbyte Version: 0.35.23 Source name/version: MongoDB Destination name/version: Redshift Step: sync failed without error Description: after some successful sync tests using a few data, we tried to sync all our source data (~400 millions rows) but the synchronization failed without any error message. The sync attempt is marked as failed but the log file doesn't show any error message.

Hey Everyone, Is this your first time deploying Airbyte: No OS Version / Instance: EC2 - Amazon Linux 2 Memory / Disk: 8Gb Airbyte Version: 0.32.3-alpha Source name/version: Smartsheet Destination name/version: Postgres Description: I am using airbyte to connect to one of my smartsheets and I see a problem. The order of the columns in the smartsheet is not maintained in the json data (brought in by airbyte) because of which the data is being saved rather erratically (for example the Date column gets stored with data from some other column) causing the integration to break and no data comes in. Does anyone know how I can solve this?

The docs for the Shopify connector might need to be updated given that private apps are depreciated now. Also its unclear how to connect using the using 

OAuth 2.0

 option

Ask for help submission from @Pras *Is this your first time deploying Airbyte * No *OS Version / Instance * GKE, n2-standard-4, cos_containerd *Memory / Disk * left blank *Deployment * Kubernetes *Airbyte Version * v0.35.12-alpha *Source name/version * left blank *Destination name/version * left blank *Step * left blank *Description *

JOB_MAIN_CONTAINER_*

settings are not applied to jobs. This is a recently updated deployment, first time using resource limits settings with new variables. I can confirm its in env config map. I can confirm all of airbyte-server/airbyte-scheduler/airbyte-worker containers have this injected and available. Seems I am missing something maybe? Anyway to debug more?

Is this your first time deploying Airbyte yes OS Version / Instance Mac Memory / Disk 32G/1T Deployment docker Airbyte Version v0.35.28-alpha Source name/version Postgres 9.6.13 Destination name/version Postgres 13 Step Setting up the connection Description Hi all - i’m trying to setup connection between one postgres and another and my source postgres only has views (no tables - it’s long story) im reading the postgres docs and it says it supports replicating views but under the “Select the data you want to sync” there are no rows so doesnt seem to be picking anything up even though it can successfully connect to the database and when i connect as well in a different program I can see the views / query them. Any idea what might be up?

Hello, I am new to airbyte and I am trying to deploy it locally following these instructions, but I get the following error (I am on Ubuntu 18.04) :

~/airbyte$ sudo docker-compose up
WARN[0000] The "RUN_DATABASE_MIGRATION_ON_STARTUP" variable is not set. Defaulting to a blank string. 
WARN[0000] The "SECRET_PERSISTENCE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "WORKER_ENVIRONMENT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "SECRET_PERSISTENCE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "WORKER_ENVIRONMENT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "SECRET_PERSISTENCE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "WORKER_ENVIRONMENT" variable is not set. Defaulting to a blank string. 
[+] Running 0/8
 ⠙ db Pulling                                                                                                                                                                                         
 ⠙ server Pulling                                                                                                                                                                                     
 ⠙ airbyte-temporal Pulling                                                                                                                                                                           
 ⠙ webapp Pulling                                                                                                                                                                                     
 ⠙ scheduler Pulling                                                                                                                                                                                  
 ⠙ init Pulling                                                                                                                                                                                       
 ⠙ worker Pulling                                                                                                                                                                                     
 ⠙ bootloader Pulling                                                                                                                                                                                 
error getting credentials - err: exit status 1, out: `Error calling StartServiceByName for org.freedesktop.secrets: Timeout was reached`

Can you please help me?

is there anything to fix this

Hi Team. I am very new to Airbyte and was exploring it as an option for our CDC needs. I was able to complete the setup as mentioned in tutorial and docs, however I have some questions regarding the same: 1. I do not see the Incremental de-duped history Sync mode available for hands on testing. Is that not available now? 2. While configuring connections with MySQL v.5.7, I noticed Basic Normalisation syncs fail. On reading up, found this open issue tracking this (https://github.com/airbytehq/airbyte/issues/4989). Do we have any timeline for resolving this? 3. I have read in several blogs how Airbyte is built on top of Embedded Debezium without going the Kafka way. Can you point me to any relevant docs exploring HA in Airbyte and related failure scenarios, if any. Thanks in advance 🙂

Hello Team, Is this your first time deploying Airbyte: NO OS Version / Instance: E Memory / Disk: 30 GB Airbyte Version: 0.35.35 Step: Export Airbyte Configuration and Data Archive OR Import Airbyte Configuration is not working properly Description: Migration script for Export and Import not working properly. Error when exporting and then importing the same configuration and data archive. Works alright before

{
  "status": "failed",
  "reason": "io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. \nErrors: $.id: is missing but it is required, $.catalog: is missing but it is required, $.catalogHash: is missing but it is required \nSchema: \n{\n  \"$schema\" : \"<http://json-schema.org/draft-07/schema#>\",\n  \"$id\" : \"<https://github.com/airbytehq/airbyte/blob/master/airbyte-config/models/src/main/resources/types/AttemptFailureSummary.yaml>\",\n  \"title\" : \"ActorCatalog\",\n  \"description\" : \"Catalog of an actor.\",\n  \"type\" : \"object\",\n  \"additionalProperties\" : false,\n  \"required\" : [ \"id\", \"catalog\", \"catalogHash\" ],\n  \"properties\" : {\n    \"id\" : {\n      \"type\" : \"string\",\n      \"format\" : \"uuid\"\n    },\n    \"catalog\" : {\n      \"type\" : \"object\",\n      \"existingJavaType\" : \"com.fasterxml.jackson.databind.JsonNode\"\n    },\n    \"catalogHash\" : {\n      \"type\" : \"string\"\n    }\n  }\n}"
}

Is the anyway to change

SYNC_JOB_MAX_TIMEOUT_DAYS

while Airbyte is running?

Have been hitting this error for the first time these last few days. Anyone seen it before? Happens when I try to access Airbyte via localhost. Happens each time I set up the Square -> Postgres connector. Haven’t found much online!

Hello everyone, I have deployed Airbyte locally and I am trying to connect to mongodb, but I get this error!

Could not connect with provided configuration. Error: Timed out after 30000 ms while waiting for a server that matches com.mongodb.client.internal.MongoClientDelegate$1@282308c3. Client view of cluster state is {type=REPLICA_SET, servers=[{address=172.31.23.101:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake}, caused by {java.io.EOFException: SSL peer shut down incorrectly}}, {address=172.31.23.102:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake}, caused by {java.io.EOFException: SSL peer shut down incorrectly}}, {address=172.31.23.103:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake}, caused by {java.io.EOFException: SSL peer shut down incorrectly}}]

Is this your first time deploying Airbyte: no OS Version / Instance: AWS EC2 t2.large Deployment: Docker Airbyte Version: 0.35.4-alpha Is this your first time deploying Airbyte: no OS Version / Instance: AWS EC2 t2.large Deployment: Docker Airbyte Version: 0.35.4-alpha Source: postgres 0.4.7 Destination : BigQuery 0.6.8 Description: Hi, My Postgres to BigQuery connection is failing since this morning. Attached are the logs, I can't extract any valuable insights from it. Are you able to ? Note that I upgraded the postgres source from 0.4.6 this morning, could it be the cause of this issue ? If yes, how can I downgrade it ? Thanks

Hi Team, anyone received this error while running the command on a windows machine: pip install -r requirements.txt

hi guys I wanna install airbyte as a dependency to my chart, [3:20 PM] what should I put in the repository and version and even name

#  - name: airbyte
#    version: "?.?.?"
#    repository: "https://???-???.???.???.???/"
#    condition: airbyte.enabled

Is this your first time deploying Airbyte: No OS Version / Instance: Windows 11 Memory / Disk: 8GB RAM Deployment: Docker Airbyte Version: 0.35.2-alpha Source name/version: airbyte/source-mongodb 0.1.11 Destination name/version: airbyte/destination-bigquery 0.6.8 Step: I am experiencing a strange behaviour in my connection, where Airbyte changes the field name and type from

amountPaid

to

amountPaid_airbyte_transform

. the new field is all NULL but there are values in the field. Please I need help to resolve this. Thank you.

Is this your first time deploying Airbyte: No OS Version / Instance: Mac Deployment: Docker Airbyte Version: 0.33.12-alpha Source name/version: Google Ads Description: I am trying to setup a query for the custom GAQL query. However when I enter the valid query below there is no output at my destination. Although, the Airbyte default tables are coming through as expected. What I am doing wrong? The problem is on the query? How I know if the new table using this feature worked? Do this must create a table on

connection settings

? Thanks for the support!

SELECT
campaign.id,
campaign.name,
metrics.clicks,
metrics.impressions,
metrics.conversions,
metrics.cost_micros,
metrics.ctr,
metrics.bounce_rate,
metrics.active_view_cpm,
metrics.active_view_ctr,
metrics.active_view_impressions,
metrics.interactions,
metrics.video_views
FROM campaign

Is this your first time deploying Airbyte: No OS Version / Instance: EC2 m5.2xlarge, Amazon Linux 2 Memory / Disk: 32Gb / 500 GB EBS Deployment: Docker Airbyte Version: 0.35.32-alpha Source name/version: BigQuery 0.1.6 Destination name/version: Local CSV 0.2.9 Step: Try ➡️ failure to sync data Description: I'm trying to sync data from BigQuery and it fails.

Is this your first time deploying Airbyte: Yes OS Version / Instance: Ubuntu 20.04 Memory / Disk: 16Gb / 1Tb SSD Deployment: Docker Airbyte Version: 0.35.32-alpha Source name/version: - Destination name/version: - Step: Starting docker-compose.yaml Description: Cannot start docker-compose. Seems like issue with airbyte/temporal - perhaps file permissions. Excerpt from docker logs airbyte-temporal

touch: CONTAINER_ALREADY_STARTED_PLACEHOLDER: Permission denied
...
ERROR   Unable to create SQL database.  {"error": "pq: database \"temporal\" already exists", "logging-call-at": "handler.go:97"}

I'm running docker-compose as the same user that the repository files are owned by, in one of /home subdirs. Thanks in advance :)

hi guys, trying to deploy airbyte with helm with existing minio instance, and I get in the ui the following error

Internal Server Error: Error creating S3 log client to Minio

and in the worker logs:

Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Malformed escape pair at index 21: <http://minio-service>:%!d(<nil>)

or if I insert 9000:

Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Malformed escape pair at index 21: <http://minio-service>:%!d(float64=9000)

as you can see it is not parsing well the minio port, even when I insert:

9000

or

'9000'

or

"9000"

or nothing which is this error. here’s my values.yaml file: ``````