Hi all - I am new to AirByte and looking for some help. I've searched the troubleshooting forum but didn't have any luck. I am using AirByte cloud and trying to connect to Google Cloud SQL (MySQL 8.0) When I try to connect using my GCP public IP as the host, I get the following error:

Could not connect with provided configuration. Error: Cannot create PoolableConnectionFactory (Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.)

Any general support would be appreciated, but here are some specific questions: • Do I need to add an authorized network IP address to my Google Cloud SQL instance? Or use the Cloud SQL Auth proxy? This issue makes me thing the current MySQL connector does not support connecting to a Google Cloud SQL instance • Do I need to use an SSH tunnel? This post makes me thing an SSH tunnel to a MySQL DB on Google Cloud SQL is not possible

Hi there, does someone has an example of a custom python source modifying dynamically the json_schema per record? (as some results have different fields) Many thanks!

Hello Team, Hello World Excited to be here, getting a error on my first walkthrough on the exchange rate api

Good day. I am trying to follow along with this documentation in order to progress to custom transformations in some cases where basic normalization is not working correctly for some existing connections. I would like to extract the generated SQL, but I have been unable to locate the referenced models folder on the server or worker pod (the installation I am working with is on kubernetes) and cannot exec into the normalization pod because it is shut down after the job completes. Further, I cannot locate the destination_catalog.json file referred to though the parent folder from which it is being copied is present on the worker pod (but the folder is empty). Could you advise on where I am going wrong looking for the model assets referred to in the documentation? The relevant portion of the log file is below. Thanks for any assistance.

2022-04-28 16:53:25 INFO i.a.w.t.TemporalAttemptExecution(get):107 - Cloud storage job log path: /workspace/145/0/logs.log
2022-04-28 16:53:25 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.35.42-alpha
2022-04-28 16:53:25 INFO i.a.w.DefaultNormalizationWorker(run):46 - Running normalization.
2022-04-28 16:53:25 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):122 - Running with normalization version: airbyte/normalization:0.1.69
2022-04-28 16:53:25 INFO i.a.w.p.KubeProcessFactory(create):111 - Attempting to start pod = normalization-sync-145-0-mkkvy
2022-04-28 16:53:25 INFO i.a.w.p.KubeProcessFactory(create):114 - normalization-sync-145-0-mkkvy stdoutLocalPort = 9016
2022-04-28 16:53:25 INFO i.a.w.p.KubeProcessFactory(create):117 - normalization-sync-145-0-mkkvy stderrLocalPort = 9017
2022-04-28 16:53:25 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$8):550 - Creating stdout socket server...
2022-04-28 16:53:25 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):568 - Creating stderr socket server...
2022-04-28 16:53:25 INFO i.a.w.p.KubePodProcess(<init>):507 - Creating pod...
2022-04-28 16:53:25 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):307 - Waiting for init container to be ready before copying files...
2022-04-28 16:53:25 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):320 - Init container present..
2022-04-28 16:53:26 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):323 - Init container ready..
2022-04-28 16:53:26 INFO i.a.w.p.KubePodProcess(<init>):513 - Copying files...
2022-04-28 16:53:26 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):257 - Uploading file: destination_config.json
2022-04-28 16:53:26 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - kubectl cp /tmp/f6f49d48-e353-44c6-a6b0-035ea96affe6/destination_config.json airbyte-2/normalization-sync-145-0-mkkvy:/config/destination_config.json -c init
2022-04-28 16:53:26 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):268 - Waiting for kubectl cp to complete
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):282 - kubectl cp complete, closing process
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):257 - Uploading file: destination_catalog.json
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - kubectl cp /tmp/3d2f578e-42f5-4ea7-aace-8afd7d09f24a/destination_catalog.json airbyte-2/normalization-sync-145-0-mkkvy:/config/destination_catalog.json -c init
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):268 - Waiting for kubectl cp to complete
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):282 - kubectl cp complete, closing process
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):257 - Uploading file: FINISHED_UPLOADING
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - kubectl cp /tmp/0bc14080-2fa2-456d-87ed-a6dc9cc42f69/FINISHED_UPLOADING airbyte-2/normalization-sync-145-0-mkkvy:/config/FINISHED_UPLOADING -c init
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):268 - Waiting for kubectl cp to complete
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):282 - kubectl cp complete, closing process
2022-04-28 16:53:27 INFO i.a.w.p.KubePodProcess(<init>):516 - Waiting until pod is ready...
2022-04-28 16:53:47 normalization > Running: transform-config --config destination_config.json --integration-type redshift --out /config
2022-04-28 16:53:47 normalization > Namespace(config='destination_config.json', integration_type=<DestinationType.redshift: 'redshift'>, out='/config')
2022-04-28 16:53:47 normalization > transform_redshift
2022-04-28 16:53:47 normalization > Running: transform-catalog --integration-type redshift --profile-config-dir /config --catalog destination_catalog.json --out /config/models/generated/ --json-column _airbyte_data
2022-04-28 16:53:47 normalization > Processing destination_catalog.json...
2022-04-28 16:53:47 normalization >   Generating airbyte_ctes/quarantine/covid_epidemiology_ab1.sql from covid_epidemiology
2022-04-28 16:53:47 normalization >   Generating airbyte_ctes/quarantine/covid_epidemiology_ab2.sql from covid_epidemiology
2022-04-28 16:53:47 normalization >   Generating airbyte_ctes/quarantine/covid_epidemiology_ab3.sql from covid_epidemiology
2022-04-28 16:53:47 normalization >   Generating airbyte_incremental/quarantine/covid_epidemiology.sql from covid_epidemiology
2022-04-28 16:53:47 normalization > detected no config file for ssh, assuming ssh is off.
2022-04-28 16:53:48 normalization > Running with dbt=0.21.1
2022-04-28 16:53:48 normalization > Unable to do partial parsing because ../build/partial_parse.msgpack not found
2022-04-28 16:53:49 normalization > WARNING: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
2022-04-28 16:53:49 normalization > There are 2 unused configuration paths:
2022-04-28 16:53:49 normalization > - models.airbyte_utils.generated.airbyte_views
2022-04-28 16:53:49 normalization > - models.airbyte_utils.generated.airbyte_tables
2022-04-28 16:53:49 normalization > 
2022-04-28 16:53:49 normalization > Found 4 models, 0 tests, 0 snapshots, 0 analyses, 520 macros, 0 operations, 0 seed files, 1 source, 0 exposures
2022-04-28 16:53:49 normalization > 
2022-04-28 16:53:49 normalization > 16:53:49 | Concurrency: 4 threads (target='prod')
2022-04-28 16:53:49 normalization > 16:53:49 | 
2022-04-28 16:53:50 normalization > 16:53:50 | 1 of 1 START incremental model quarantine.covid_epidemiology................................................. [RUN]
2022-04-28 16:53:50 normalization > 16:53:50 + "airbyte_incoming".quarantine."covid_epidemiology"._airbyte_ab_id does not exist yet. The table will be created or rebuilt with dbt.full_refresh
2022-04-28 16:53:45 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$8):559 - Setting stdout...
2022-04-28 16:53:46 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):571 - Setting stderr...
2022-04-28 16:53:47 INFO i.a.w.p.KubePodProcess(<init>):531 - Reading pod IP...
2022-04-28 16:53:47 INFO i.a.w.p.KubePodProcess(<init>):533 - Pod IP: 192.168.56.129
2022-04-28 16:53:47 INFO i.a.w.p.KubePodProcess(<init>):540 - Using null stdin output stream...
2022-04-28 16:54:02 INFO i.a.w.p.KubePodProcess(getReturnCode):733 - Exit code for pod normalization-sync-145-0-mkkvy is 0
2022-04-28 16:54:02 INFO i.a.w.p.KubePodProcess(exitValue):748 - Closed all resources for pod normalization-sync-145-0-mkkvy
2022-04-28 16:54:02 INFO i.a.w.p.KubePodProcess(exitValue):748 - Closed all resources for pod normalization-sync-145-0-mkkvy
2022-04-28 16:54:02 INFO i.a.w.p.KubePodProcess(exitValue):748 - Closed all resources for pod normalization-sync-145-0-mkkvy
2022-04-28 16:54:02 INFO i.a.w.p.KubePodProcess(exitValue):748 - Closed all resources for pod normalization-sync-145-0-mkkvy
2022-04-28 16:54:02 INFO i.a.w.p.KubePodProcess(exitValue):748 - Closed all resources for pod normalization-sync-145-0-mkkvy
2022-04-28 16:54:02 INFO i.a.w.p.KubePodProcess(exitValue):748 - Closed all resources for pod normalization-sync-145-0-mkkvy
2022-04-28 16:54:02 INFO i.a.w.DefaultNormalizationWorker(run):69 - Normalization executed in 37 seconds.
2022-04-28 16:54:02 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling...
2022-04-28 16:54:02 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):235 - Stopping temporal heartbeating...
2022-04-28 16:54:01 normalization > 16:54:01 | 1 of 1 OK created incremental model quarantine.covid_epidemiology............................................ [SELECT in 11.18s]
2022-04-28 16:54:01 normalization > 16:54:01 | 
2022-04-28 16:54:01 normalization > 16:54:01 | Finished running 1 incremental model in 11.73s.
2022-04-28 16:54:01 normalization > 
2022-04-28 16:54:01 normalization > Completed successfully
2022-04-28 16:54:01 normalization > 
2022-04-28 16:54:01 normalization > Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

Hi team, I have deployed airbyte on a kubernetes cluster. I am trying to externalize the database in the postgres as per these instructions. But, the airbyte server pod errors out without any error. The last logs which I see on the pod are:

2022-04-28 22:57:37 INFO i.a.s.ServerApp(assertDatabasesReady):132 - Checking configs database flyway migration version..

2022-04-28 22:57:37 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:57:37 WARN i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):74 - Waiting for database to become available...

2022-04-28 22:57:37 INFO i.a.d.i.BaseDatabaseInstance(lambda$isDatabaseConnected$5):127 - Testing airbyte configs database connection...

2022-04-28 22:57:39 INFO i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):99 - Database available!

2022-04-28 22:57:39 INFO i.a.d.Databases(createPostgresDatabaseWithRetry):57 - Database available!

2022-04-28 22:57:39 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:57:45 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:57:45 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:57:51 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:57:51 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:57:57 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:57:57 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:58:03 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:58:03 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:58:09 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:58:09 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:58:15 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:58:15 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:58:21 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:58:21 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:58:27 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:58:27 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:58:33 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:58:33 INFO i.a.d.i.c.ConfigsDatabaseInstance(lambda$isInitialized$4):65 - The airbyte configs database is not initialized; initializing it with schema

2022-04-28 22:58:39 INFO i.a.d.i.MinimumFlywayMigrationVersionCheck(assertDatabase):42 - Waiting for database...

2022-04-28 22:58:39 ERROR i.a.s.ServerApp(main):249 - Server failed

java.lang.RuntimeException: Timeout while connecting to the database..

at io.airbyte.db.instance.MinimumFlywayMigrationVersionCheck.assertDatabase(MinimumFlywayMigrationVersionCheck.java:45) ~[io.airbyte.airbyte-db-lib-0.35.65-alpha.jar:?]

at io.airbyte.server.ServerApp.assertDatabasesReady(ServerApp.java:133) ~[io.airbyte-airbyte-server-0.35.65-alpha.jar:?]

at io.airbyte.server.ServerApp.getServer(ServerApp.java:160) ~[io.airbyte-airbyte-server-0.35.65-alpha.jar:?]

at io.airbyte.server.ServerApp.main(ServerApp.java:247) [io.airbyte-airbyte-server-0.35.65-alpha.jar:?]

Could anyone please advise what could be the issue or point me in the right direction? The database I have created is an Azure managed postgresql instance and has an empty database available. After some time,

Hi, I am doing a PoC for airbyte with mysql and redshift source. And wanted to know if I am creating a connection for first time and only want to sync partial data. Lets say data of 1 day and not whole table, is there a way to achieve this? Also lets say I run the connection manually can this be achieved to sync only partial data

Hi everyone, new to airbyte. Is there a way to load data from Zoho suite of tools?

Hi Airbyters! Is there a way to setup a specific time to run a scheduled sync?

Hi guys

Anyone integrated gitlab connector here please? I have got this integrated but do not see full data in streams.. Only 20% data is replicated to destination

Can you please help me out ?

I'm trying to set up a Pipedrive source (which should be available, right?) but I don't see it in the list of sources in the dashboard... what am I missing?

Hello! Is it possible to set up airbyte with GA (UA tags) or only GA4 is supported ? I am getting this error 2022-04-29 202725 INFO i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):366 - Waiting for kubectl cp to complete 2022-04-29 202725 INFO i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):373 - kubectl cp complete, closing process 2022-04-29 202725 INFO i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):355 - Uploading file: FINISHED_UPLOADING 2022-04-29 202725 INFO i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):363 - kubectl cp /tmp/2905c0b5-269e-4d8c-bccf-7893261354f5/FINISHED_UPLOADING jobs/orchestrator-repl-job-147104-attempt-2:/config/FINISHED_UPLOADING -c main 2022-04-29 202725 INFO i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):366 - Waiting for kubectl cp to complete 2022-04-29 202725 INFO i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):373 - kubectl cp complete, closing process 2022-04-29 202753 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):267 - Stopping temporal heartbeating... 2022-04-29 202753 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-04-29 202753 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):155 - sync summary: io.airbyte.config.StandardSyncOutput@13fc3c94[standardSyncSummary=io.airbyte.config.StandardSyncSummary@1aca6077[status=failed,recordsSynced=0,bytesSynced=0,startTime=1651264056604,endTime=1651264073401,totalStats=io.airbyte.config.SyncStats@70dcf9e5[recordsEmitted=0,bytesEmitted=0,stateMessagesEmitted=0,recordsCommitted=0],streamStats=[]],normalizationSummary=<null>,state=<null>,outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@2133b72f[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@53592701[stream=io.airbyte.protocol.models.AirbyteStream@1ad92cc5[name=daily_active_users,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_1dayUsers”{“type”[“null”,“integer”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@d5dfdaa[stream=io.airbyte.protocol.models.AirbyteStream@31ebb9e1[name=devices,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_users”{“type”[“null”,“integer”]},“ga_browser”{“type”[“string”]},“ga_exitRate”{“type”[“null”,“number”]},“ga_newUsers”{“type”[“null”,“integer”]},“ga_sessions”{“type”[“null”,“integer”]},“ga_pageviews”{“type”[“null”,“integer”]},“ga_bounceRate”{“type”[“null”,“number”]},“ga_avgTimeOnPage”{“type”[“null”,“number”]},“ga_deviceCategory”{“type”[“string”]},“ga_operatingSystem”{“type”[“string”]},“ga_sessionsPerUser”{“type”[“null”,“number”]},“ga_avgSessionDuration”{“type”[“null”,“number”]},“ga_pageviewsPerSession”{“type”[“null”,“number”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@2b90f1f2[stream=io.airbyte.protocol.models.AirbyteStream@787c4f41[name=four_weekly_active_users,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_28dayUsers”{“type”[“null”,“integer”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@1a2e5d49[stream=io.airbyte.protocol.models.AirbyteStream@7666db5c[name=locations,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga city”{“type”[“string”]},“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_metro”{“type”[“string”]},“ga_users”{“type”[“null”,“integer”]},“ga_region”{“type”[“string”]},“ga_country”{“type”[“string”]},“ga_exitRate”{“type”[“null”,“number”]},“ga_newUsers”{“type”[“null”,“integer”]},“ga_sessions”{“type”[“null”,“integer”]},“ga_continent”{“type”[“string”]},“ga_pageviews”{“type”[“null”,“integer”]},“ga_bounceRate”{“type”[“null”,“number”]},“ga_subContinent”{“type”[“string”]},“ga_avgTimeOnPage”{“type”[“null”,“number”]},“ga_sessionsPerUser”{“type”[“null”,“number”]},“ga_avgSessionDuration”{“type”[“null”,“number”]},“ga_pageviewsPerSession”{“type”[“null”,“number”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@5ad5bb03[stream=io.airbyte.protocol.models.AirbyteStream@4049f15f[name=monthly_active_users,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_30dayUsers”{“type”[“null”,“integer”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@2865380a[stream=io.airbyte.protocol.models.AirbyteStream@6819ba61[name=pages,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_exits”{“type”[“null”,“integer”]},“ga_exitRate”{“type”[“null”,“number”]},“ga_hostname”{“type”[“string”]},“ga_pagePath”{“type”[“string”]},“ga_entrances”{“type”[“null”,“integer”]},“ga_pageviews”{“type”[“null”,“integer”]},“ga_bounceRate”{“type”[“null”,“number”]},“ga_entranceRate”{“type”[“null”,“number”]},“ga_avgTimeOnPage”{“type”[“null”,“number”]},“ga_uniquePageviews”{“type”[“null”,“integer”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@1fe309c2[stream=io.airbyte.protocol.models.AirbyteStream@10afc068[name=two_weekly_active_users,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_14dayUsers”{“type”[“null”,“integer”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@110b1f1f[stream=io.airbyte.protocol.models.AirbyteStream@34e475ed[name=website_overview,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_users”{“type”[“null”,“integer”]},“ga_exitRate”{“type”[“null”,“number”]},“ga_newUsers”{“type”[“null”,“integer”]},“ga_sessions”{“type”[“null”,“integer”]},“ga_pageviews”{“type”[“null”,“integer”]},“ga_bounceRate”{“type”[“null”,“number”]},“ga_avgTimeOnPage”{“type”[“null”,“number”]},“ga_sessionsPerUser”{“type”[“null”,“number”]},“ga_avgSessionDuration”{“type”[“null”,“number”]},“ga_pageviewsPerSession”{“type”[“null”,“number”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@334ed7aa[stream=io.airbyte.protocol.models.AirbyteStream@5d0ced2b[name=weekly_active_users,jsonSchema={“type”:[“null”,“object”],“$schema”:“http://json-schema.org/draft-07/schema#”,“properties”{“ga date”{“type”[“string”],“format”“date”},“view_id”{“type”[“string”]},“ga_7dayUsers”{“type”[“null”,“integer”]}},“additionalProperties”:false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[ga_date],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[ga_date],destinationSyncMode=append,primaryKey=[],additionalProperties={}]],additionalProperties={}],failures=[io.airbyte.config.FailureReason@1436f272[failureOrigin=source,failureType=<null>,internalMessage=io.airbyte.workers.DefaultReplicationWorker$SourceException: Source process exited with non-zero exit code 1,externalMessage=Something went wrong within the source connector,metadata=io.airbyte.config.Metadata@3eab604[additionalProperties={attemptNumber=2, jobId=147104}],stacktrace=java.util.concurrent.CompletionException: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source process exited with non-zero exit code 1 at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source process exited with non-zero exit code 1 at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$5(DefaultReplicationWorker.java:312) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more ,retryable=<null>,timestamp=1651264071712]]] 2022-04-29 202753 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):236 - Stopping temporal heartbeating...

Hi all! I'm experimenting with Airbyte and trying to setup a GCP CloudSQL instance as a source. It seems like my options are, for the open source offering (assuming the CloudSQL instance has a private IP and I'm deploying Airbyte in the same VPC): 1. mount IAM credentials & setup cloud_sql_proxy inside one of the containers (worker container?), and tunnel connections through the proxy 2. add the Airbyte server IP as an "authorized IP" to the CloudSQL instance For the cloud offering, assuming the CloudSQL instance has a public IP: 1. Add the Airbyte cloud IPs as an "authorized IP" to CloudSQL instance. The certificate auth option looks like it has yet to be implemented. Does this sound right? Are there any recommended approaches here?

Hey everyone! I am facing a very basic issue and I am sure the solution is something simple. I probably need a bit of hand holding to get started off, and then I would be able to work my way through. I am not able to setup (Bigquery -> MongoDB ) connection. I keep getting the following error:

Failed to fetch schema. Please try again

I tried looking in the troubleshooting forum but could not find much. Thanks in advance!

horizontal scaling question... I understand there exists a setting to control max workers (syncs). But how about parallel / scale out execution for a single connection with huge amount of tables? Eg a k8s deployment with a single connection should ideally be split across many available nods in the pool. Current situation is that you should split the load manually across multiple connections, being careful to have distinct streams. Not nice, and should be a setting to lessen the maintainance... Ideally there should be a que of streams and auto assigned to available resources

Hi everyone. I'm not seeing Amazon Ads under sources. I saw it on the site before I signed up. Am I doing something wrong?

Hello Airbyte Team! Quick question, which sync mode acts like a CDC?

Hello all ! Would you know if there is an easy way to remove old images version of connectors ? My hosted Vm crashed because of the memory disk being filled by depreciated connectors images versions. (All fine now) I see that the request as already been made https://github.com/airbytehq/airbyte/issues/2322 Would you know if It has been analysed?

Hello Airbyte team. I'm trying to set-up Facebook Marketing connector on open-source version. While requesting rate increase limit to Ads Management Standard Access I get this notification. Any idea how can I get this 1500 Marketing API calls?

Silly noob question here... I'm moving data from one Snowflake instance to another, but when I'm creating the connection it defaults to selecting all tables. I can't for the life of me figure out how to set it to select none so that I can turn on the 5 or 6 tables that I want without having to deselect the hundreds that I don't.

Hey everyone. I am trying to create an Amazon ADS source but I am receiving the following error:

Have anyone seen this error before?

Hey, team. I'm testing AirByte for PG > RedShift sync (CDC, wall2json). both source and target work perfectly. created a connection and mapped 2 tables with "Incremental | Deduped + History" method. primary key = 1 int field in PG • first sync goes well • if I modify record on source side - see new record appearing in ***_sdc table • record in destination table gets updated • new record in source - also goes through BUT • when delete in source • after sync see deletion record in ***_sdc table in destination • but record in destination table just remains. even after 2-3 syncs. what might be wrong? why deletion event reaches SDC table in destination, but not applied to destination table itself? thanks in advance for your help

Hello everyone! I am facing a issue. When I am using a python script which uses airbyte API to create approx 50 connections with 12 hours frequency many of the time after creating 5-6 connections the script gets stuck and the API stops responding. The 5-6 connections which have been created start their sync but the script gets stuck as API doesn't respond and not able to create more connections.

Hello Airbyte Team! Where can we put some request? I'd like to request a connector to Bubble.io's databases please :)

Hello Airbyte team! Recently we are noticing that the JSON array to get the custom reports from Google Analytics is not working in Airbyte. We are only able to sync the supported streams. Is this part of a new release?

Here is a screenshot of the error. Kindly let me know, if it is something on the Airbyte side.

Hello Airbyte Team! I don't know what time my jobs will run again? 😅 Please help!

Hello guys ! I've made some Data Migration from Redshift to Snowflake but I've noticed that the DDL isn't transferred. Not even the primary keys. Is there any solution to that or I really need to transfer the DDL info using a third-party tool ? Thank you very much in advance!