Hi everyone, I am new to Airbyte and using free open-source Airbyte. It looks I need to keep the docker running all the time in order to sync the data from the source to the destination, is it correct?

Hi everyOne, I´m trying to use external database to the config. But even changing parameters at .env file airbyte is not using new db. Any ideas??

Hi all, can someone explain to me the reason for the different versioning of the latest releases across the Airbyte ecosystem? The latest release of Octavia CLI via docker hub is 0.42.1, the latest release of the airbyte app platform is 0.44, yet the latest release of the helm charts is 0.45.11. Why are they all different?

Hi, is it normal to have all columns as "TEXT" with max 1024 chars for BigQuery --> MySQL? There is no error in the log. Airbyte 0.40.22 MySQL 0.1.20

Hi Team, I am trying to deploy airbyte in EKS cluster using helm chart latest version, I am using external postgres RDS instance instead of airbyte-db hence kept the postgres.enable to false in values.yaml file, when deploying I am getting issue with airbyte-temporal service its giving me error

Error: sql schema version compatibility check failed: unable to read DB schema version keyspace/database: temporal error: pq: relation "schema_version" does not exist

please note that I had to request to create temporal db manually and grant all permission due to limited access to RDS Instance

Hi everyone! I have a question regarding Airbyte OSS deployment on AWS: is it possible to use an encrypted external database?

This message was deleted.

Hi, I want to deploy airbyte to eks with our own helm. I can not use airbyte's prepared charts and have to build my own, but I am not succeeding to understand if I have to configure all components that I found here https://github.com/airbytehq/airbyte-platform/tree/main/charts

Getting this error when trying to connect to databricks, can anyone help?

State code: 08S01; Error code: 500593; Message: [Databricks][DatabricksJDBCDriver](500593) Communication link failure. Failed to connect to server. Reason: HTTP Response code: 403, Error message: Unknown.

I've managed to create a connector so far and now I want to know how I can use DBT to transform this data. Could I get some guidance on this

HubSpot connector seems to fail to often to bring data, can someone help ? @Marcos Marx (Airbyte)

Hey is there an issue with the site? I cannot login, and if I try in a fresh window it won't even let me sign up for a new account? I get this message every time... trying different browsers and all the same!

Airbyte open source edition and we keep getting this error: 090222 (42601): View definition too large.

Hello 👋 , Today’s office hour is going to happen in 10 minutes here you can enter to chat about features, issues or other topics related to Airbyte.

Hi everyone, Do we have any option to do basic transformation at the time of data extraction or in-between before loading it into destination in Airbyte? Something similar to [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html) feature of Meltano? Thanks in advance! 🙏

I'm getting the following error in the docker-compose logs, resulting in a bad gateway 502 when I try to reach Airbyte. Anyone an idea what's causing this? A previous instance was running fine but I had to restart my vm and now i'm getting this

airbyte-webapp                    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
airbyte-webapp                    | /docker-entrypoint.sh: Configuration complete; ready for start up
airbyte-webapp                    | 2023/04/18 14:47:11 [emerg] 1#1: unknown "airbyte_version" variable
airbyte-webapp                    | nginx: [emerg] unknown "airbyte_version" variable
airbyte-webapp exited with code 0

Hi team, could anyone please let me know on how we can get rid of this error

2023-04-18 14:52:23 INFO i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed. 
errors: $: null found, string expected
2023-04-18 14:52:23 INFO i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed. 
errors: $: null found, string expected

Hello I have Minio S3 server running with my own CA signed Certificate. When I am trying to access the S3 endpoint I am getting following error in the worker.

2023-04-18 14:42:18 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):313 - ERROR i.a.i.d.s.BaseS3Destination(check):52 Exception attempting to access the S3 bucket: com.amazonaws.SdkClientException: Unable to execute HTTP request: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1207) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1153) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5437) ~[aws-java-sdk-s3-1.12.6.jar:?]
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5384) ~[aws-java-sdk-s3-1.12.6.jar:?]
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5378) ~[aws-java-sdk-s3-1.12.6.jar:?]
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:927) ~[aws-java-sdk-s3-1.12.6.jar:?]
	at io.airbyte.integrations.destination.s3.S3BaseChecks.testIAMUserHasListObjectPermission(S3BaseChecks.java:133) ~[io.airbyte.airbyte-integrations.bases-base-java-s3-0.42.0.jar:?]
	at io.airbyte.integrations.destination.s3.BaseS3Destination.check(BaseS3Destination.java:46) ~[io.airbyte.airbyte-integrations.bases-base-java-s3-0.42.0.jar:?]
	at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:126) ~[io.airbyte.airbyte-integrations.bases-base-java-0.42.0.jar:?]
	at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:101) ~[io.airbyte.airbyte-integrations.bases-base-java-0.42.0.jar:?]
	at io.airbyte.integrations.base.adaptive.AdaptiveDestinationRunner$Runner.run(AdaptiveDestinationRunner.java:87) ~[io.airbyte.airbyte-integrations.bases-base-java-0.42.0.jar:?]
	at io.airbyte.integrations.destination.s3.S3DestinationRunner.main(S3DestinationRunner.java:15) ~[io.airbyte.airbyte-integrations.connectors-destination-s3-0.42.0.jar:?]
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:371) ~[?:?]
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:314) ~[?:?]
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:309) ~[?:?]
	at sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1357) ~[?:?]
	at sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1232) ~[?:?]
	at sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1175) ~[?:?]
	at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]
	at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:480) ~[?:?]
	at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:458) ~[?:?]
	at sun.security.ssl.TransportContext.dispatch(TransportContext.java:201) ~[?:?]
	at sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1505) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1420) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426) ~[?:?]
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) ~[httpclient-4.5.14.jar:4.5.14]
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) ~[httpclient-4.5.14.jar:4.5.14]
	at com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:142) ~[aws-java-sdk-core-1.12.6.jar:?]
	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.14.jar:4.5.14]
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[httpclient-4.5.14.jar:4.5.14]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
	at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.conn.$Proxy28.connect(Unknown Source) ~[?:?]
	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.14.jar:4.5.14]
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.14.jar:4.5.14]
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.14.jar:4.5.14]
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.14.jar:4.5.14]
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.14.jar:4.5.14]
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.14.jar:4.5.14]
	at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1331) ~[aws-java-sdk-core-1.12.6.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145) ~[aws-java-sdk-core-1.12.6.jar:?]
	... 17 more

I have the CA added in the worker container by following commands

keytool -importcert -keystore /usr/lib/jvm/java-19-amazon-corretto/lib/security/cacerts -storepass changeit -file CA_cert.crt -alias "ca-root"

to the with os

update-ca-trust

My curl command is not giving any cert error to minio endpoint from the worker container. need some help to fix the cert error,

Connector UI related question While working on Connector UI, I need a field that is multiline string and able to expand the multiline field (like a text box) so that I can see the whole field at glance. Is there an option for that?

Environment • Airbyte version: Airbyte Cloud • Source Connector and version: Salesforce 2.0.9, as of today • Destination Connector and version: Snowflake 0.4.61, as of today • Step where error happened: reading string columns which appear like very long numbers Current Behavior We have custom columns in our Salesforce objects (who doesn't?) and sometimes these are held in string columns with values like

821580693639061174

. Unfortunately the Salesforce connector is fooled into thinking these are bigints or similar, and what ends up in our warehouse is

8.215806936390611e+17

. Not only is it wrong to assume this is a number when the schema explicitly calls out the column as a string (see screenshot), but worse yet some of the characters are truncated from the end when the string is longer than 17 characters, therefore information is lost. This problem happens on both full syncs and incremental/de-dupe syncs.

Hi I am new here. We are using the open source Airbyte newest version, deployed on K8s. I have a question about customize DBT transformation in airbyte, somehow my transformation always fails without a clear error. The dbt repository is a public github repository. I’m wondering if it is because I am not specifying profiles.yml file in the command or in the repository?

2023-04-18 19:59:40 [32mINFO[m i.a.w.n.DefaultNormalizationRunner(close):190 - Terminating normalization process...
2023-04-18 19:59:40 [32mINFO[m i.a.w.n.DefaultNormalizationRunner(close):203 - Normalization process successfully terminated.
2023-04-18 19:59:40 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$5):198 - Completing future exceptionally...
io.airbyte.workers.exception.WorkerException: Dbt Transformation Failed.
	at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:72) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
	at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:26) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: io.airbyte.workers.exception.WorkerException: DBT Transformation Failed.
	at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:68) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
	... 3 more
	Suppressed: io.airbyte.workers.exception.WorkerException: Dbt transformation process wasn't successful
		at io.airbyte.workers.general.DbtTransformationRunner.close(DbtTransformationRunner.java:141) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
		at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:57) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
		at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:26) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
		at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2023-04-18 19:59:40 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-04-18 19:59:40 [33mWARN[m i.a.c.i.LineGobbler(voidCall):119 - generic gobbler IOException: Socket closed. Typically happens when cancelling a job.
2023-04-18 19:59:40 [32mINFO[m i.a.c.t.TemporalUtils(withBackgroundHeartbeat):283 - Stopping temporal heartbeating...
2023-04-18 19:59:40 [33mWARN[m i.t.i.a.ActivityTaskExecutors$BaseActivityTaskExecutor(execute):114 - Activity failure. ActivityId=530ea8b2-c62d-3bce-b73e-66890d2008a1, activityType=Run, attempt=1
java.lang.RuntimeException: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Dbt Transformation Failed.
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:281) ~[io.airbyte-airbyte-commons-temporal-0.41.0.jar:?]
	at io.airbyte.workers.temporal.sync.DbtTransformationActivityImpl.run(DbtTransformationActivityImpl.java:101) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.activity.RootActivityInboundCallsInterceptor$POJOActivityInboundCallsInterceptor.executeActivity(RootActivityInboundCallsInterceptor.java:64) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.RootActivityInboundCallsInterceptor.execute(RootActivityInboundCallsInterceptor.java:43) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.ActivityTaskExecutors$BaseActivityTaskExecutor.execute(ActivityTaskExecutors.java:95) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.ActivityTaskHandlerImpl.handle(ActivityTaskHandlerImpl.java:92) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handleActivity(ActivityWorker.java:241) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:206) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:179) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.17.0.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Dbt Transformation Failed.
	at io.temporal.serviceclient.CheckedExceptionWrapper.wrap(CheckedExceptionWrapper.java:57) ~[temporal-serviceclient-1.17.0.jar:?]
	at io.temporal.internal.sync.WorkflowInternal.wrap(WorkflowInternal.java:461) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.activity.Activity.wrap(Activity.java:52) ~[temporal-sdk-1.17.0.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:167) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
	at io.airbyte.workers.temporal.sync.DbtTransformationActivityImpl.lambda$run$3(DbtTransformationActivityImpl.java:132) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:276) ~[io.airbyte-airbyte-commons-temporal-0.41.0.jar:?]
	... 14 more
Caused by: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Dbt Transformation Failed.
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:161) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
	at io.airbyte.workers.temporal.sync.DbtTransformationActivityImpl.lambda$run$3(DbtTransformationActivityImpl.java:132) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:276) ~[io.airbyte-airbyte-commons-temporal-0.41.0.jar:?]
	... 14 more
Caused by: io.airbyte.workers.exception.WorkerException: Dbt Transformation Failed.
	at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:72) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
	at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:26) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
	... 1 more
Caused by: io.airbyte.workers.exception.WorkerException: DBT Transformation Failed.
	at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:68) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
	at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:26) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
	... 1 more
	Suppressed: io.airbyte.workers.exception.WorkerException: Dbt transformation process wasn't successful
		at io.airbyte.workers.general.DbtTransformationRunner.close(DbtTransformationRunner.java:141) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
		at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:57) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
		at io.airbyte.workers.general.DbtTransformationWorker.run(DbtTransformationWorker.java:26) ~[io.airbyte-airbyte-commons-worker-0.41.0.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.41.0.jar:?]
		at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2023-04-18 19:59:42 [32mINFO[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to get state

Hi, I am new to Airbyte and using free open-source Airbyte. How and where can I find .env file to change the password after initial login? Thanks.

(Latest Airflow release and connectors are up-to-date) I am trying to set up s3 as the source. There are multiple folders in the bucket. Each folder has different parquet schema definition (i.e. different tables). The files look like

folder1_xxx/**
folder1_yyy/**
folder2_xxx/**

I set airbyte to read prefix

folder1_

, and pattern of files to replicate to

**

I was hoping it will pick up table

xxx

and

yyy

as two separate namespaces with different schema. It only picked up

xxx

How can I make it discover all tables with different schema in different folders with the same prefix?

I am new to Airbyte, so pardon me if my question seems obvious, I was trying to figure out if Airbyte has a stable release or is still in Beta?

I use Airbyte provider for Apache airflow to trigger a sync through Airflow.

with DAG(dag_id='airflow_airbyte_integration',
        default_args=default_args,
        schedule_interval=None,
   ) as dag:

   trigger_airbyte_sync = AirbyteTriggerSyncOperator(
       task_id='airbyte_trigger_sync',
       airbyte_conn_id='airflow-airbyte-integration',
       connection_id=AIRBYTE_CONNECTION_ID,
       asynchronous=True,
       wait_seconds=5
   )

   wait_for_sync_completion = AirbyteJobSensor(
       task_id='airbyte_check_sync',
       airbyte_conn_id='airflow-airbyte-integration',
       airbyte_job_id=trigger_airbyte_sync.output
   )

   trigger_airbyte_sync >> wait_for_sync_completion

My airbyte deployment running in kubernetes pods, and the pods gets restart sometimes due to DB connection timeouts. However, Airbyte sync completed successfully, but Airflow mentioned it failed because Airflow found the connection issue while the pod get restarted. The failure occurs at

wait_for_sync_completion

task. Is there a way to add a retry for

AirbyteJobSensor

without failing at the first instance?

This message was deleted.

Hi Team, Developing one custom connector using Python Http Api source(POST method). Have to change the post request for each query in request body json , Request Body Json 1 : "filters": [ customer_id=1 ] Request Body Json 2 : "filters": [ customer_id=2 ] have to save all response in separate files, could you please help how to achieve this This customer id i have to get via another http GET request and use it in above request body json. Aim is to get each response saved in different files.

airbyte/init:0.44.0, airbyte/bootloader:0.44.0 docker containers exit after starting up for a few seconds, is that the expected behaviour?

This message was deleted.

This message was deleted.