anyone any help on this?

Hi team, I’m using the Salesforce connector and would like to review the query jobs locally using the

/services/data/vXX.X/jobs/query/queryJobId API endpoint

. I have retrieved the Job Id from the log

2023-05-08 00:01:20 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7502j00000AFkjHAAT to complete. Current state: InProgress

and formulate the API endpoint:

<INSTANCE_URL>/services/data/v57.0/jobs/query/7502j00000AFkjHAAT

. However, I’m getting

404 Not Found

. Any idea?

[
    {
        "errorCode": "NOT_FOUND",
        "message": "The requested resource does not exist"
    }
]

Hi folks, We have upgraded the Airbyte version to 0.44.2 and we are facing an issue where the Transformation section for all connectors displays the message

Normalization and Transformation operations are not supported for this connection

. Additionally, we are using BigQuery as our destination which is not a file storage. Has anyone else experienced a similar issue?

Hi, I’d like to verify locally the functionality of Airbyte before deploying it to our cloud environment. Specifically the AWS Datalake destination. However, my AWS Access Key uses MFA, thus I • either need to be able to pass on the temporary key to the destination pod. It would expire within a few hours, but that’s enough to verifty functionality. I modified the

.env

and

docker-compose.yml

file to pass the relevant AWS environment variables. But as the destination runs in a newly run Docker, which doesn’t pass the variables, that didn’t work. • Somehow use a service user locally from my laptop. Is there a workaround that I can locally do to use my own AWS credentials (specifically

aws_session_token

) in a destination?

Hey all, I'm having an issue setting up MongoDB standalone source with TLS.

io.airbyte.db.exception.ConnectionErrorException: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=<http://preproduction-mongo-1.vpc.travelbank.com:27017|preproduction-mongo-1.vpc.travelbank.com:27017>, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}}]
	at io.airbyte.integrations.source.mongodb.MongoDbSource.getAuthorizedCollections(MongoDbSource.java:145)
	at io.airbyte.integrations.source.mongodb.MongoDbSource.lambda$getCheckOperations$0(MongoDbSource.java:79)
	at io.airbyte.integrations.source.relationaldb.AbstractDbSource.check(AbstractDbSource.java:85)
	at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:121)
	at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:97)
	at io.airbyte.integrations.source.mongodb.MongoDbSource.main(MongoDbSource.java:52)
Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=<http://preproduction-mongo-1.vpc.travelbank.com:27017|preproduction-mongo-1.vpc.travelbank.com:27017>, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}}]
	at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:181)
	at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:41)
	at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:144)
	at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:101)
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:291)
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:183)
	at com.mongodb.client.internal.MongoDatabaseImpl.executeCommand(MongoDatabaseImpl.java:195)
	at com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:164)
	at com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:159)
	at com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:149)
	at io.airbyte.integrations.source.mongodb.MongoDbSource.getAuthorizedCollections(MongoDbSource.java:130)
	... 5 more

Hello, anyone encountered this issue before?

hi all, i am looking to switch our airbyte kustomize deployment to helm now that we have a beta helm chart. Are there any concrete docs on how to make this transition?

is there an existing destination for salesforce, or just the source?

Hello, I would like to know if there is any way to make any changes to the connection, but when I save it, it no longer syncs then, do it only at the scheduled time?

Hey all, I'm having an issue setting up MongoDB standalone source with TLS.

io.airbyte.db.exception.ConnectionErrorException: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=<http://preproduction-mongo-1.vpc.travelbank.com:27017|preproduction-mongo-1.vpc.travelbank.com:27017>, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}}]
	at io.airbyte.integrations.source.mongodb.MongoDbSource.getAuthorizedCollections(MongoDbSource.java:145)
	at io.airbyte.integrations.source.mongodb.MongoDbSource.lambda$getCheckOperations$0(MongoDbSource.java:79)
	at io.airbyte.integrations.source.relationaldb.AbstractDbSource.check(AbstractDbSource.java:85)
	at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:121)
	at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:97)
	at io.airbyte.integrations.source.mongodb.MongoDbSource.main(MongoDbSource.java:52)
Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=<http://preproduction-mongo-1.vpc.travelbank.com:27017|preproduction-mongo-1.vpc.travelbank.com:27017>, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}, caused by {sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target}}]
	at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:181)
	at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:41)
	at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:144)
	at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:101)
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:291)
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:183)
	at com.mongodb.client.internal.MongoDatabaseImpl.executeCommand(MongoDatabaseImpl.java:195)
	at com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:164)
	at com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:159)
	at com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:149)
	at io.airbyte.integrations.source.mongodb.MongoDbSource.getAuthorizedCollections(MongoDbSource.java:130)
	... 5 more

Hi Team, I am trying to use airbyte MySQL connector as destination for one of our integration. It seems to be working fine with our Preproduction RDS instance (MYSQL Engine). However, with the same access level, we could not make the connection in production RDS instance (Aurora MySQL Engine). We were getting the following error when we try to use the MySQL connector as destination, Could anyone please help on that. (FYI - The MySQL connector as source works fine with the same Production RDS instance)

Could not connect with provided configuration. HikariPool-1 - Connection is not available, request timed out after 60005ms.

Hello guys, can someone help us with that error ?

2023-05-09 11:30:44 - Additional Failure Information: message='activity timeout', timeoutType=TIMEOUT_TYPE_HEARTBEAT

Source SQLSERVER Destination BIGQUERY These connector was running without errors and today we did 10 attempts without success.

I'm facing a

heartbeat_timeout

issue when synchronizing large tables from a Shopify source. The error message is:

"failureOrigin" : "source",
  "failureType" : "heartbeat_timeout",
  "internalMessage" : "Heartbeat has stopped. Heartbeat freshness threshold: 10800 secs Actual heartbeat age: 10825 secs",
  "externalMessage" : "The source is unresponsive",

I'd appreciate any advice on adjusting the heartbeat timeout settings or other suggestions for resolving this problem with large Shopify tables. Additionally, is there an environment variable that can be configured to control the

DEFAULT_TIMEOUT_CHECK_DURATION

?

any workarounds around this issue https://github.com/airbytehq/airbyte/issues/7379 I feel like mongo needs more support, it’s one of the few databases which everyone needs to ETL due to it not being suitable for analytics

Does anyone have a recommendation for connecting Airbyte to a datasource that can be accessed through an ODBC driver? We are looking to connect a client's on-prem sage50.

I have setup a connection to Marketo but I can not find where to customise the max number of API calls, as mentioned under Performance considerations here https://docs.airbyte.com/integrations/sources/marketo/ Where can I reduce the max number of API calls the source connector makes?

Hi there! I have been using Airbyte in my local machine to stream data from MongoDB to Bigquery for the last 5 months and so far everything has worked perfectly. Now i have deployed Airbyte in a Google cloud VM so i can automate the data sync process. To do this, i have used the same configuration details i’ve used before to set up the source from Mongo (This worked fine) but when I’m trying to set up the Bigquery destination with the same configuration i use in my local machine, it does not connect and i get the following error message:

[1;31mERROR[m i.a.w.g.DefaultCheckConnectionWorker(run):125 - Unexpected error while checking connection: 
io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/destination-bigquery:1.3.3

I believe it has something to do with the fact that i’m not running it now from my local machine but from the VM ip address i’ve created Can someone assist please?

Hello folks, I'm trying to sync data from Zendesk Support here... I can't extract data from 4 streams. Is this your first time deploying Airbyte: No Memory / Disk: 8Gb / 16GB Deployment: EKS Airbyte Version: 0.41.0 Source name/version: Zendesk Support/ Destination name/version: S3 CSV/S3 Parquet Source Connector Version: 0.2.29 Step: Running Zendesk Support connection Description: I can't extract data from tickets, users, ticket_audits and ticket_comments streams. I got the same detail from the log: "Skipping stream tickets: Check permissions, error message: {'title': 'Forbidden', 'message': 'You do not have access to this page. Please contact the account owner of this help desk for further help.'}." However, I could extract tickets data using the Zendesk API. We use the same user and token on Stitch... without permission issues for these 4 streams. Can you give me some help with that?

Hello, I can configure the custom DBT transformation, looking for a subdirectory inside git. For example https://github.com/user/repository/SUB_DIR... Or would it only be in the main repository? Thanks.

Hi All, I am trying to extract the auto-generated destination normalisation queries. The Airbyte documentation only talks about doing this from Airbyte hosted in a docker container: https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-sql/ Could you advise on which Airbyte pod to extract these sql files from for Airbyte hosted in Kubernetes?

Does anyone have advice on how to avoid a bad Airbyte sync corrupted healthy destination (BigQuery) tables? When a sync fails, I expect my destination tables to be stale. It's way worse when Airbyte zeroes them out, due to some buggy behavior. Here is a snapshot of a BigQuery table that had many rows, but now has zero rows because of a failed Airbyte sync:

This message was deleted.

Hi everyone. Out connection to google ads to our snowflake warehouse failed today after having run successfully once this day already. According to the error logs:

google.auth.exceptions.RefreshError: ('invalid_grant: Bad Request', {'error': 'invalid_grant', 'error_description': 'Bad Request'})

,retryable=<null>,timestamp=1683740480982], metadata: {connector_repository=airbyte/source-google-ads, workspace_url=<http://localhost:8000/workspaces/97616a89-8d1e-4a08-b894-b025230d2a39>, failure_type=system_error, connector_command=check, connector_name=Google Ads, workspace_id=97616a89-8d1e-4a08-b894-b025230d2a39, airbyte_version=0.40.23, connection_url=<http://localhost:8000/workspaces/97616a89-8d1e-4a08-b894-b025230d2a39/connections/1eecbab0-fcf5-44c8-9a1f-4da2960deccf>, connector_definition_id=253487c0-2246-43ba-a21f-5116b20a2c50, failure_origin=source, connection_id=1eecbab0-fcf5-44c8-9a1f-4da2960deccf, connector_release_stage=generally_available, job_id=47179, deployment_mode=OSS}

2023-05-10 17:41:21 INFO i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed.

errors: $.auth_type: must be a constant value OAuth2.0, $.auth_type: does not have a value in the enumeration [OAuth2.0], $.access_token: is missing but it is required, $.refresh_token: is missing but it is required

2023-05-10 17:41:21 INFO i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed.

errors: $.auth_type: must be a constant value Key Pair Authentication, $.auth_type: does not have a value in the enumeration [Key Pair Authentication], $.private_key: is missing but it is required

We have not changed any settings and it seems like the access and refresh tokens are missing, although they were never deleted. Has anyone come across this before?

Hello, I have a connection with sql server origin and postgresql destination, using CDC, I created index manually in some table in postgresql. But when I sync using CDC, it removes the index, I understand it should keep?

I found this first-time issue https://github.com/airbytehq/airbyte/issues/21262 published by @Greg Solovyev (AirByte). My team will try to fix it and RP according to the contributors’ guide. Can I ask if we need to fix it both in front-end and back-end? Can I get assigned to this issue? I will let you know the result in a week.

Any timeline for supporting AWS documentDB as a source?

Hi, I'm totally beginner, just installed Airbyte. Can you please tell me how can I find and change the .env file to change username and password?

Hello there, Simple question: Is it possible to disabled source and destination checks ? I have thousands of connectors using the same destination and running 1000+ pods to execute the same ops a day isn't very effective

Dear all, could you help me with Google ADS -> Airbyte -> PostgreSQL connection? So, overall it works fine, but I have an issue with duplication rows with Incremental Append mode. The issue - Each time of incremental sync it sync all record again and again... Airbyte Version: OpenSource 0.44.4 by Docker Google ADS Plugin: 0.2.14

In the Hubspot connector I am not seeing the incremental connector for property_history. Has someone experienced this?