Hey folks, I have set up Airbyte on an EC2 instance and I want to set up DNS on it. For that, I need to change the port for the webapp to 443. Does anyone know how to do this? I tried changing in .env file (WEBAPP_URL), but that did not work for me.

Hi All, I'm testing the waters with airbyte. I downloaded the docker image and spun up the containers. I see there's a

temporal

container running but I'm unable to reach it (it's not exposing any port). Is this normal behavior? What are the steps to test the temporal integration?

set the channel topic: Discussions about our Self-Managed offers on airbyte.com/pricing

Hey everyone! I recently started using Airbyte for my startup on the open-sourced version. Everything is working smoothly except for the Google Sheets destination. I get the error message "Internal message: 'str' object has no attribute 'json'" did anyone have the same problem?

Hi All, I'm using Airbyte with PG source and mysql destination, and i have a little trouble. The Sync job get a table entirely and write it on the destination. Source table contain about 35k lines. Sometime the job tell me he proceed 10k only without any error. I don't know why because the source table is always about 35k lines. any idea about a possible limitation ?

Hello, everyone. My team and I are looking at Airbyte for some data integration work flows we are building. One of our potential needs is low latency syncing between two data stores (SLA in minutes). I understood that Airtbyte Cloud can sync source to destination as often as 1 hour. Can the self-hosted application perform more frequent incremental syncs? Also if the source is a stream like Kafka does a sync period still apply or can Airbyte manage continuous syncing from Kafka to a database destination?

has renamed the channel from "airbyte-self-managed" to "premium-support-qa"

Hi Team , need a help on one of my scenarios

if I can leverage Airbyte

hello! apologies if this isnt the right channel. I see in the connectors documentation there is an out of the box connector for microsoft dynamics customer engagement on prem. is there a connector for the cloud based product? if not, are there any plans to support it in the near future? thanks in advance

👋 I am working on a self-hosted (k8s) Airbyte at BetterUp with @Mauricio Alarcon where we need to build custom alpine-based images for all Airbyte containers including the sources and destinations. We have custom connectors setup for all platform images and our sources and destinations, but for the Snowflake Destination Connector that we're working on (forked from airbyte-integrations/connectors/destination-snowflake) we are encountering some issues with the normalization image tied to our snowflake connector. We followed the Connector Development documentation very closely but are still having issues. We set the

.data.normalizationConfig

object in the

metadata.yaml

to point to our custom built

normalization-snowflake

image and built all the images with gradle, but we are not seeing the "Transformations" tab in the UI or anything indicating that Airbyte is picking up our Snowflake normalization image. Can someone help us get a custom Snowflake normalization image up and running in our cluster?

I want to run airbyte in my windows, i have installed docker desktop, clone the repo but when i run bash run-ab-platform.sh- it throws below error run-ab-platform.sh: line 2: $'\r': command not found run-ab-platform.sh: line 7: $'\r': command not found

Hello, I am using airbyte to sync data from postgresql to elasticsearch. When I was doing full synchronization of a large table, our EC2 IO was fully loaded, reaching 125M/s. Then my instance went down due to high load. Is there any way to batch pull less data to avoid this problem? Is there anything that can be configured? Direct Message

Hello, I am currently facing a challenge with a full refresh sync operation from my source database to my destination database. I understand that the batch size for these operations isn’t directly configurable, and as a result, the process seems to be attempting to extract all the data at once, which subsequently leads to a crash due to memory overload. I am interested in exploring possible solutions to this issue. Could you kindly guide me on how I could perform a full refresh sync operation in a paginated or chunked manner? This way, the memory burden would potentially be reduced, thereby preventing the process from crashing. Any guidance on this would be greatly appreciated.

hello, i am facing a issue in airbyte, i am connecting sql server management studio to big query gcp there are tables where i have columns whose data type is 'datetime' but when i am connecting as a source it changes those columns data type to 'string' so how to deal with that

Hello, Our team is currently working on an application, that utilizes Airbyte for seamless data transfer from MongoDB to PostgreSQL. However, we've run into an issue concerning the transfer of DateTime data. During the process, Airbyte automatically converts it into text format, which poses a problem for us. Is there a possible solution to configure the mapping data or resolve this issue? We eagerly await your response and appreciate your assistance. Thank you kindly.

hi, i just have a quick question, would the db2 connector also work for a db2 for i database

Hi, I am buidling a source which uses oauth2 authentication using python CDK(htttp api method). I have an access_token which has 30 days expiry & a refresh_token which get expired once we use it to generate a new access_token & refresh_token. I have tried implemeting with SingleUseRefreshTokenOauth2Authenticator. But I am facing an issue like once the access_token gets expired, the newly generated access_token & refresh_token is not setted properly for the next sync(i.e it is taking the tokens which I have given initially) How could I solve this?

Is this a "help" channel, or is it to discuss this product?: https://airbyte.com/talk-to-sales-premium-support Seems like all of the questions are "help" related, so... Perhaps a better channel description can guide the users on the topic?

Hello - does anyone know if the source connector for MSSQL runs queries using READ UNCOMMITED / NOLOCK by default? If not, is there a place in the config where I could specify this?

I am trying to deploy airbyte using helm on EKS, getting this unknown error, can anyone help:

Error: UPGRADE FAILED: template: airbyte/charts/webapp/templates/secrets.yaml:14:60: executing "airbyte/charts/webapp/templates/secrets.yaml" at <.Values.global.secrets>: wrong type for value; expected map[string]interface {}; got interface {}

Hi! I'm trying to run airbyte locally and I'm trying to setup a connection between Snapchat and BQ. I use standard inserts, all checks pass but I get this error when syncing:

Unhandled error while executing model.airbyte_utils.organizations
Pickling client objects is explicitly not supported.
Clients have non-trivial state that is local and unpickleable.
25 of 56 ERROR creating table model airbyte_ingestion_test.organizations................................................ [ERROR in 1.09s]

Anyone who knows how to fix this?

Hello, I'm evaluating airbytes integration in our system. I tried cloud version, but I am more want to use the enterpise version hosted on our GCP cloud. But before pursuing that option, I tried the community to install on our our cloud first. After installing the CE version, i try setup the connection to

facebook marketing

but I couldn't find the button "Authorize your facebook marketing" as the following in setup on the cloud

Hello guys, I’m evaluating Airbyte OpenSource but I want to use it as cloud / enterprise, but I have some premieses that I don

Hello guys, I’m evaluating Airbyte OpenSource but I want to use it as cloud / enterprise, but I have some premises that I don’t see how to solve on Cloud. 1. I need data residency on Brazil 2. I need to create custom private connectors (source and destinations of our systems) 3. I need to create those flows through API as we are configuring the connections though another software. Is this something possible on the cloud edition or I should stick with the OpenSource hosted version, but it seems to lack the API to communicate as well.

Hello all, I trying to integrate with the Salesforce marketing cloud and want to retrieve data from the data extension but I am getting errors while generating an Access token.

"status": 400,
  "body": {
    "error": "unsupported_grant_type",
    "error_description": "Use \"authorization_code\" or \"refresh_token\" or \"client_credentials\" or \"urn:ietf:params:oauth:grant-type:jwt-bearer\" as the grant_type.",
    "error_uri": "<https://developer.salesforce.com/docs>"
  },
  "headers": {
    "x-mashery-message-id": "ef559b26-cf94-40db-a6fb-7e1d553cab21",
    "x-mashery-responder": "13-at03",
    "strict-transport-security": "max-age=15552000; includeSubDomains; preload",
    "Content-Security-Policy": "upgrade-insecure-requests",
    "x-xss-protection": "1; mode=block",
    "x-frame-options": "DENY",
    "x-content-type-options": "nosniff",
    "cache-control": "no-cache, must-revalidate, max-age=0, no-store, private",
    "Referrer-Policy": "strict-origin-when-cross-origin",
    "Content-Length": "252",
    "Content-Type": "application/json",
    "Date": "Tue, 12 Sep 2023 12:45:41 GMT",
    "Connection": "keep-alive",
    "Keep-Alive": "timeout=5"
  }
}
Connector Logs

Hey Team: Iam trying to evaluate an POC for the connectors to be OEMd. Is there a pricing sheet or pricing info that you can share ? We want it on onprem/self managed mode

Hi team, I have requested for a call with sales team to understand the costing for our requirements. We got some response from @Jen Sitiaev (Airbyte) but looks like she is on PTO till 26-Sep. Is there any chance that we can get assistance from anyone else? Thanks

Hi, wondering if there's anyone available to discuss enterprise pricing for our organisation?

Hi Can some one connect me with Sales team to discuss enterprise pricing / Support model