Have you ever added elastic search source into airbyte with using terraform: resource "airbyte_source_custom" "custom" { name = "Elasticsearch" workspace_id = var.workspace_id definition_id = "7cf88806-25f5-4e1a-b422-b2fa9e1b0090" configuration = jsonencode({ authentication_method = { type = "Username/Password" username = "username" password = "password" } endpoint = "endpoint" }) } i cannot add properly. endpoint writing correctly but authentication_method, username, password cannot be selected. Am i missing something?

Hi everyone. I'm having trouble submitting an update to the Airbyte documentation. My PR is failing the Community CI workflow in particular the step to "Check if a review is required from Connector teams on fork". It seems this step is failing for lots of Community CI workflows. Does anyone know what may be causing it? Thanks for your help.

Hey 👋, did anyone succeed to use

datadog

to monitor

airbyte

deployed with

abctl

?

Hello everyone ! Has anyone ever developed a Docker custom connector for a GKE Airbyte deployment from a macbook ? I am using airbyte-ci to build a the connector, but when pulling the image from the Airbyte UI, the pod has this error :

exec /usr/bin/sh: exec format error

. I already tried

--architecture=linux/arm64

and other similar fixes, nothing seem to work. Has anyone managed to solve this issue ?

Hello everyone, can help me with this error

"internalMessage" : "Unable to start the source"

"stacktrace" : "io.airbyte.workers.exception.WorkerException: Unable to start the source\n\

, I'm using abctl, the connection is mongo -> bigquery

I would like to apply the change of

MAX_SYNC_WORKERS

on my Airbyte instance (installed via abctl)

worker:
  extraEnvs:
    - name: MAX_SYNC_WORKERS
      value: "2"

Do I need to run through

abctl uninstall

and

abctl install

or there is some way to do it on running instance?

Hello, I’ve been trying to sync my Postgres database to a Pinecone index, and although the sync process succeeds, my index’s record count remains as 0, is there anything I should check to troubleshoot this issue??

Airbyte transformation I have migrated from docer compose to abctl aon my airbyte server deployed in an ec2 instance. I used to have a transformation tab, but after the upgrade, it looks like the transformation tab is no longer available. Can an expert in this area please tell me why and what solutions they are offering to user who need to run dbt transformation right after a sync. Is there still a way to do Transformation? And any documentation on this topic? Before upgrade from v0.50.40 Airbyte transformation I have migrated from docer compose to abctl aon my airbyte server deployed in an ec2 instance. I used to have a transformation tab, but after the upgrade, it looks like the transformation tab is no longer available. Can an expert in this area please tell me why and what solutions they are offering to user who need to run dbt transformation right after a sync. Is there still a way to do Transformation? And any documentation on this topic? Before upgrade from v0.50.40 - transformation available After upgrade:

I've set up airbyte (using abctl) behind nginx and am able to load the main page, but I'm getting unauthorized errors when I try to click "get started". I think this is maybe due to the proxy configuration? Any idea how I can get this working? My nginx proxy config is as follows:

server {
  listen 443 ssl;
  server_name <http://my.domain.com|my.domain.com>;
  client_max_body_size 200M; 
  ssl_certificate /etc/ssl/certs/airbyte.crt.pem;
  ssl_certificate_key /etc/ssl/certs/airbyte.key.pem;

  location / {
    proxy_pass <http://localhost:8000>;
    proxy_set_header Cookie $http_cookie; 
    proxy_read_timeout 3600;
  }
}

Databricks Lakehouse v3.2.1

Hello! We’ve been experiencing a recurrent error in our Databricks destination syncs. The error is:

...
Caused by: com.databricks.sql.transaction.tahoe.DeltaAnalysisException: [DELTA_MISSING_DELTA_TABLE_COPY_INTO] Table doesn't exist. Create an empty Delta table first using CREATE TABLE `<catalog>`.`<schema>`.`airbyte_check_test_table`. 
...

The issue appears to stem from a concurrency problem in the Databricks destination connector when multiple connections attempt to write data simultaneously. The current implementation checks for the existence of a

airbyte_check_test_table

Delta table and attempts to create it if it doesn’t exist. However, this check and creation process is not synchronized, leading to race conditions where the table creation fails because another process is creating or has just created the table - the issue may be connected to this part of the destination source code. Has anyone else encountered this issue?

Hello anyone knows to update cpu limitation in

abctl

?

When syncing from Shopify to Postgres, our syncs fail with a variety of errors. Does anyone recognise these errors ("statement already exists") and how to resolve them?

Hey everyone! I am trying to install airbyte locally on an old macbook using abctl and then exposing the instance to the public by port forwarding on my router. When I try this I get nginx 404 Not Found error in the browser, I think this is because with abctl the webapp is bound to localhost. I have zero networking experience and wanted to see if anyone else has tried this sort of hacky airbyte deployment and succeeded.

I am trying to install with abctl v0.14.1 and the default 0.524.0 chart I have custom --secret file and a custom --values file When I run with -v I get this:

DEBUG   Successfully assigned airbyte-abctl/airbyte-abctl-airbyte-bootloader to airbyte-abctl-control-plane
  DEBUG   helm - DEBUG: Add/Modify event for airbyte-abctl-airbyte-bootloader: MODIFIED
  DEBUG   helm - DEBUG: Pod airbyte-abctl-airbyte-bootloader pending
  DEBUG   Container image "airbyte/bootloader:0.64.1" already present on machine
  DEBUG   Created container airbyte-bootloader-container
  DEBUG   Started container airbyte-bootloader-container
  DEBUG   helm - DEBUG: Add/Modify event for airbyte-abctl-airbyte-bootloader: MODIFIED
  DEBUG   helm - DEBUG: Pod airbyte-abctl-airbyte-bootloader running
  DEBUG   helm - DEBUG: Add/Modify event for airbyte-abctl-airbyte-bootloader: MODIFIED
  DEBUG   helm - DEBUG: Pod airbyte-abctl-airbyte-bootloader running
  DEBUG   helm - DEBUG: Add/Modify event for airbyte-abctl-airbyte-bootloader: MODIFIED
  DEBUG   helm - DEBUG: warning: Upgrade "airbyte-abctl" failed: pre-upgrade hooks failed: 1 error occurred:
                * pod airbyte-abctl-airbyte-bootloader failed
  DEBUG   helm - DEBUG: release upgrade failed: pre-upgrade hooks failed: 1 error occurred:
                * pod airbyte-abctl-airbyte-bootloader failed
  ERROR   Failed to install airbyte/airbyte Helm Chart
  ERROR   Unable to install Airbyte locally
  ERROR   unable to install airbyte chart: unable to install helm: pre-upgrade hooks failed: 1 error occurred:
                * pod airbyte-abctl-airbyte-bootloader failed%

which gives me a fact “pre-upgrade hooks” failed but fails to provide any explanation as to why the pre-ugrade hooks failed I am running on a freshly-minted Amazon Linux 2023. This was working yesterday (with v0.13.1) but has failed everytime I have tried it today. Where do I look to find an explanation for why the pre-upgrade hook failed. What do i do to build helm chart that has sufficient diagnostics enabled to allow me to understand this failure?

Heya, anyone encounter workload launcher fail to resolve the temporal host? trying out the new architecture of OSS Airbyte

Hi, Can anyone help me with setting up a Sharepoint connector? I have the client ID, Client secret and the tenant ID but I do not know what value I should use for the "User Principal Name"? I am getting the error below when trying to create the source: ['Traceback (most recent call last):\n File "/usr/local/lib/python3.9/site-packages/office365/runtime/client_request.py", line 38, in execute_query\n response.raise_for_status()\n File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status\n raise HTTPError(http_error_msg, response=self)\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://graph.microsoft.com/v1.0/users/airbyte/drive\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/airbyte/integration_code/source_microsoft_sharepoint/utils.py", line 51, in execute_query_with_retry\n return obj.execute_query()\n File "/usr/local/lib/python3.9/site-packages/office365/runtime/client_object.py", line 52, in execute_query\n self.context.execute_query()\n File "/usr/local/lib/python3.9/site-packages/office365/runtime/client_runtime_context.py", line 183, in execute_query\n self.pending_request().execute_query(qry)\n File "/usr/local/lib/python3.9/site-packages/office365/runtime/client_request.py", line 42, in execute_query\n raise ClientRequestException(*e.args, response=e.response)\noffice365.runtime.client_request_exception.ClientRequestException: (\'ResourceNotFound\', \'User not found\', \'404 Client Error: Not Found for url: https://graph.microsoft.com/v1.0/users/airbyte/drive\')\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py", line 81, in _check_list_files\n file = next(iter(stream.get_files()))\n File "/airbyte/integration_code/source_microsoft_sharepoint/stream_reader.py", line 143, in get_matching_files\n files = self._get_files_by_drive_name(self.drives, self.config.folder_path)\n File "/airbyte/integration_code/source_microsoft_sharepoint/stream_reader.py", line 131, in drives\n my_drive = execute_query_with_retry(\n File "/airbyte/integration_code/source_microsoft_sharepoint/utils.py", line 75, in execute_query_with_retry\n raise AirbyteTracedException.from_exception(ex, message="Caught unexpected exception")\nairbyte_cdk.utils.traced_exception.AirbyteTracedException: (\'ResourceNotFound\', \'User not found\', \'404 Client Error: Not Found for url: https://graph.microsoft.com/v1.0/users/airbyte/drive\')\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py", line 62, in check_availability_and_parsability\n file = self._check_list_files(stream)\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py", line 87, in _check_list_files\n raise CheckAvailabilityError(FileBasedSourceError.ERROR_LISTING_FILES, stream=stream.name) from exc\nairbyte_cdk.sources.file_based.exceptions.CheckAvailabilityError: Error listing files. Please check the credentials provided in the config and verify that they provide permission to list files. Contact Support if you need assistance.\nstream=leo stream\n']

Hello, how can I temproraly stop abctl running without uninstalling?

When trying to sync the source I get:

Workload failed, source: workload-launch

attaching the logs: what is happening here? How could I debug?

Hi all, I am using macOS, airbyte is set up using abctl, I am setting up both source and destination with MongoDB. I have established the connection. However, the sync process won't work with the error message as below. Any idea for solving this? Thank you all in advance

Hi all, I have just installed a new oss version with abctl on a local docker instance, but am having issues with my first login. I have set my username/password with abctl credentials and they both appear correctly in the output but when i try to login via the browser i get the error invalid username or password. For the password i tried entering it into abctl as a base64 encoded value and in plain text but neither version worked. Any ideas what im missing?

Is it possible the connector auto downgrade itself?

Hi , i'm building a Custom Connector, i'm trying to format a unix timestamp to datetimr format, how to achieve it in airyte UI for E.G if my response is "date_start_session":1724386430 how to Convert this UNIX to datetime Format, ?

Warning from replication: Something went wrong during replication Destination process is still alive, cannot retrieve exit value. why I get this error while trying to sync for the first time after connecting the source and destination in the cloud

Hi Airbyte team, I am trying to install aibyte using kubernetes on an on prem server that has no FQDN. I followed the instructions in the documentation - https://docs.airbyte.com/deploying-airbyte/ And when it came to configuring ingress, with reference to previous slack conversations, I had put host as airbyte.local as my server don't have a FQDN. And also added the private ip to my /etc/hosts file in my local machine from where I am trying to use the Airbyte's Ui. After that, when I try to access the Ui - I get 502 bad gateway error. Please let me know if I can try something different to access Airbyte hosted on a private server

Hi all I created a job through the builder that pulls data from the API and writes it to Clickhouse, and I ran into this problem: Code: 60. DB:Exception Table database._airbyte_raw_users does not exist. Maybe you meant _airbyte_raw_users_? Can you tell me how to fix the problem?

Hi everyone. I'm desesperate it's been weeks i'm stuck on this problem and anywhere on internet, support I can find a solution. Here is the message error after I updated connector source : Failure in destination: You must upgrade your platform version to use this connector version. Either downgrade your connector or upgrade platform to 0.63.7. Weeks and nothing I tried worked. Does anyone have any clue please ? I'm using airbyte on cloud with digital ocean.

Hi all, I’m currently facing issues on the #Looker connector : https://docs.airbyte.com/integrations/sources/looker. When I’m feeding the config to the connector, I am facing the following “’Unable to connect to the Looker API. Please check your credentials.’“. This issue is faced when trying to authenticate. However, when I’m trying to connect using the API and the same value, I’m receiving a 200. Does anyone is facing the same issue while using this connector ? Thanks for your help !