Hi Guys, I'm trying to create a new connector in the connector builder but I'm having trouble with the authentication. The authentication appears to complete successfully but the access_token field is not being set. (Screenshot of the error below). I have attached the associated YAML file. The access token response is simple JSON object as follows: { "access_token": “eyJhbGciOiJSUzI1NiIsInR5cCI1IkpXVCJ9.ZRtjDHtsyK6c25YJEj_vxxx”, "refresh_token": “7c9234163d5b5a2e480a9263fb789f027c2a8e4306c7f9c098c8xxx”, } I have tried adding details for the output specification, but this just returns HTTP 500 error. So I have removed it. complete_oauth_output_specification: required: - access_token - refresh_token properties: access_token: type: string path_in_connector_config: - access_token refresh_token: type: string path_in_connector_config: - access_token Can anyone help guide me in the right direction to resolve this?

Hi there, how are you? I'm trying to get my head around

DpathExtractor

yaml config: • Given following array as input

[{
    "name": {
      "common": "Comoros",
      "official": "Union of the Comoros",
      "nativeName": {
        "ara": {
          "official": "الاتحاد القمري",
          "common": "القمر‎"
        },
        "fra": {
          "official": "Union des Comores",
          "common": "Comores"
        },
        "zdj": {
          "official": "Udzima wa Komori",
          "common": "Komori"
        }
      }
    },
    "cca2": "KM",
    "cca3": "COM"
  }]

• how would one extract

[{
  "name": { "common": "Comoros" },
  "common_name": "Comoros",
  "cca2": "KM",
  "cca3": "COM"
}]

Please advise 🙏

Hello airbyte team, I need help with my current set up. I was told to upgrade my connection for Stripe to BigQuery, I have upgraded, but now I am seeing that my tables are not populating anymore. When I check the error this is what I got: > Warning from source: Workload failed, source: unknown > Failed to find output files from connector within timeout of 9 minute(s). Is the connector still running? Can you direct me to the right documentation to troubleshoot this?

Hi Airbyte Community and Team, We're encountering an issue where columns with the

ARRAY

data type in our PostgreSQL source are being read as strings when synced to our Snowflake destination using the PostgreSQL connector. Is this expected behavior? If not, could anyone familiar with the PostgreSQL connector share how they’ve handled array-type columns in similar setups? Any insights or suggestions would be greatly appreciated. Thanks in advance!

@everyone I am having issue as amazon sp api is getting connectes but reports data is not visible in my schema can any one help me out regarding this? @channel

Hi, how are you doing? I found a problem using non-ascii characters in Custom Connector Stream name as well as Field name, e.g. given a Stream Name "καταστήματα" and fields name "όνομα" while loading data to PostgreSQL destination table is created with "_" instead of non-ascii symbols. At the same time, destination database is created with UTF8 support and allows "CREATE TABLE" and "INSERT INTO" with non-ascii table names and field names. Is there any settings which I should use on the connector to use UTF8 symbols in Stream and Field names? Thank you!

Hey @Ian Alton, tagging you since you previously posted about helm chart release + Marcos seems to be gone. The fix for this issue has been merged. However, it is not included in the latest release / tag on that repo nor in the helm chart releases in https://github.com/airbytehq/helm-charts. Is it possible to a get a new patch release cut to include this fix?

Hello, how are you? Please advise the way to troubleshoot connector builder (OAuth in particular case), e.g. where can I find results of Authorise (Consent) and Token endpoint logs? Currently I'm getting 500 which is not really informative. Basically, I would like to build a connector to Allegro which has following auth flow: • Authorize

<https://allegro.pl/auth/oauth/authorize?response_type=code&client_id=a21...6be&redirect_uri=http://exemplary.redirect.uri>

• Token

curl -X POST \
  <https://allegro.pl/auth/oauth/token> \
  -H 'Authorization: Basic base64(clientId:secret)' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'grant_type=authorization_code&code=pOPEy9Tq94aEss540azzC7xL6nCJDWto&redirect_uri=<http://exemplary.redirect.uri>'

Thank you!

My Connector Builder is not returning any records, yet the response is 200 OK! I have properly applied the record selector to fetch results from the response payload, but this is not happening. See the attached screenshot for reference. Please let me know how to get out of this blocker. Request Payload:

{
  "url": "<https://backstage.taboola.com/backstage/api/1.0/sinoinc-nal-plaudus-sc/reports/campaign-summary/dimensions/day?start_date=2025-01-01&end_date=2025-08-05>",
  "headers": {
    "User-Agent": "python-requests/2.32.4",
    "Accept-Encoding": "gzip, deflate",
    "Accept": "*/*",
    "Connection": "keep-alive",
    "Authorization": "Bearer ****"
  },
  "http_method": "GET",
  "body": ""
}

Response payload:

{"status": 200,
  "body": {
    "last-used-rawdata-update-time": "2025-08-05 05:00:00.0",
    "last-used-rawdata-update-time-gmt-millisec": 1754395200000,
    "timezone": "PDT",
    "results": [
      {
        "date": "2025-08-05 00:00:00.0",
        "date_end_period": "2025-08-05 00:00:00.0",
        "clicks": 628,
        "impressions": 271757,
        "visible_impressions": 147123,
        "spent": 105.26,
        "conversions_value": 0,
        "roas": 0,
        "roas_clicks": 0,
        "roas_views": 0,
        "ctr": 0.2310888036002752,
        "vctr": 0.4268537210361398,
        "cpm": 0.39,
        "vcpm": 0.72,
        "cpc": 0.168,
        "campaigns_num": 20,
        "cpa": 6.192,
        "cpa_clicks": 6.579,
        "cpa_views": 105.264,
        "cpa_actions_num": 17,
        "cpa_actions_num_from_clicks": 16,
        "cpa_actions_num_from_views": 1,
        "cpa_conversion_rate": 2.7070063694267517,
        "cpa_conversion_rate_clicks": 2.5477707006369426,
        "cpa_conversion_rate_views": 0.1592356687898089,
        "currency": "USD"
      },

Hi! Is it possible to create a docker image from a connector using

source-declarative-manifest

as the base and copying the manifest.yaml generated with the Custom Builder UI? Can't find documentation about it

Has anyone experience working with the new bulk-cdk for Databases (source)? I‘m currently trying to implement it for source-oracle by reverse engineering source-mysql since there is only limited documentation available

hey everyone, is it possible to use a composite key for cdc tracking when using the Postgres connector? e.g. a table doesn’t have a PK

hi working on creating a custom connector via the guide shared elsewhere in this slack for a makeshift setup but looking at fivetran docs would their setup process work for airbyte with some modification? https://fivetran.com/docs/connectors/applications/microsoft-dynamics/business-central/setup-guide

Hi everyone, I’ve built a custom Python connector for GitLab. GitLab’s OAuth system uses a single-use refresh token, which means that after each authentication, the refresh token becomes invalid and must be replaced with a new one from the response. Here’s an example of the response:

{
  "access_token": "c97d1fe52119f38c7f67f0a14db68d60caa35ddc86fd12401718b649dcfa9c68",
  "token_type": "bearer",
  "expires_in": 7200,
  "refresh_token": "803c1fd487fec35562c205dac93e9d8e08f9d3652a24079d704df3039df1158f",
  "created_at": 1628711391
}

To re-authenticate, I need to use the new refresh token from the response. I’m using the

SingleUseRefreshTokenOauth2Authenticator

to handle this, which saves the new

refresh_token

,

access_token

, and

expire_time

in memory for the next authentication. The issue is that while these config values are correctly saved and used in memory, they are not being persisted in storage for future runs. When I create a new source, a validation check is performed, which passes and creates the source. However, after this check, the OAuth refresh token becomes invalid, and the new one isn’t saved to storage. As a result, I can’t create a new connection based on this source since the refresh token isn’t being updated in storage. Is anyone faced this issue before and knows how to solve it?

Hi All! We have Postgreql source databases that are running in SQL_ASCII encoding and have some number of tables with non-UTF8 compatible characters. I believe I've discovered that the JDBC driver used by the primary Airbyte Postgreql doesn't support us connecting to the source using SQL_ASCII client encoding. We are interested in pursuing the posibility of creating a custom connector that will allow us to connect to Postgreql without using a JDBC Driver. Does anyone out here in slack world have any experience or knowledge in how possible this might be? I'd love to be able to learn from your mistakes before we head off on what might be a difficult path. Thanks!

Has anyone had any issues with source-facebook-pages syncing with post_insights / page_insights 🙂 post_insights: DefaultBackoffException(‘Bad request. Please check your request parameters.’) page_insights: DefaultBackoffException(‘Bad request. Please check your request parameters.’)

Is there a standard way to validate a connector locally? Everything I can find on google seems to be incorrect.

docker run --rm \
  -v $(pwd)/connector.yaml:/airbyte/connector.yaml \
  -v $(pwd)/config.json:/config.json \
  airbyte/source-yaml-connector:latest \
  check --config /config.json
Unable to find image 'airbyte/source-yaml-connector:latest' locally
docker: Error response from daemon: pull access denied for airbyte/source-yaml-connector, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Hello, I would like to develop a destination connector. This is my frist time and I didn't find a proper example to follow to try it. is there an example somewhere I couldd rely on ?

Hi folks! I’m evaluating Airbyte for ingesting events from RabbitMQ and I’d love feedback on whether this is a good fit and how to design a reliable source. What I think I know (please sanity-check me): • In the current protocol, a source doesn’t receive the

AirbyteStateMessage

that the destination emits. With a regular RabbitMQ queue (where messages are removed on ack), that makes it hard to ack only after the destination has safely persisted the record. If the source acks early and something fails downstream, messages could be lost. • RabbitMQ Streams might be a better fit because they don’t delete messages on read. The source could emit state with an incremental offset, and the destination could emit state once writes are durable. On recoveries, the source would resume from the last committed offset without relying on RabbitMQ acks. ◦ Open question: should the destination validate monotonic offsets / detect gaps (“holes”) to guard against missed messages, or does Airbyte’s internal delivery/ordering guarantee make that unnecessary in practice? Replay / backfill: Is it supported (and recommended) to “replay” from a specific timestamp/offset by editing the connection state in the UI/API? My understanding is that editing state is possible; is this the right mechanism for time/offset-based reprocessing with Streams? Does this line of thinking make sense? In your experience, is Airbyte a good fit for this use case, and are there any challenges you’d foresee with this approach?

I'm trying to follow the Tutorial for setting up connectors in the Connector Builder: https://docs.airbyte.com/platform/connector-development/connector-builder-ui/tutorial. However, I'm unable to save or test the connector, because of an "Unauthorized" error. I've populated the API key. This issue persists even when making requests to unsecured endpoints. Any help would be appreciated. I'm in a local airbyte deployment, most up-to-date versions of both abctl and airbyte EDIT: this issue was resolved for my by downgrading to 1.6.3. https://github.com/airbytehq/airbyte/issues/61677

Forking Bing Ads v.2.23.0 and I get this error: Your YAML manifest is not compatible with the Connector Builder UI, due to the following error: Error handling request: Validation against json schema defined in declarative_component_schema.yaml schema failed Clicking confirm will erase any changes you have made in the YAML editor before switching back to the Builder UI. In order to save your current YAML, cancel and click the Export YAML button.

Hey! I'm creating a custom connector and I'm seeing that Airbyte is adding a

lookback_window

of

32243

into the state that was not specified. Can this be changed?

Hi everyone, We’re currently building a custom Amazon Ads connector in Airbyte using the Connector Builder. The flow works fine most of the time: • Report creation request is fast. • Polling requests also succeed when we test manually. • File download works as expected. However, we regularly run into this error in Airbyte:

Internal Server Error: java.net.SocketTimeoutException: timeout

What’s confusing is that sometimes the exact same flow works (even 10+ times in a row), and then suddenly the error pops up again during the polling phase. It looks like Airbyte is not even making the polling request when this happens, as if it times out internally before sending it. We already tried increasing the polling timeout significantly (up to 2000 minutes), but the error still appears, especially overnight runs. Has anyone experienced similar issues with the Amazon Ads connector (or other async-reporting APIs) where Airbyte gives a

SocketTimeoutException

even though the API itself is responsive? Could this be related to Airbyte’s internal HTTP client or how long it keeps connections alive? Any insights, workarounds, or configuration tips would be really appreciated. Thanks!

Hi all, I am using a unique version of postgres greenplum with a unique in-house developed beam storage engine. I come across this error message when I am trying to use source-postrgres connector but resulted in this error:

io.airbyte.cdk.integrations.source.relationaldb.state.FailedRecordIteratorException: java.lang.RuntimeException: java.lang.RuntimeException: org.postgresql.util.PSQLException: ERROR: feature not supported on beam relations

. This does not happen when i am extracting from a normal table. If i were to develop a fork of postgres connector that works with this beam engine, can anyone point me to where to start?

I try to build a connector for Walmart using the Cloud version and run into the issue that I can not have a Header like “WM_SVC.NAME” as of the use of a period. Anyone experienced the same before?

Hi Team, I'm currently trying to understand the criteria in Connector support levels documentation (Connector support levels | Airbyte Docs) Can anyone confirm whether the information collected below are correct ? Support cloud- what kind of help or support you can expect when using a connector in Airbyte Cloud (the fully managed, hosted version of Airbyte) + Official support channels (like the Airbyte support portal) Support: Powered by Airbyte - the level of support you can expect when using Airbyte connectors within products or services that are “Powered by Airbyte” (i.e., platforms or solutions that embed Airbyte’s technology). Support: Self-Managed Enterprise - This criterion describes the level of support you receive for different types of Airbyte connectors when you are using the Self-Managed Enterprise version of Airbyte (i.e., you are running Airbyte on your own infrastructure with an enterprise license)

Hi Team, I'm developing a custom source connector for Report data and facing a challenge with sync modes and normalization behavior. Current Setup

Source: Custom Report API (custom connector)
Sync Requirements:
First sync: Full Refresh | Overwrite (pull all historical data)
Subsequent syncs: Append only (pull only previous fiscal year data)
Using Basic Normalization
Tracking sync state in external database (sync_completed_date)

Sample Code

class Report(ReportStream):
    report_name = "Report"
    primary_key = ""
    
    def path(self, ...):
        if not self.sync_completed_date:
            # First sync - get all historical data
            start_year = date.today().year - 30
            start_date = f"{start_year}-01-01"
        else:
            # Subsequent syncs - get only previous fiscal year data
            start_date, end_date = self.get_fiscal_year_dates(self.fiscal_month)`
        return (
            f"/reports/{self.report_name}&"
            f"start_date={start_date}&"
            f"end_date={end_date}&"
            f"columns={self.output_columns}"
        )

Issue

When I set the stream to "Full Refresh | Overwrite" in the UI:
First sync works correctly (pulls all historical data)
Second sync:
Connector correctly pulls only previous fiscal year data
But normalization still performs overwrite, clearing all historical data
Results in losing data from before the previous fiscal year

Question

How can I control normalization behavior to append data after first sync, even when UI is set to "Full Refresh | Overwrite"?
Is there a recommended pattern for implementing this kind of mixed sync behaviour (full refresh first, then always append)?

👋 Hi, I’m looking for a developer to create a custom destination connector for Airbyte OSS. The goal is to build a mock/test connector using a MockAPI service as the target, serving as a POC. The connector should: • Be developed with the Airbyte CDK (Python or Java). • Simulate writing data into a mock API (e.g., MockAPI.io or similar). • Include a simple field mapping system (similar to the Airbyte HubSpot destination connector), allowing basic source → target field mapping. • Follow the Airbyte protocol standards (handling streams, catalog, normalization of messages, etc.). 🎯 The purpose is to have a working base connector that I can later extend and adapt to real-world destinations.

Subject: Access to GCP_GSM_CREDENTIALS for Local Testing Hi, just to confirm my understanding: From the answer I got from AI, it seems that GCP_GSM_CREDENTIALS used by Airbyte are not available for local use, and there isn’t a process for contributors to request them. Instead, for local testing, I’d need to set up my own GCP project and credentials in Google Secret Manager. Can you confirm that this is correct—that Airbyte’s CI credentials are CI-only and not something community contributors can obtain? Thanks!

Hi Airbyte Team/Community, I want to use https://airbytehq.github.io/PyAirbyte/airbyte.html and develop Field level Selection and support of Scheduling Using cdk . I see currently we have way to select Stream but there is no way to select fields in native connector builder code . This limitations block us creating dynamic field configuration and object during runtime airbyte connection creation for given source and destination My template code please refer below

import airbyte as ab
from airbyte.cloud import CloudWorkspace
from airbyte.cloud.connectors import CloudSource
from airbyte_api import Workspaces
from anyio.to_interpreter import run_sync

source: ab.Source = ab.get_source("source-hubspot",version="5.8.20", docker_image=True, install_if_missing=True)


source.set_config(
    config={
        "credentials": {
            "credentials_title": "Private App Credentials",
            "access_token": "[token]"
        }
    }
)

source.select_streams(streams=["contacts"])

source.read()

# Optionally, set a custom cursor field
source.set_cursor_key("contacts", "updatedAt")


#read_result  = source.read()
destination:ab.Destination = ab.get_destination("destination-s3")


destination.set_config(
    config={
        "s3_bucket_region": "us-east-1",
        "format": {
            "format_type": "JSONL",
            "flattening": "No flattening"
        },
        "access_key_id": "ACCESS KEY",
        "secret_access_key": "SECRET KEY",
        "s3_bucket_name": "sumit-digi",
        "s3_bucket_path": "data_sync/hubspot_contacts/data",
        "file_name_pattern": "{date:yyyy_MM}_{timestamp}_{part_number}.{sync_id}"

    }
)


workspace = CloudWorkspace(client_secret="secret",
                           client_id="client id",
                           workspace_id="workspace id",
                           api_root="<https://api.airbyte.com/v1>",
                           )


import uuid
cloudsource:CloudSource = workspace.deploy_source(source=source,name="sumit-final")
clouddest = workspace.deploy_destination(destination=destination, name="raina-final")
conn = workspace.deploy_connection(connection_name="sumit-raina-vastika",source=cloudsource,selected_streams=["contacts"],destination=clouddest)

conn.run_sync()

print("connection created")

Here we can select streams and set to source

source.select_streams(streams=["contacts"])