https://linen.dev logo
Join Slack
Powered by
# ask-ai
  • t

    Temitope Sam

    07/17/2025, 12:20 PM
    are there specific kind of data mongodb can take? I created a MongoDB destination and tried to sync a slack + MongoDB connection but it didn't sync @kapa.ai
    k
    • 2
    • 1
  • i

    Ishan Anilbhai Koradiya

    07/17/2025, 12:36 PM
    Hi @kapa.ai in our temporal pod i am seeing this error Unable to load configuration: config file corrupted: yaml: line 17: found unknown escape character. My password does have a backslash
    k
    • 2
    • 10
  • p

    Prabu Dzaky

    07/17/2025, 12:57 PM
    @kapa.ai I got 500 internal server error to
    /api/v1/web_backend/connections/update
    when trying to update cron schedule. I just did a downgrade from 1.7.1 to 1.6.2 because 1.7.1 introduce API breaking change.
    k
    • 2
    • 1
  • i

    Ishan Anilbhai Koradiya

    07/17/2025, 1:13 PM
    @kapa.ai i am seeing a lot of times out for schema change updates using the api
    k
    • 2
    • 4
  • m

    Martin Brummerstedt

    07/17/2025, 1:14 PM
    @kapa.ai I get the following error when setting up our Google Analytics (GA4) connector: An unknown error occurred. (HTTP 504) We are running Airbyte community on a VM. All other connectors work and the Google analytics error only occurs when multiple properties are included which makes the initial checks take longer. I can see that the issue happens after 60 seconds. We have noticed that the issue is on the front end, but when checking backend logs, I see this
    Copy code
    172.30.0.1 - - [17/Jul/2025:13:05:52 +0000] "POST /api/v1/sources/check_connection HTTP/1.0" 200 8780 "<https://thevintagebar.com/workspaces/e53acc36-0cae-4282-9f4e-b81a40f7a160/source/26d1f8e9-dd40-4946-a290-304d69e46881>" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0" 1710 62.246 [airbyte-abctl-airbyte-abctl-airbyte-webapp-svc-http] [] 10.244.0.229:8080 8780 62.246 200 ff28c396ab4ab55feb18e1773dff3e9e
    nginx ingress is waiting for 62 seconds and correctly getting response. I assume that workers will work fine Can you help me debug how to solve this?
    k
    • 2
    • 1
  • b

    Brian H

    07/17/2025, 1:17 PM
    @kapa.ai - how can I best implement a custom self signed JWT authentication flow in your tool?
    k
    • 2
    • 1
  • t

    Temitope Sam

    07/17/2025, 1:25 PM
    @kapa.ai so if I decide to use snowflake instead and provided it with a schema, warehouse, and database that are not already created, will airbyte create them for me
    k
    • 2
    • 1
  • p

    Pablo Martin Calvo

    07/17/2025, 1:28 PM
    @kapa.ai Is there a way to retrieve the service account that was used to set up a google sheets source?
    k
    • 2
    • 4
  • m

    Mariano Benedettini

    07/17/2025, 1:41 PM
    @kapa.ai experiencing
    message='activity ScheduleToClose timeout', timeoutType=TIMEOUT_TYPE_SCHEDULE_TO_CLOSE
    in a very long running sync, any advice?
    k
    • 2
    • 7
  • o

    Omar Ahmed Mohamed Abdelkhalek

    07/17/2025, 1:47 PM
    @kapa.ai There is something wrong with our scd tables, for example they have missing unique values while the final deduplicated table does contain accurate up to date data
    k
    • 2
    • 1
  • h

    Hari Haran R

    07/17/2025, 2:04 PM
    @kapa.ai i'm building a connector. it's a cookie based authentication, how to keep refreshing the cookie if expired
    k
    • 2
    • 7
  • s

    Slackbot

    07/17/2025, 2:23 PM
    This message was deleted.
    k
    a
    • 3
    • 11
  • g

    Gabriele Baronio

    07/17/2025, 2:32 PM
    @kapa.ai I'm working on Postgres DB connectors synced to a Redshfit cluster; I'm analyzing results and I can't make much sense of the two columns: _ab_cdc_updated_at and _airbyte_extracted_at; sometimes one if after the other, sometimes one is before the other. On which of these should I base incremental updates to subsequent tables on?
    k
    • 2
    • 4
  • a

    Ashish Kumar Mohanty

    07/17/2025, 2:48 PM
    @kapa.ai Can i add my custom connector created using python cdk to airbyte cloud?
    k
    • 2
    • 3
  • t

    Temitope Sam

    07/17/2025, 3:35 PM
    can I create multiple connections to a destination @kapa.ai
    k
    • 2
    • 1
  • m

    mike Trienis

    07/17/2025, 5:52 PM
    @kapa.ai how does airbyte automatically detect a schema, what if a field only appears one out of multiple records? How to ensure it’s included?
    k
    • 2
    • 25
  • m

    Martin Andonov

    07/17/2025, 6:54 PM
    @kapa.ai Can you please help me set up the Builder configuration correctly to achieve the below pull? Goal: I have a response that contains a nested array in a column called vendorItems. I want to pull only the values in the nested array, without the remaining data. Approach: I'm attempting to use the Record Selector in the Builder UI to drill into the nested array. Below is my current setup: type: RecordSelector extractor: type: DpathExtractor field_path: - '*' - result - '*' - results Outcome: I get the error "No records could be extracted from the response. Check the Response tab to see the raw response. If this looks correct, make sure the Record Selector is configured correctly." Response: Below is a sample of the API response for reference: { "status": 200, "body": [ { "vendorId": "928b864a-9656-4dea-822f-d7cc4e94430f", "contactName": "", "currencyId": "88285281-a77e-4e96-ad51-6cfa2d0bbd31", "customFields": { "custom1": "", "custom2": "", "custom3": "", "custom4": "", "custom5": "", "custom6": "", "custom7": "", "custom8": "", "custom9": "", "custom10": "" }, "defaultAddressId": null, "defaultCarrier": "", "defaultPaymentMethod": "", "defaultPaymentTermsId": null, "discount": "0.00", "email": "", "fax": "", "isActive": true, "isTaxInclusivePricing": false, "lastModifiedById": "89aebc56-db26-4ee8-8325-0b9518519e94", "lastModifiedDttm": "2024-02-12T215104.2057167+00:00", "leadTimeDays": null, "name": "1000Bulbs", "phone": "", "remarks": "", "taxingSchemeId": null, "timestamp": "0000000004CA71B7", "website": "", "vendorItems": [] }, { "vendorId": "6677fe2e-081f-485b-baf0-e0de68787cd6", "contactName": "", "currencyId": "88285281-a77e-4e96-ad51-6cfa2d0bbd31", "customFields": { "custom1": "", "custom2": "", "custom3": "", "custom4": "", "custom5": "", "custom6": "", "custom7": "", "custom8": "", "custom9": "", "custom10": "" }, "defaultAddressId": null, "defaultCarrier": "", "defaultPaymentMethod": "", "defaultPaymentTermsId": null, "discount": "0.00", "email": "", "fax": "", "isActive": true, "isTaxInclusivePricing": false, "lastModifiedById": "89aebc56-db26-4ee8-8325-0b9518519e94", "lastModifiedDttm": "2023-02-28T195126.5667782+00:00", "leadTimeDays": null, "name": "220 Electronics", "phone": "", "remarks": "", "taxingSchemeId": null, "timestamp": "0000000003EF04F6", "website": "", "vendorItems": [] }, { "vendorId": "07c9f0e3-d7f5-41cc-93c8-aee3a5913d2b", "contactName": "", "currencyId": "88285281-a77e-4e96-ad51-6cfa2d0bbd31", "customFields": { "custom1": "", "custom2": "", "custom3": "", "custom4": "", "custom5": "", "custom6": "", "custom7": "", "custom8": "", "custom9": "", "custom10": "" }, "defaultAddressId": null, "defaultCarrier": "", "defaultPaymentMethod": "", "defaultPaymentTermsId": null, "discount": "0.00", "email": "", "fax": "", "isActive": true, "isTaxInclusivePricing": false, "lastModifiedById": "d5da1d41-953c-4896-a41c-2e133e3dc6d0", "lastModifiedDttm": "2016-07-21T134201.62+00:00", "leadTimeDays": null, "name": "220togo", "phone": "", "remarks": "", "taxingSchemeId": null, "timestamp": "000000000027A931", "website": "", "vendorItems": [ { "vendorItemId": "a322e7d1-fe27-4926-95e9-5ec342f6a4a1", "cost": "0.00000", "leadTimeDays": null, "lineNum": null, "productId": "1d2d5fa8-3e05-4113-8b3e-1612880dae53", "timestamp": "0000000001520AC6", "vendorId": "07c9f0e3-d7f5-41cc-93c8-aee3a5913d2b", "vendorItemCode": "" }, { "vendorItemId": "039ea565-cd6e-4c8c-9b02-3e12e77b6d65", "cost": "69.95000", "leadTimeDays": null, "lineNum": null, "productId": "450b5f66-a3d6-4ba4-bbe5-8acd02bfda7f", "timestamp": "000000000027D277", "vendorId": "07c9f0e3-d7f5-41cc-93c8-aee3a5913d2b", "vendorItemCode": "" }, { "vendorItemId": "a4df002e-e812-4423-8a41-637d62c241d3", "cost": "0.00000", "leadTimeDays": null, "lineNum": null, "productId": "7bdd5145-e6df-42dc-8be9-e9513cad9942", "timestamp": "0000000001520613", "vendorId": "07c9f0e3-d7f5-41cc-93c8-aee3a5913d2b", "vendorItemCode": "" }, { "vendorItemId": "d78e43ec-2365-4409-be10-da34006c5ae9", "cost": "0.00000", "leadTimeDays": null, "lineNum": null, "productId": "1515cb1b-9bf8-4f91-932d-ed903402bee4", "timestamp": "0000000001520643", "vendorId": "07c9f0e3-d7f5-41cc-93c8-aee3a5913d2b", "vendorItemCode": "" }, { "vendorItemId": "40664f08-56f2-4b8a-a843-e4df9d23d637", "cost": "129.00000", "leadTimeDays": null, "lineNum": 1, "productId": "a2dd5938-9421-435f-b1ff-34711c2180aa", "timestamp": "00000000034DCB12", "vendorId": "07c9f0e3-d7f5-41cc-93c8-aee3a5913d2b", "vendorItemCode": "Whirlpool 25 Liter microwave with Grill #WMGO9SDE" } ] },
    k
    • 2
    • 4
  • s

    Sean Wallace

    07/17/2025, 7:26 PM
    s3 datalake destination error Equality field ids shouldn't be null or empty when creating equality-delete writer It is a table with 50 records and the pkey is a unique number on the table, not null or blank
    k
    • 2
    • 2
  • m

    Mariano Benedettini

    07/17/2025, 8:23 PM
    1. @kapa.ai is there a way to modify the default 10000 rows that airbyte batches up before unloading that chunk to the destination
    k
    • 2
    • 4
  • i

    Ioannis Nikolakis

    07/17/2025, 9:49 PM
    @kapa.ai where can i find the AIRBYTE_API_KEY on the airbyte open source community edition service ?
    k
    • 2
    • 1
  • l

    Lorenzo

    07/17/2025, 10:21 PM
    @kapa.ai is it possible to add a new stream to an existing connector?
    k
    • 2
    • 4
  • p

    Pol Ancochea

    07/18/2025, 6:03 AM
    @kapa.ai Hi, when connecting to salesforce and click "Authenticate your salesforce account" a window appear and dissapears and get the message OAuth login failed: errors.http.internalServerError
    k
    • 2
    • 1
  • p

    Pol Ancochea

    07/18/2025, 6:07 AM
    @kapa.ai We have many postgresql connectors, some for very big databases (2 bilion rows transfered during the inital load) and some small ones. Some databases (big and small) work flawlessly, but others, at some point, start reporting the following error: Saved offset is before replication slot's confirmed lsn. Please reset the connection, and then increase WAL retention and/or increase sync frequency to prevent this from happening in the future. See https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting#under-cdc-incremental-mode-there-are-still-full-refresh-syncs for more details. And we have to re-create the replication slot and the publication.
    k
    • 2
    • 8
  • r

    Rahul

    07/18/2025, 6:53 AM
    @kapa.ai does airbyte have an Amazon S3 connection?
    k
    • 2
    • 7
  • o

    Onur Musaoglu

    07/18/2025, 7:53 AM
    @kapa.ai I have a manifest only connector and want to add some environment variables to docker image I generated. I know this is possible for python connectors using build_customization.py file. However, build_customization.py is not recognized by airbyte-ci for manifest only connectors. How can I add environment variables to my connector image for manifest only connectors?
    k
    • 2
    • 4
  • p

    Pablo Martin Calvo

    07/18/2025, 8:11 AM
    @kapa.ai I have deployed OSS Airbyte on a kubernetes cluster. When trying to setup a new connection, I'm getting
    An unknown error occurred. (HTTP 524)
    . How can I check what's the problem? The pod that is launched is called
    le-sheets-discover-f2ba9114-64d4-43b0-a758-44f81f4f3c2e-0-anfaz
    in Lens (it's supposed to connect a google sheets to a PostgreSQL). I don't get the logs of the pod, what could be the problem? Is there a way to do the connection differently?
    k
    • 2
    • 10
  • h

    Hasan Özçiftci

    07/18/2025, 8:27 AM
    @kapa.ai getting Virtual host domain is required while accessing a specific bucket while connection s3 datalake destination connection over nessie catalog my endpoint is <https://<name>.<region>.myhuaweicloud.com>
    k
    • 2
    • 1
  • l

    loganeales

    07/18/2025, 8:28 AM
    @kapa.ai trying upgrade from 1.6.0 to 1.7.0. using abctl. error encountered: " ERROR airbyte-bootloader: Failed to verify database connection. "
    k
    • 2
    • 11
  • u

    原田貴弘

    07/18/2025, 9:36 AM
    I am using the OSS version of Airbyte. Please tell me the following. After installation and when you log in for the first time, a screen like this appears and asks you to set your email address and organization name. I would like to change this email address. How can I do this?
  • h

    Hasan Özçiftci

    07/18/2025, 10:52 AM
    @kapa.ai Could not connect with provided configuration. Error: The specified key does not exist. (Service: S3, Status Code: 404, Request ID: 000001981D261D578049B1B29BDA84CA, Extended Request ID: I’m using the Iceberg S3 Data Lake destination connector in Airbyte with a Nessie REST catalog backend. My configuration looks like this: bucket: dev I’ve configured a
    warehouse_location
    like: s3://dev/airbyte-test/ However, it looks like Airbyte is incorrectly concatenating the bucket name into the path, resulting in: s3://dev/dev/airbyte-test/ This causes a
    404
    or “The specified key does not exist” error. Please help me understand: 1. Why is Airbyte duplicating the bucket name in the S3 path? 2. How should I correctly set the
    warehouse_location
    or related config to avoid this path concatenation issue? 3. How does the connector internally resolve the full S3 URI, and what config (bucket name vs. warehouse URI) takes precedence?
    k
    • 2
    • 4
1...4445464748Latest