https://linen.dev logo
Join Slack
Powered by
# help-connector-development
  • g

    Gabriel Levine

    05/30/2023, 10:03 PM
    I’m using the Connector Builder (Airbyte 0.44.5) to build a low-code connector. The when an object is occasionally null, the schema detection will produce a specification like this:
    Copy code
    "some_object": {
          "anyOf": [
            {
              "type": "null"
            },
            {
              "properties": {
                "id": {
                  "type": "integer"
                },
                "name": {
                  "type": "string"
                }
              },
              "type": "object"
            }
          ]
        }
    as opposed to when there are no nulls, it will produce:
    Copy code
    "some_object": {
          "properties": {
            "id": {
              "type": "integer"
            },
            "name": {
              "type": "string"
            }
          },
          "type": "object"
        }
    The second is correctly recognized as an object, while the first is classed as Unknown which prevents it being synced. I can’t tell if this comment is highlighting the same issue: https://github.com/airbytehq/airbyte/issues/25669#issuecomment-1559850741
    k
    s
    j
    • 4
    • 7
  • c

    Chidambara Ganapathy

    05/31/2023, 6:27 AM
    Hi all, any update on QuickBooks online issue?
    k
    • 2
    • 2
  • r

    Rishav Sinha

    05/31/2023, 6:37 AM
    Hi everyone, i am building custom connector for ServiceDesk Plus and having trouble in implementing pagination
    Copy code
    <https://mydomain.com/api/v3/endpoint?input_data={>"list_info":{"start_index":1,"row_count":"2000","get_total_count":true}
    this is how they expect us to pass data for pagination, can someone guide me on this please ?
    k
    j
    • 3
    • 12
  • c

    Chidambara Ganapathy

    05/31/2023, 7:38 AM
    Hi all, I am trying Airbyte connector builder UI to add an additional stream for wrike connector. Base URL: https://www.wrike.com/api/ Method: GET /folders https://www.wrike.com/api/v4/folders is working fine. The response returns multiple records along with a record named childIds as an array of values eg:
    Copy code
    "childIds": [
          "IEXX6MLXXXXXX",
          "IEXX6MLXXXXXX"
        ]
    The stream that I am trying to add is Method: GET /folders/{folderId},{folderId},... - up to 100 IDs where folderId is the childId Kindly let me know how to convert the childIds array into individual elements so that it can be passed into the request and get the response
    k
    j
    • 3
    • 5
  • h

    Harshil Dhariwal

    05/31/2023, 4:22 PM
    We are trying to create a DataBricks source connector. Local tests seems to be running fine. The connector was running fine until recently. Could someone help us understand the error we are receiving ?
    k
    • 2
    • 2
  • o

    Octavia Squidington III

    05/31/2023, 7:45 PM
    🔥 Office Hours starts in 15 minutes 🔥 Topic and schedule posted in #C045VK5AF54 octavia loves At 1PM PDT click here to join us on Zoom!
  • c

    Chidambara Ganapathy

    06/01/2023, 11:58 AM
    Hi Team, Is it possible to build AWS Cost Explorer source by using Airbyte connector builder? Thanks
    k
    j
    • 3
    • 6
  • o

    Octavia Squidington III

    06/01/2023, 1:45 PM
    🔥 Office Hours starts in 15 minutes 🔥 Topic and schedule posted in #C045VK5AF54 octavia loves At 16:00 CEST / 10am EDT click here to join us on Zoom octavia loves
  • j

    Jamshid Hashimi

    06/01/2023, 4:40 PM
    Hi, I am using the Builder to fetch data from here: http://openlibrary.org/subjects/technology.json (when I test fetch it, it says 12 records on the Airbyte) When I push this to the Typesense destination, it takes it as one document, instead of creating 12 documents in a collection. Is there a way to configure this on the Airbyte side? Thanks!
    k
    • 2
    • 3
  • w

    Wade Warren

    06/01/2023, 6:00 PM
    I'm building a Front integration. It is paginated via a
    page_token
    query param in a returned URL. Is there a way to only inject the date range query parameters in the first request? I ask because the next path includes the previous query parameters, so after a few paginations the path includes many date range params.
    Copy code
    "next": "<https://wingspan.api.frontapp.com/events?q[after][0]=1685577600&q[after][1]=1685577600&q[after][2]=1685577600&q[after][3]=1685577600&q[before][0]=1685641246&q[before][1]=1685641246&q[before][2]=1685641246&q[before][3]=1685641246&page_token=2d018a5809eb90d349bc08c52cb1f4988be5b03b8262ad76bb8228d1fe6b9cf6770311505076bbcb11f23f82ce05881a8b015c46b2ebf9419adec233df2863bc>"
    j
    • 2
    • 8
  • l

    Luis Valor

    06/01/2023, 7:37 PM
    Hello All - I'm trying to set up Metabase as a source in Airbyte. I'm having trouble completing this process because Airbyte is looking for an API endpoint that Metabase removed a week ago. The below are the default endpoints the connector is trying to hit. I am unable to specify which endpoints to hit until after the connection has been set up. However, since the connection keeps failing on the
    activity
    endpoint, which no longer exists, I'm sort of stuck with this connector. Is there anyway to specify which endpoints to hit, or a way to upgrade the connector? Currently using the latest version. 1.
    activity
    - deprecated by Metabase a week ago 2.
    card
    3.
    collections
    4.
    dashboard
    5.
    user
    k
    j
    • 3
    • 3
  • e

    Eduardo Kose

    06/01/2023, 9:33 PM
    Hi all, quick question Is it possible to add a label to a pod job using Helm? I can add annotations and tolerations, but what about labels?
    k
    • 2
    • 1
  • c

    Chuck Chan

    06/01/2023, 10:37 PM
    Not sure if I am asking in the right channel, but I've got a webhook of airbyte to slack, with notifications for when a sync job fails. Is there a way of configuring these messages, because they often point to the wrong url I.E. the actual url I am expecting to see would be http://airbyte-stg.eu/workplace-blah-blah-blah, as it is the hostname I have configured but the default url displayed is http://airbyte-eu-svc/workplace-blah-blah-blah, and it is of course not a valid link TLDR: the url that airbyte webhooks generate for slack is slightly different from my actual domain name
    🙏 1
    🫡 1
    k
    • 2
    • 2
  • d

    Disha

    06/01/2023, 11:56 PM
    Has anyone used self hosted github and run locally? Is there option where we can change url to organization url and pass credentials?
    k
    • 2
    • 2
  • d

    Disha

    06/01/2023, 11:58 PM
    Can we use self hosted github as a source when we run locally ? It bu defaults take to github.com is there way to modify it to organization url
    k
    • 2
    • 2
  • j

    Joey Taleño

    06/02/2023, 4:27 AM
    Hi Team! Quick question, is the new Airbyte Custom Connector Builder available in OSS version? https://airbyte.com/connector-development-kit
    k
    j
    • 3
    • 3
  • a

    Abhishek Battu

    06/02/2023, 9:14 AM
    We connected our google ads to air byte and we storing our data in big query,but we are get are getting wrong data from airbyte,can you help me to resolvev the issued
    k
    • 2
    • 3
  • c

    Chidambara Ganapathy

    06/02/2023, 9:21 AM
    Is OAuth disabled for Okta source connector in Airbyte?
    k
    • 2
    • 2
  • m

    Manish Khatri

    06/02/2023, 2:19 PM
    Hi, I’m trying to implement a CommerceTools connector using the Connector Builder UI on Airbyte Cloud. Is there a way to provide Basic Authentication as part of the
    client_credentials
    grant type on the UI? The API seems to expect this, otherwise it fails to get a token (
    401 Client Error: Unauthorized for url: https://….
    ) due to the fact it is unable to authenticate the request. https://docs.commercetools.com/api/authorization#client-credentials-flow
    k
    j
    • 3
    • 21
  • j

    Jake Kagan

    06/02/2023, 4:58 PM
    hey guys im not able to see a zoom recording - so hoping someone can help me out here: i made a change to one of the connectors that come with the deployment
    source-pardot
    and i spun up the docker image, but having a hard time finding it on the airbyte webui
    k
    • 2
    • 20
  • j

    Jason Pilon

    06/02/2023, 5:13 PM
    Hi, is there a way to add a custom image for the Custom Connector?
    m
    j
    • 3
    • 3
  • l

    Luis Valor

    06/02/2023, 6:11 PM
    Hi All - is the new Connector Builder available using the open source version of Airbyte?
    k
    a
    • 3
    • 3
  • j

    Jamshid Hashimi

    06/02/2023, 7:24 PM
    Hi All, when connecting a custom builder to typesense, I need to set
    Copy code
    "enable_nested_fields": true
    in the collection schema. Is there a way I can do that? typesense doesn’t allow updating that field after a collection is created.
    k
    • 2
    • 2
  • o

    Octavia Squidington III

    06/02/2023, 7:45 PM
    🔥 Community Office Hours starts in 15 minutes 🔥 At 1pm PDT click here to join us on Zoom!
  • a

    Abdul Hameed

    06/04/2023, 5:54 PM
    Hi Team, I am learning to build custom connector for HTTP API request, but facing this error when running generator.sh file I am using Docker, Windows 11, and git bash to run the commands
  • b

    Bert

    06/05/2023, 7:17 AM
    Hi All, I'm trying to find a way to load data from our microsoft dynamics data lake into snowflake. The data is stored in csv files without a header and an accompanying manifest json file that holds the information about the table. In spark I can use
    spark.read.format("com.microsoft.cdm")
    to read the format, in python I had to make my own parser to read the manifest and create a table based on this. Would it be possible to use airbyte to read the manifest to create a table, and check for schema drift, and to read the csv files when updated?
    k
    • 2
    • 2
  • r

    Robert

    06/05/2023, 7:21 AM
    Hi, I'm developing a destination connector in Python. If something went wrong in write function and I want to notify the framework to stop without any retry, should I raise a specific exception or something else?
    k
    • 2
    • 2
  • a

    Abdul Hameed

    06/05/2023, 8:20 AM
    Hi Team, I am learning to build custom connector for HTTP API request, but facing this error when running generator.sh file I am using Docker, Windows 11, and git bash to run the commands
  • a

    Abdul Hameed

    06/05/2023, 9:06 AM
    while trying to create a airbyte source connector facing this error Running generator... While trying to generate a connector, an error occurred on line 38 of generate.sh and the process aborted early. This is probably a bug.
    k
    • 2
    • 2
  • s

    Steven UZAN

    06/05/2023, 5:56 PM
    Hello there! Is it possible to change the timeout for a given connection?
    k
    m
    • 3
    • 9
1...111213...21Latest