Does airbyte have a workday connector being developed?

Is it possible to override the url_base parameter in the manifest.yaml file for a particular stream?

Does anyone know what is causing this issue: Issue produced while running python acceptance test python -m pytest integration_tests -p integration_tests.acceptance Error while fetching server API version: HTTPConnection.request() got an unexpected keyword argument 'chunked'

Hi, does anyone know why I got the "non-json response" when I tried to create a custom connector in the Airbyte UI? I created the same connector in localhost and tested the endpoint in Postman, both of them are working fine.

How do I start writing a custom S3 data connector for the open source Airbyte?

Hi, team. I'm one of the

clickhouse destination

maintainers. One on

clickhouse destination

customers has the following error:

Missing columns: '_airbyte_unique_key' while processing query: 'SELECT number, _deleted, _creationTime, _ab_cdc_updated_at, _ab_cdc_deleted_at, _ab_cdc_lsn, _id, _ts, _airbyte_ab_id, _airbyte_emitted_at, _airbyte_no

on a subsequent run with

Incremental | Deduped + history

strategy. What could be a reason, considering that Airbyte itself creates the table? How can we debug it?

how would I set a record filter if i want to only display the records which have the selected field to be not empthy in my manifest.yaml file

Hello, We are a SaaS company that is looking to do mirroring for contents and content-permissions from external systems (SharePoint, Salesforce) for our customers to allow content searching. We want to do it in 2 phases: • Full content and permission pull at some cadence • Fast Data Refresh using webhooks, incremental updates, push APIs to maintain data freshness Is the open-sourced Airbyte suited for this task? How so? Thanks!

I’m looking into implementing Vitally through the Python CDK instead of using the existing low-code version, there is one endpoint where you need to query all parent entities before you can query each parent’s children (conversation -> messages). What are some good example implementations for this that don’t run into rate limit issues and run optimally?

Hi everyone, I am working on a source connector that pulls data from a system with a varying amount of streams. I created a generator function to create the stream objects and overwrote the

get_json_schema

function to fetch the correct json - which seems to work fine. I am now having issues to dynamically adjust the catalog. I checked the

streams

list in the

def streams(self, config: Mapping[str, Any]) -> List[Stream]:

function and it contains the object for the generated streams. This might be just an issue when testing it in the CLI and specifying a catalog for testing through the

--catalog integration_tests/configured_catalog.json

parameter but I am wondering if I missed anything. Thanks in advance for any help!

Hi everyone, I created a custom connector couple days ago, if I changed the

source.py

and

streams.py

files of the custom connector, do I need to create a new connector or just use

docker push

to update the connector? Thank you.

Hello, I have encountered problems while using source-s3. I hope to get some help with this

where does airbyte store the csv files when we use the local csv as a destination??

Hey, what version of the amazon ads api are you guys using? Still 2 or is it already updated to 3?

My sync fails with error “Checking source connection failed…” I test connection for Bing Ads source and BigQuery destination separately and it passes without error. Does anyone have the same issue?

hello airbyte gang - I am building a custom connector for Shipbob and I am able to successfully build the connector and transformations but I am only seeing a fraction of the number records that I am expecting. has anyone else run into an issue like this?

Hello guys! - I was following an issue/request about AWS s3 destination. The issue was about writing file in s3 partitions to increase performance in the next ETL of our pipelines. Any updates on this matter?

Do you happen to know when the Apple App Store connector could be available on the cloud? I was running it on my iMac for months with great success however, when I upgraded to the 4.x.x release it stopped working, and I’ve been using Precog.com to pull this data since then.

When developing a source with the Python CDK, what is the best way to prefix a stream and thereby group it in the user Interface?

Hi everyone, following connection is used: Stripe 3.4.1 -> Snowflake 0.4.63 I'm using Incremental/Append Mode for most of the streams. The cursor is based on "created". Problem: With every Sync (once per Day), records will be written in the table, which have been written with the sync the day before. So e.g. if a promotion code is emitted by the sync on day 1, it will also be emitted on day 2, even if there was no change at all on this entity. Does anyone have a solution on this behaviour, so that the sync does not add already existing records to the table?

Hello! Please let me know if this is the wrong channel to post this question in. Is there a way to log the meta data for when a source schema has changed? I'm looking to build a history table to see data around tables added/removed and fields in those tables that have been added/removed. Thank you!

What is the best approach to logging and reporting errors in a connector (using python CDK)?

Hi everyone. I’m trying to build a custom source connector using Connector Builder UI. The structure of API request should be as follows:

{
  "id": "1",
  "method": "SerpstatKeywordProcedure.getKeywords",
  "params": {
    "keyword": "iphone",
    "se": "g_us",
    "minusKeywords": [
      "app",
      "apple"
    ],
    "filters": {
      "lang": "en"
    },
    "sort": {
      "region_queries_count": "asc"
    },
    "page": 1,
    "size": 2
  }
}

I can use Request Body in the UI to set key-value pairs like “id”: “1” but for params object it doesn’t work. Is it possible to set nested objects in API request body in Connector Builder UI?

Hi everyone!✋ 🧠I am breaking my head and searching for a solution! 🙏 We use Airbyte❤️airbyte to download and manage third party data (e.g. zendesk of a client company). But it is very hard to educate clients on how to create certain API keys, etc. I wonder, are there alternatives on how to get the credentials at the first place? Is there a solution/company that would take care of authorizing the users and forwarding the credentials our way? Thanks everyoneairbyte rocket PS, apologies in case it is a wrong channel to ask🙈

I am using a snowflake connector for source. Connection tested okay, Can log in using the user separately and read all the tables too, but when creating a connection there is no stream show up from source (Snowflake)

Hi everyone, I’ve created my first PR for a new source connector : https://github.com/airbytehq/airbyte/pull/25790 I don’t know what is missing to be review if someone can help me on this PR Thanks in advance for your help

Hi All, I hope I'm in the right channel for this question, please say so if I'm not 🙂 I'm looking to integrate airbyte for a client and they have asked me to showcase the capabilities by loading data from a dynamics data lake into snowflake. The data in the data lake is on blob storage and is a combination of CDM (common data model) json files which describe the tables, and underlying csv's with the data, but without headers. Is it possible to create a connector that can somehow parse these json files and then read the csv's?

Hey guys, it seems the steps in https://docs.airbyte.com/connector-development/testing-connectors/testing-a-local-catalog-in-development/#steps don't work anymore. The

generateOssConnectorCatalog

task doesn't exist. Is there a new way of generating the catalog?

Has anyone started to work on a

skio

connector?