Hi there, I’m writing a custom connector using the low-code CDK. How can I add the parent id as a column to a substream? Example: I have a path url like this:

courses/{{ stream_slice.course_id }}/analytics

The response contains no data about the

course_id

used, so I currently have no way to connect it to the correct course.

I had upgraded from version

0.39.42-alpha

to

0.40.30

But the state management was acting weird and it was not keeping track of the cursor for incremental syncs. It was doing a full data sync everytime. The worker was looking for cursor stats in the

state

table, but the table had outdated entries & the cursor stats were seen in the

jobs

table. Any idea how to fix this?

btw, i’m here to say i also want https://github.com/airbytehq/airbyte/issues/14391 fixed 😄

Hi, we need to run custom transformation inside airgapped on premise network, but we don't have access to GitHub, and azure repository we have access within network is not accepted. Are there any ways around it ? (url on screenshot just to show azure repository address format)

Hi, I'm trying to set up an Airtable source on OSS, but the instructions are not clear to me. It says "Authenticate your Airtable account", but I'm not sure where this option is... (see image) In addition, I've previously set up the Airtable source on Cloud using an API key, which was fine. When I try the same using OSS, I get the following error:

The connection tests failed.

ValueError("This authenticator expects a value under the ('credentials', 'access_token') field path. Please check your configuration structure or change the access_token_config_path value at initialization of this authenticator.")

Hi? We have been building Airbyte source connectors, mainly HTTP, with the Python CDK and it's been going great. I wanted to know whether it's plausible to build a HTTP destination connector. One that submits records, batching or otherwise, via an API POST request. Most of the destinations I've seen are to a DB or file and I'm assuming those use drivers of some sort? Thanks

Hi! I’m implementing a custom connector using the low-code CDK. I’m trying to get incremental sync working, however the field in the API which would make sense to stream slice on is in UNIX timestamp format. Is there a way to get the stream slicer to work with this? batman think

Hi! Quick question: I'm on v0.40.25 using helm and wondering why it goes up to v0.44.0-helm, but there is no realease logs after v0.40.32. Are this versions on alpha or beta? What is the difference?

Hey folks, do we have an ETA on the next Airbyte version release?

Can someone explain this error to me? I'm implementing a full refresh stream and not an incremental one

Hey, i want to work on the amazon seller partner connector but following the "developing-locally.md" i already get errors by just building the platform. (https://discuss.airbyte.io/t/error-building-clean-clone/3912) Is it possible to only build a connector and implement it in a local airbyte docker environment for testing? Or do i need the builded platform?

Does changing the parameters of a source automatically update the connection? If yes, then it seems to be buggy for the Amazon Ads Connector. For instance, I changed the StartDate from a specific date (2023-02-04) to an empty field (this should default to today, 2023-02-14) but the sync is working according to my old inputs. Is this a known issue?

Hello everyone! I’m Jerri Comeau, the manager of the Community Assistance Team, and I’m here to tell you about some big changes coming to our current Github repository structure and processes. Executive summary: 1. The current repo, airbytehq/airbyte, is being separated into two repos to simplify the deployment process both internally and externally. 2. Connectors will continue to live in airbytehq/airbyte and

docker-compose up

will still work from this directory. 3. Platform code is being consolidated in the new airbytehq/airbyte-platform repo and users will need to switch to this repo to actually start and run Airbyte. Detailed Information: In order for our Engineering, Product, and Design teams to more quickly and efficiently deploy new code, we decided that the best thing to do was to split up our repositories into two already-existing pieces: the connector functionality and the platform/deployment functionality. This will allow us to make isolated changes discreetly and improve the development workflow. We’ll be making this change on 20 February but we’re letting you know now so you can prepare your workflows ahead of the change. We have done our best to limit the impact of this change, and we want everyone to know ahead of time what we’re doing. We’ll also be reminding you again before the rollout happens on 20 February.. We’ll also be updating our documentation and walkthroughs to make sure they reflect the change. Any questions? Please don’t hesitate to let us know.

Hi Airbyte Team, I'm new at airbyte. Currently I have setup the connector which source is Http API(pokeapi) and destination is MySQL(8.0.28), and the data sync with basic normalization works fine. However, I want to customize column name for the destination, so I follow the documents (part 1 ~ 3) copy the normalization folder and make some modification then push the modified dbt project to my public GitHub. When I sync the data again it raise the exception

fatal: unable to access '<https://github.com/HYLin0524/airbyte_dbt_poc.git/>': server certificate verification failed. CAfile: none CRLfile: none

. How can I solve this issue ? Seems the CAfile not set correctly ? ***NOTE* How do I start the airbyte on my local: 1. my host is macOS BigSur with Docker Desktop 2. clone the airbyte source code 3. run

docker-compose up

Hi All,

Hopefully an easy question: After loading a table from S3 into Redshift serverless - how do I put some grants on the tables so that our dbt environment can then see the newly generated tables?

Are there any options (or recommended best practices) for handling the staging files left over after loading data to Snowflake internal storage? I saw the "Purge Staging Files and Tables" option for S3 staging areas, but couldn't find any information on the other storage options.

Hey Airbyte team, just wanted to flag that the ‘Check the docs’ link here goes to Facebook documentation instead of the Braze one.

Has anyone successfully retrieved a customEvent parameter from the Google Data API for GA4 property?

Hi All, @Guru Swamy and I are attempting to spin up a snapchat connector. It works fine when we do it locally, but when we try to create it on an EKS cluster it fails. Other connectors (for example FB and AWS) work fine, so it's unclear what the issue is. Finally we can't seem to find any logs using the admin tab on either on the server or in the scheduler. Is there another place we should be looking for logs? The output in the source reads like a Python error: "AttributeError("'NoneType' object has no attribute 'status_code') and there is cloud storage log path /workspace/our_workspace/0/logs.log but we aren't sure how to access those either, any insight here?

Hey Guys, I deployed Airbyte on a Google VM and am looking to trigger a sync using the API from an external server. I know it’s not safe to expose the Airbyte instance publicly, but could you give me a couple pointers on how people usually do it? The options I’m considering are: 1. Create a VPC and create another safe service that can forward the requests to the airbyte cluster 2. Mount a Queue, where I post messages and airbyte can poll to figure out if it needs to do anything (very IoT approach) Thank you!

I want to host airbyte open source on AWS EC2. Is it possible to connect to it using only Octavia CLI without using SSH?

Hi everyone! Has anyone who is using the Google Analytics 4 (GA4) source found a way to retrieve extensive amounts of records without getting the 427 error? (which indicates the API has reached its limit)

Has anyone integrated airbyte with Constant Contact?

I am using Google Cloud Engine to run Airbyte, is there a way to run python script automatically after sync is finished to send me an email or something? I want to let my team know that the sync was successful without having to manually check all the time.

One thing I don’t understand is what would a connector do with an endpoint in the progress of oauth2, specifically how connector handles web pop-up then retrieve auth token from it? much appreciate for any clues.

Hi team, I wanna to ask a question: How can I test my custom normaliztion image?

I am trying to calculate Ad Spend by date wise for TikTok Marketing using

company-data.titktok_company.ads_reports_daily_metrics

. But I don't see the date field in this table. How can I calculate AdSpend by date? Can anyone help me to point out which table I refer to for calculating AdSpend by date?

Hello, I have an issue with my S3 connector. When I try to sync I have the following error message :

Failure Origin: source, Message: The connector encountered an error while processing the file(s): lizeo/speedy/test_competitors/test_compet.csv. only single character unicode strings can be converted to Py_UCS4, got length

My file format settings in the connector is as in picture and the csv file format is as below:

"competitor_name","ean","latest_update","competitor_price"

"<http://oponeo.fr|oponeo.fr>","3528704686216","2022-12-02 16:12","78.43"

"<http://1001pneus.fr|1001pneus.fr>","3528704686216","2022-12-02 16:12","108.17"

Is there something wrong with the connector settings ? Thanks for your help

AppStore source not shown in Airbyte cloud. When it will be available?