Hi team, I am trying to deploy airbyte on kubernetes . While configuring database I am getting an error Unable to create SQL database : temporal. But in my RDS instance I already have database with name temporal and temporal_visibility used by other applications . Is it possible to change the database names in airbyte ? Can I use names like airbyte_temporal and airbyte_temporal_visibility instead of temporal and temporal_visibility?

Hi Team, I'm using airbyte to sync data to MySQL destination, as the document mentioned, MySQL needs enable

local_infile

. But it's some security issues so we decide disable this feature, is there other way that without

local_infile

?

Hi all, It seems to me that there is a small issue with the Google Ads - Campaigns - Incremental | deduped connector.

segments.hour

field was added in version 0.2.3, which is great, but it is not in the primary keys when we setup incremental sync. This means that, when applying normalization, for a fixed

segments_date

and

campaign_id

, the final campaigns table contains only one of the

segments_hour

out of the 24 possible (from 0 to 23). I found a workaround by working with the

campaigns_scd

table, but if I had not seen this issue I would have been working on wrong data, so I think it's worth the warning here! Is there a planned fix for this issue? Thanks!

Hi All, Would need help in getting airbyte version

Why is there no source of dameng? Can someone help me out?

Hi guys! Good morning! Can someone help me with this question about PBA? When I create a Facebook Ads connection I wanted it to have the default streams, just like as I did from the UI. How can I achieve that??

Hi, WHen I run airbyte on Kubernetes with helm, how do I configure for example that it has multiple workers running etc?

the docs don’t really specify this at all

Hi Team, I use documentation https://docs.airbyte.com/deploying-airbyte/on-gcp-compute-engine/ to setup airbyte on GCP VM. After execution without error below I cannot access localhost:8000. I use MacOS.

gcloud --project=$PROJECT_ID beta compute ssh $INSTANCE_NAME -- -L 8000:localhost:8000 -N -f

Hey guys, good morning, I'm running airbyte on EKS, and I would like to know how configure like nodeport to reach the web-service from outside

hi team, I am trying to use airbyte with a custom database for temporal and temporal_visibility. Can anyone please provide the schema for these databases.

Priority of Operations

Hello there 👋, I have 1000+ connectors running on Airbyte but jobs seems never ending. Very quickly, the source and destination pods are complete but the dbt normalization pod takes hours of delay before triggering. I think it's because the dbt normalization operation have a low priority so I have to wait for all source / destination to run before launching normalization and finally have successful syncs. It is a bad UX as it seems to me that syncs are stuck; and at once, out of the blue, they all succeed. Do you know if we can setup priority of these normalization pod so they launch as soon as the destination is finished?

hey. i just joined this slack. i submitted an issue on github and hacked a PR on the actual connector. it's obviously breaking because it changes the output format of the tap but i don't know how much disruption it could cause. no idea what the process is 🙂

Hello, I am modifying a connector to a REST API, built using the python CDK. The Tickets resource (/tickets) has a child resource (/tickets/<id>/survey). What is the recommended practice for getting this child resource information?

couldn't get into airbyte and spent many hours this week trying to figure out why before I found note on run-ab-platform.sh. Using GCP. I have an instance that has been live for months, but upon finally getting in, I see no connections in dashboard at all. How do I get them back in my dash as I need to manage asap?

After upgrading to the latest helm chart version from 0.42.4 to 0.44.7 I'm getting the following errors:

airbyte sql schema version compatibility check failed "temporal". Expected version: 1.6 cannot be greater than Actual version: 1.4

any idea? I'd expect that the bootloader is responsible for upgrading the schema version and doing the migration?

Hello everyone, and happy Thursday! I’m @Jerri Comeau (Airbyte) from the Community Assistance team, and I’m asking for a little bit of time from you, the community members using this channel, on how to improve the experience of getting help (and helping others) here. We’ve created a small survey for folks: Weekend survey time! We appreciate all the feedback everyone is giving on the answers coming through. We’d also like to get your thoughts on how it’s going, so we created a little survey: https://forms.gle/ziEuXHLb8NPcXXfy7 We’ll be closing the survey on Tuesday so get your feedback in ASAP! If you feel inclined to fill out the survey, we’re also doing a raffle for some Airbyte merch as a thank you for your time and attention.

• Is this your first time deploying Airbyte?: No • OS Version / Instance: Debian • Memory / Disk: 32GB memory / 50GB disk • Deployment: docker-compose • Airbyte Version: 0.40.26 • Source name/version: source-facebook-marketing 0.2.79 • Step: retry_pattern / reduce_request_record_limit() method • Description: When I run a Sync on the “Activities” Stream, the batch request would fail sometimes due to the limit parameter set to 100. The

reduce_request_record_limit()

method should divide the limit by 2 and try again with the limit at 50. But this method does not work with the Activities stream and the limit parameter stays at 100 leading to another API error.

** Custom connector integration test error ** Hi Airbyte folks. I’m trying to create a new HTTP source, using the CDK. I followed the guide, connected to the source with

python main.py check --config secrets/config.json

command and got the right response after running

python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json

. Now, after creating the docker image with this command:

docker build . -t airbyte/source-tidsbanken-api:dev --no-cache

and testing it with the command below:

python -m pytest integration_tests -p integration_tests.acceptance

I got one error, that is quite bizarre and is referring to the incorrect primary key, even though I checked three times, it is correct. Any ideas about what am I missing here?

Hi, I'm trying to set up airbyte on my local machine, but I'm unable to create a working s3 connection due to

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)

. Full stacktrace in description. Anyone got any pointers how to deal with that ? Thanks in advance!

Hi Team, I am trying to set up airbyte locally in windows machine. When I ran docker-compose up I am stuck at airbyte-temporal. I am using WSL, Ubuntu 20.04(LTS) set. I see there were some issues regarding this earlier in this group, after going through the comments I find it a resource issue, am I right. In that case, how do I proceed? Detailed steps would be preferred. Thanks in advance.

Hey Guys! I want this one to run a daily full refresh - overwrite and the end-date parameter should always be yesterday. How do I get this done in the connector builder ui?

Hi Airbyte team, where is the good channel to request help to configure Bing Ads connector? I followed the text instruction but still getting errors

Hi team i need help. Env: server : docker on AWS ec2 source: Mysql (AWS rds aurora ) destination: Snowflake Problem: • io.debezium.DebeziumException: Failed to deserialize data of EventHeaderV4{timestamp=1678066339000, eventType=EXT_UPDATE_ROWS, serverId=1202208725, headerLength=19, dataLength=7889, nextPosition=80846231, flags=0} I try to change DB variable slave_net_timeout and thread_pool_idle_timeout. but AWS RDS not allow change variable. (cc. debezium_faq ) Is it my DB binlog problem? how can I fix it? I'm sharing Full Log in this thread I'm really bad at English, sry 😢

I saw this note in the Airbyte Postgres source documentation:

The Postgres source performs best on small databases (under 100GB).

What if my DB is an order of magnitude larger?

I already know, thanks

I added a new source Dameng, and I made an error when compiling, and the error message is shown in the figure, what is the reason for this?

【/airbyte/base.sh】 What’s going on with this file

I tried adding transformation but nothing is happening when I click add transformation button