Hi everyone! Has anyone encountered a bug were the setup guides stopped appearing in every single source and destination? I recently upgraded to airbyte 0.41.0 and they just stopped showing up on the right panel when creating a source or a destination. Anyone know how to fix this?

Using a fresh install of 0.41.0, creating a Snowflake source using the UI causes Java exception:

Is the Vertica connector being developed? This ad is on the website for the past 5+ months. If yes, who is developing? https://airbyte.com/connectors/vertica

Java exception creating Snowflake source: https://airbytehq.slack.com/archives/C021JANJ6TY/p1677864150946339. Debug help?

I just upgraded our dev cluster to run Airbyte 0.41.0 and now am not able to load the UI. /api/v1/web_backend/check_updates fails with an HTTP 500 error. I’ve attached the server logs that show the reason for the internal failure.

I came here because I have the same issue as @Sean Zicari -- I am just getting started with airbyte, deploying into k8s. The error logs on my end look the same. The issue appears to be it's trying to download a catalog from https://connectors.airbyte.com/oss_catalog.json but this URL is not returning JSON, it's trying to redirect to a login page.

Hey y’all! Happy Friday from the Community Assistance Team. As part of our efforts to improve responsiveness and followup, we’re currently in the process of testing not one but TWO ai assistants to help answer quick questions that have documented resources to share. We have created two private channels so that interested community members can help with our Beta Testing. If you’d like to be part of this, and put the test to these tools to see which is best, then please DM me with a note that you’d like to be part of testing. We’ll be kicking this off on Monday, March 6th at 8AM Pacific time, so any time before then just let me know and I’ll get you added in. If you didn’t see this before and it’s after Monday, March 6th at 8AM, but you still want to be part of the Beta, please DM me and I’ll add you to the channels! This Beta test will run until March 23rd, so please, put these tools through their paces. We appreciate your time, attention, and assistance while we work to make our community better.

Hi Airbyte Community 🙂 I have been dwelling into Airbyte since a couple of months. Currently facing a blocker. I am trying to sync data from mongo db to AWS s3 location. But the airbyte UI is not reading few columns that have data in them. I wish to understand the criteria(if any) that might force airbyte to skip columns. Any help will be highly helpful. cc: @Sachin Tripathi @Vaidehi Khalpada

Hi, All Is there any configuration to transfer data in large batches? source connector: MongoDB destination connector: ClickHouse

Just started playing with airbyte I’ve deployed it to eks, however I’ve learnt that nodeInstanceRole doesn’t have AmazonEBSCSIDriverPolicy by default I had to attach it to successfully deploy, writing it here it maybe useful to someone Also I’m new to the whole kubernetes world so I’m slow.

I need help understanding the root cause of a Java null pointer exception in testing connections. It happens when I create sources for BigQuery, Snowflake, and Clickhouse - so it’s something that happens independent of source type. I have multiple AirByte instances running on separate EKS clusters, all installed via helm. This error is only happening on one of them! There’s obviously something different about the configuration of the one that’s failing, but how can I find out what that is? Starting thread…

Issue with Mixpanel Source. I have scheduled Sync at 8 AM and 8 PM IST everyday. Surprisingly every day, 8 AM Sync is being failed and 8 PM sync is being successful. However, in both the cases, the data is being loaded. Attaching the sync history screenshot and the error message in the failed sync, and connector properties. Could someone help regarding this, please?

How do I fix these warnings?

2023-03-05 07:46:34 destination > Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-03-05 07:46:34 destination > Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword

I googled and saw that they occur for multiple destinations, but didn't see a way to address them or even what they mean.

Hi, I am facing an issue with Airbyte-supported data types. Airbyte fetch Timestamp as a String that is not right behaviour. Source: Microsoft SQL Server 2012 Destination: Clickhouse 22.2.2.1

Is there a good documentation about the docker compose .env file which all the options and possible values are available? Only thing i got is a on GitHub with what i think is all the options but would love to see if there is some possible values for it or definitions of it https://github.com/airbytehq/airbyte-platform/blob/main/airbyte-config/config-models/src/main/java/io/airbyte/config/EnvConfigs.java I think that one is with all the environment variables. but if somebody can point me to a list that would be helpful

Does anyone have an idea why would only 1 worker be busy vs. the others? Why would it need 6GB of ram when no sync is active (no job pods are up). Airbyte OSS / EKS / V.40.17

Hi, I am trying to start airbyte opensource in podman. Unfortunately some of the containers are not running. I have modified the run-ab-platform.sh file to start the containers in podman

Init container was not created in podman itself. When I try to login through the web UI, I receive 502 bad gateway error.

Is there any recommendations on deploying airbyte in podman or windows machine as standalone installation?

hi , I am trying to deploy airbyte on kubernetes but am getting this error

2023-03-06 12:32:05 ERROR o.f.c.i.l.s.Slf4jLog(error):57 - Migration of schema "public" to version "0.35.62.001 - AddJobIndices" failed! Changes successfully rolled back.
61
2023-03-06 12:32:05 ERROR i.a.b.Application(main):25 - Unable to bootstrap Airbyte environment.
60
org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException: Migration failed !

Hello there 👋 Not much a technical question, more like a 'best-practise' consideration asking: • What's the best 'Airbyte'-way to create a

computed field

in destination (mongodb here) from two fields in source ? Ideally, this method would also allow to normalize field on the fly (lower-case basically here) I saw a section about transformers, but it seems a bit engaging for dpt part (in term of maintenance over Airbyte releases). We've also considered a second-round browsing over all data in destination after a sucessful sync, but it seems redundant with what Airbyte is already doing If we could have a mixin around the both, without modifying connectors, it would be awesome! Please, let me know if you have done anything comparable to this use-case 🙏

Does anyone know what the APPLY_FIELD_SELECTION environment variable is for?

I am planning to bring up AirByte in EKS. Is this the best guide to follow? (it says it is "beta" but I don't see an alternative) https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm Is there anyone that has followed and have some tips they wish they would have known before they did it? Anyone have a good guide for how to run Dagster, AirByte, and DBT in EKS together?

Still stuck on null pointer exception after install. Any ideas how to triage? https://airbytehq.slack.com/archives/C021JANJ6TY/p1677993341470109

Hi do we have any plan for connector planhat ? if no then is there any general connector for api ?

Hello together, i have a small question regarding the

buildDockerImage

build step in the connector-builder-server.

task prepareBuild(type: Copy) {
    from layout.projectDirectory.file(".")
    exclude '.*'
    exclude 'build'
    exclude '**/*.pyc'

    into layout.buildDirectory.dir("docker")
}

tasks.named("buildDockerImage") {
    dependsOn prepareBuild
    dependsOn copyDocker
}

Since the copyDocker Task is a

Sync

-Task, which deletes the

build/docker

directory, how is ensured, that it runs before the prepareBuild Task? Also, why is

'.*'

excluded in the prepareBuild Task, this will exclude for example the

.dockerignore

file right? So when sending the files to the docker-context no ignore file will be present. Thanks in advance!

Hi Airbyte team! I just got started with Airbyte and Github as source and immediately ran into check connection timeout when connecting to a Github org with very large number of repos. I also a PR ready https://github.com/airbytehq/airbyte-platform/pull/201 and would appreciate if someone could review the issue/PR. Thanks!

Question about salesforce source connectors: do we support to pull only the latest data ? Right now, the full refresh mode seems still pull everything including historical update. It seems to me pulling historical update has already been covered by incremental mode (we could always omit the state input). Why can’t we change the full refresh mode only pull the latest ?

hi everyone! my team is working with the open source code, and we want to experiment renaming columns in the stream processor used during the transform catalog step. Is this the correct file to modify?

airbyte-integrations\bases\base-normalization\normalization\transform_catalog\stream_processor.py

If so, what is the correct command to run so that the code change is reflected when running the source code in docker? Would it be this command in `gradlew`:

./gradlew :airbyte-integrations:bases:base-normalization:build

Hi Airbyte team , i'm facing an issue while creating a connections with Prestahop. All the information needed are set up but i'm facing this issue and i don't really know where that is coming from :