Hi getting this error when starting airbyte locally on docker, any ideas?

airbyte-server   | 2021-06-14 12:52:38 ERROR i.a.s.v.VersionMismatchServer(getServer):68 - {workspace_app_root=/tmp/workspace/server/logs} - Version mismatch between 0.25.0-alpha and 0.24.7-alpha.

About the Structured/Repeated fields in BigQuery… Since BigQuery does not allow

Arrays of Arrays of XXX

, you need to build

Arrays of Record of Arrays of XXX

instead I know Stitch is choosing to add a 

.value

 every-time you have a record/array field when creating the columns in BQ: https://www.stitchdata.com/docs/replication/loading/google-bigquery-storing-nested-data-structures#storing-nested-records Actually, whenever there is an

Array of

they automatically build a

Record of Array of

using that extra

.value

field. So you might end up with things like *`friend_ids.value.value` *in the case of arrays of arrays… I’m wondering how does other do it? or how should Airbyte do this?

Hi guys, i think that exists backlog to create a connector to Pendo (https://github.com/airbytehq/airbyte/issues/2977). Somebody knows what is the estimated date to be used?

Hello, couple of n00b observations in Airbyte protocol yaml file. Details in the thread.

Master build is broken because of

format

and an unpublished image. I am working on the fix.

I am assuming that we've added all of this templating to help our with some of the contractors working with us. If it's not relevant to what you're doing please remove it. All templates in the PRs / Issues are meant to be a guideline, but you should always feel free to override a template with your judgment. Ideally PR descriptions should only have meaningful text in them, so if your change doesn't involve following this check list, imo it's best not to include it.

Does anyone know what this actually impact in in Airbyte docker, how it work? From docker-compose.yaml

x-logging: &default-logging
  options:
    max-size: "100m"
    max-file: "5"
  driver: json-file

I am trying to connect to a mongodb to get the oplog records into Snowflake using Airbyte. For Oplog we need to set up the mongodb as a replicaset (single node replicaset). For that single node mongo, when I tried to setup the source, I was not able to connect, it gives issue - 2021-06-17 133245 INFO (/tmp/workspace/9793cd22-4195-46de-98b8-678211e99c98/0) DefaultAirbyteStreamFactory(internalLog):110 - [2021-06-17 133150 +0000] WARN : | "MONGODB | Error running ismaster on mongo-vm27017 SocketError: getaddrinfo: Try again" To rule out a network issue, I wrote a pymongo script in the airbyte VM and was able to query the mongo replicaset. Is it related to https://github.com/airbytehq/airbyte/issues/2421? Does airbyte have capability to connect to a mongo replicaset? Anything I am missing? Any workarounds? cc: @Siddharth Putuvely @Arvind Saraf

@Jared Rhizor (Airbyte) broke the AWS's us-west-2a's C5 instance types

Error: AWS EC2 instance starting error
33
Error: InsufficientInstanceCapacity: We currently do not have sufficient c5.2xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c5.2xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.
34
Error: We currently do not have sufficient c5.2xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get c5.2xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.

guys, I’m working on improving error handling in HttpStream from CDK, I would like to gather possible scenarios: 1. request needs to be repeated with some adjustment from the prev response 2. request should be skipped, error reported and we should proceed to the next slice/stream is there any other?

@Chris (deprecated profile) couple of questions 1. Can you help us with a update on this https://github.com/airbytehq/airbyte/issues/3390 2. Webhook integration - can it help us to get the api quota, this will help us to keep the sync frequency in control..Or is there any other way to get this info

Getting a failure on local gradle build:

> Task :airbyte-integrations:connectors:destination-oracle:compileIntegrationTestJavaJava FAILED
file or directory '/Users/phlair/src/airbyte/airbyte-integrations/connectors/destination-oracle/src/integrationTestJava/java', not found
Build cache key for task ':airbyte-integrations:connectors:destination-oracle:compileIntegrationTestJavaJava' is 1f126b73848c4823af9e306bab5bc2db
Task ':airbyte-integrations:connectors:destination-oracle:compileIntegrationTestJavaJava' is not up-to-date because:
  Task has failed previously.
The input changes require a full rebuild for incremental task ':airbyte-integrations:connectors:destination-oracle:compileIntegrationTestJavaJava'.
Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.
file or directory '/Users/phlair/src/airbyte/airbyte-integrations/connectors/destination-oracle/src/integrationTestJava/java', not found
Compiling with JDK Java compiler API.
/Users/phlair/src/airbyte/airbyte-integrations/connectors/destination-oracle/src/test-integration/java/io/airbyte/integrations/destination/oracle/OracleIntegrationTest.java:102: error: method does not override or implement a method from a supertype
  @Override
  ^
1 error
:airbyte-integrations:connectors:destination-oracle:compileIntegrationTestJavaJava (Thread[Execution worker for ':',5,main]) completed. Took 0.272 secs.

Hi I also have an error when running gradle - however I am in an virtualenv..

Convfigured environment is not a virtualenv

Our current copyright declaration is outdated:

Copyright (c) 2020 Airbyte

Should we just change it to

Copyright (c) 2019 - Present Airbyte

so that we don’t need to update it every year?

Does anyone know what is the actually impact in Airbyte docker, how it work? From docker-compose.yaml.

init:
    image: airbyte/init:${VERSION}
    logging: *default-logging
    container_name: init
    command: /bin/sh -c "./scripts/create_mount_directories.sh /local_parent ${HACK_LOCAL_ROOT_PARENT} ${LOCAL_ROOT}"
    environment:
      - LOCAL_ROOT=${LOCAL_ROOT}
      - HACK_LOCAL_ROOT_PARENT=${HACK_LOCAL_ROOT_PARENT}
    volumes:
      - ${HACK_LOCAL_ROOT_PARENT}:/local_parent

Hi guys wanted to know is it possible to update the state for a connection using API ?

A question to Dev team. Would AirByte convert to a closed source project in future like other open source integration projects? Recently, Streamsets did this and that was quite disheartening.

Hey need some help in understanding why sync is not started once we create connection with 24 hours sync. Earlier we thought it's because we don't have enough memory but looks like this is happening very frequently for us.

Are these failures with EC2 happening like last time again @Davin Chia (Airbyte)?

Error: GitHub self-hosted runner registration error
Checking...
Error: A timeout of 5 minutes is exceeded. Your AWS EC2 instance was not able to register itself in GitHub as a new self-hosted runner.

https://github.com/airbytehq/airbyte/runs/2893207899?check_suite_focus=true

Hi, having an issue with running a new connector in the UI - it doesn't appear in the UI although I ran gradle and built a docker image in the directory

also there is this dead link in the docs https://docs.airbyte.io/tutorials/tutorials/toy-connector#use-the-connector-in-the-airbyte-ui

Quick question, does GKE still require S3 for Configure Logs?

Hi, two questions: 1. I created my first workpace with

<http://localhost:8000/api/v1/workspaces/create>

but how can I login to that workspace on UI or how can I verify I’m in the same workspace? 2. If I create my workspace in UI, how can I verify my workspaceId? Thanks!

hello everybody, quick question: if I'm running Airbyte in GCP (I followed the instructions on the website), how do I install custom connectors which I currently have on my local computer? thanks 🙂 🍻

whew those kube e2e acceptance tests really take this long to execute? https://github.com/airbytehq/airbyte/pull/4317/checks?check_run_id=2909112274

Have a question. We did setup a 24 hour sync and then manually triggered a sync some time after now when will the next sync gets scheduled ?

Heads up Airbyte team, I've merged in https://github.com/airbytehq/airbyte/pull/3362 to try and fix the transient

checkPython

error we've been seeing. This was suggested by the plugin's maintainer. The PR's build was successful 10 times in a row which made me think it's worth giving it a shot. Please lmk if anyone still see errors caused by the

checkPython

task after this. Thanks!

hello, just wanted to check about kubernetes deployment, how is the progress and when we could expect it? 🙂

Is the S3 dependency a hard requirement for the Kubernetes deployment at this point or is it only needed for some kind of supplemental logging?

@Jared Rhizor (Airbyte) it looks like the airbyte db pod on Kube doesn't preserve data between pod restarts - is that expected?