could somebody please take a look at this? https://github.com/airbytehq/airbyte/pull/2263 Looks like master is failing because of spotlessCheck

Is somebody taking a look at

> Task :airbyte-scheduler:test
2021-03-02T21:53:49.3711488Z 
2021-03-02T21:53:49.3713432Z DefaultSchedulerJobClientTest > testCreateResetConnectionJob() FAILED
2021-03-02T21:53:49.3715154Z     org.opentest4j.AssertionFailedError: expected: <Mock for Job, hashCode: 566626087> but was: <null>
2021-03-02T21:53:49.3716825Z         at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
2021-03-02T21:53:49.3718640Z         at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:195)
2021-03-02T21:53:49.3720571Z         at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:184)
2021-03-02T21:53:49.3722536Z         at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:179)
2021-03-02T21:53:49.3724452Z         at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1124)
2021-03-02T21:53:49.3745430Z         at io.airbyte.scheduler.client.DefaultSchedulerJobClientTest.testCreateResetConnectionJob(DefaultSchedulerJobClientTest.java:139)
2021-03-02T21:53:49.3747991Z

in master?

think I spotted a typo while glancing through code. Should this error message,

throw new IOException(String.format("Imported VERSION (%s) is incompatible with current Airbyte version (%s).\n" +
    "Please Upgrade your Airbyte Archive, see more at <https://docs.airbyte.io/tutorials/tutorials/upgrading-airbyte>\n",
    importVersion, version));

actually link to

<https://docs.airbyte.io/tutorials/upgrading-airbyte>

(1 tutorial instead of 2)?

is there any info on how to construct

client.py

?

I’m getting an internal server error when querying

localhost:8001/api/v1/logs/get

. Anyone else seeing this? False alarm. Nothing to see here!

Hey guys, I'm unable to run airbyte locally. I've been following https://github.com/airbytehq/airbyte/blob/master/docs/contributing-to-airbyte/developing-locally.md and these are the steps I took 1. cloned master 2.

docker-compose up

3. went to the airbyte interface and configured a source/destination, its working as expected 4. i stopped the server

docker-compose down server

5. and when i bring the server up with

./gradlew :airbyte-server:run

i have this error: https://gist.github.com/gordalina/480ee8cb9ceb942e318643dd800d3b00

Also, where does airbyte store the sources/connection data? I looked into the postgres database and couldn't find it

Do we still need init in docker-compose?

Question:

io.airbyte.api.model.*

are defined in

airybyte-api/src/main/openapi/config.yaml

right?

Is there a way to run tests only on

airbyte-server

?

Hi Guys, Are in your plans any option to anonymization fields? It's a interesting feature for teams that need to replicate data from production to acceptance environment?

I'm trying to build airbyte locally for integration tests and I have this error: https://gist.github.com/gordalina/02ad6a951d68792bd725b4435af503da

./gradlew :airbyte-integrations:connectors:source-file:installLocalReqs

fails for me: https://gist.github.com/gordalina/39915a946816fb6b4456fe77197d8c8b it says it can't find the arrow module

I'm trying to run migrations but can't seem to pass arguments to the main class - I know am doing something wrong. I'm running:

./gradlew :airbyte-migration:run --target-version=0.17.0

Hi im trying to create the ssh tunnel to my ec2 instance with

ssh -i $SSH_KEY -L 8000:localhost:8000 -L 8001:localhost:8001 -N -f ec2-user@$INSTANCE_IP

but im getting this error

ssh: Could not resolve hostname 8000:localhost:8000: Name or service not known

Can anybody help me ?

Hi ppl !! Me again. I have now airbyte in my ec2 instance with the source and destination successfully configured, but when i try to sync the data o reset it all fails. Here are the logs of the task if anybody knows what's going on.

code style questions as I read more code: • how do people feel about using the 

var

 keyword in java? I’ve appreciated it more often than not when reading Java code. Charles seems fine. Curious to hear people’s thoughts before introducing it into the codebase. • same question about 

final

. I see it being used pretty judiciously throughout but not applied entirely. is the expectation to use final as much as possible? • can I confirm we line wrap at 150? (will update the docs)

What are people’s thoughts on Reviewable?

Hey team, I am working on differentiating between user and airbyte exception in our error tracking and would love some quick input. Right now error tracking mostly happens in the core Airbyte components (Server and Scheduler). This means more precise exception handling will require some changes to the Airbyte Protocol fields. I’m wondering where do we think these exceptions will come from and what of these are worth catching? In my (limited) experience, it seems modifying the

AirbyteConnectionStatus

will get us most of the way there in terms of user errors (don’t expect authentication/networking information to change often, is that a wrong assumption?) The alternative is amending the other protocol methods to allow them to also pass more detailed errors; obviously a bigger change.

PSA:

airbyte-db          | initdb: error: could not create directory "/var/lib/postgresql/data/pg_wal": No space left on device
airbyte-db          | initdb: removing contents of data directory "/var/lib/postgresql/data"

left me scratching my head for half an hour today. Hitting

Clean Up

in the Docker for Mac UI solved it! edit: Jared proposes the following Docker configuration:

4 cores, 4gb RAM, 1gb swap, 60gb of disk space

Hello! I am wondering, what is the use of this file in the migration module:

airbyte-migration/src/main/resources/migrations/migrationV0_14_3/airbyte_config/AirbyteProtocolAbbreviated.yaml

? (@charles maybe?)

Hello! I installed Airbyte according to Deploying Airbyte on GCP instructions. I'm on a private VPC within Google Compute Engine. Everything went OK up until:

# In your ssh session on the instance terminal
mkdir airbyte && cd airbyte
wget <https://raw.githubusercontent.com/airbytehq/airbyte/master/{.env,docker-compose.yaml}>
docker-compose up -d

Here's the error:

user:~/airbyte$ docker-compose up -d
ERROR: Couldn't connect to Docker daemon at <http+docker://localhost> - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

Any ideas?

Hello! I created a custom connector and built it afterwards. Once I have built it, I've lost the track of where the source code is located now (the path). Anyone can help me?

Hello, would you have an idea of why this tests is failing in the CI? or how do i investigate this further? https://github.com/airbytehq/airbyte/pull/2460/checks?check_run_id=2140585682 Locally it’s working fine…

How do people feel about an

experimental

folder for us to check in more hacky scripts/tools that aren’t part of the ‘main’ build? this folder is ‘special’ in that 1) the code doesn’t need to be reviewed to be checked in 2) it doesn’t need to be high quality, and tends to be more operational stuff that we might reuse down the line e.g. I recently wrote a simple script quickly insert a bunch of rows into a 5 column table as part of the Redshift work. A Dev could compose more ‘standard’ calls from our common libs or they could use something like this and save 10 - 20 mins of time.

We’ve got another weird case of failures happening in the CI on github for MySQL source connector, but it’s working perfectly fine when ran locally.. https://status-api.airbyte.io/tests/summary/source-mysql/ for example: https://github.com/airbytehq/airbyte/runs/2142398916?check_suite_focus=true

There is a bug on master where some errors that are thrown in check, discover, and get spec are not handled properly. It presents itself as the UI changing pages awkwardly when these jobs fail. This bug is not in the latest release 0.17.2. It is only on master. I'll have it fixed later today and will post here when it is done.

Am I doing something wrong? Did I configure this incorrectly?

Hi @charles and @Swayam Prakash Behera,I submitted a PR with Appsflyer integration. The only issue I had is that I no longer have an Appsflyer valid account (my project with that company has ended), so I was only able to test the connector with an invalid API token. Please let me know if there is any other issue.

FYI it’s seems like the

base-normalization

module would be failing to build from a

.venv

from scratch since 10 days ago… You would see errors about PyICU failing to be installed by pip. (This also happens in the CI but it somehow manages to fallback to an older version where it doesn’t happen) This is caused by a dependency of DBT that was upgraded in a sketchy way. So DBT also issued a patch in “dbt-19.1” and “18.2rc1" to address this by pinning the dependency to an older version: https://github.com/fishtown-analytics/dbt/pull/3161