how would people feel about adding integration tests to PR builds temporarily. it feels like i’m breaking the master build all the time on this. other people feeling this? if not, i’ll just run them myself before merging from now on.

i’m looking at master build failures. probably my fault.

useful tip: if you are looking for docs on how

tap-singer

works, Stitch has a ton of docs about it here: https://www.stitchdata.com/docs/integrations/databases/postgresql

Just published how to improve our process to update docs so we can keep the PR review process: https://github.com/datalineio/dataline/pull/327

is there a way to run our app using my local file system as mounted workspace as opposed to a docker volume? i thought this was possible, but i’m having trouble figuring it out.

for a new integration what are the steps (and can I add them to the docs): • Write integration • Write integration tests • Also write pairwise acceptance test (??? can’t remember if we said this was part of it) • Put this all up for a PR. • Write docs but in a separate PR • Push the image of the integration to docker hub. • Merge both PRs at the same time?

For anyone who has written an integration test, do you good way of getting stderr from

entrypoint.sh

when running integration tests?

just to make sure, do we plan to make it more readable?

it looks like there’s been a regression on the postgres tap. i’m looking into it.

master has the following warning when I build:

> Task :dataline-api:generateApiDocs
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]
'scheme' not defined in the spec (2.0). Default to [http] for server URL [<http://localhost:8001>]

FYI, the connection of currency to csv throws errors:

airbyte-scheduler | java.lang.RuntimeException: Could not find stream metadata
airbyte-scheduler | 	at io.airbyte.workers.protocols.singer.SingerCatalogConverters.lambda$toAirbyteSchema$5(SingerCatalogConverters.java:139) ~[airbyte-workers-0.1.0.jar:?]
airbyte-scheduler | 	at java.util.Optional.orElseThrow(Optional.java:401) ~[?:?]
airbyte-scheduler | 	at io.airbyte.workers.protocols.singer.SingerCatalogConverters.lambda$toAirbyteSchema$7(SingerCatalogConverters.java:139) ~[airbyte-workers-0.1.0.jar:?]
airbyte-scheduler | 	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
airbyte-scheduler | 	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
airbyte-scheduler | 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
airbyte-scheduler | 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
airbyte-scheduler | 	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
airbyte-scheduler | 	at java.util.stre

FYI, I merged the renaming. I also modified the organization and repo name on github

most of the website is changed to airbyte. I think I will be done by EOD

i’m going to look into better feedback that the app hasn’t started up yet in the UI, unless anyone has anything more urgent they need help on.

why do I have an untracked

api/

directory when I build?

Is there any reason we didn’t push the new pg image for this? https://github.com/airbytehq/airbyte/pull/382

guys, what version of intellij idea do you have? I try to set up code style but it throws some error.

👍 i don’t have access.

I am not receiving any front end event

Syncing currency to PG fails

Should we talk about release cycle today? We’re going to start pushing two types of changes 1. hotfixes 2. changes that should be added to next release. there are logistics we need to figure out around this, i.e. having release branches etc.

master build started failing. it had been successful on the last commit, but when the build re-ran on the same commit it failed. have meetings starting now, but can take a look after.

Added

new-integration

label so we can see integration requests easily in the issues list.

Master building failing again. Seems unrelated to your changes. Failure in integrations tests with BQ some sort of connectivity issue. I am seeing issues with a lot of other google services in general right now though. Inclined to give it some time and then check back in a bit to see if this is just a google problem.

I’m having trouble finding any real source code. Obviously I’m looking at the wrong paths, but I was curious if somebody could please give me the classpath to some postgres connector code? I’m interested in how auth is handled as well various other items. Thanks so much!

Is it a known problem or something branch specific? In case the last one - I can investigate more on it.

Was something changed in build? Do I need some additional changes to build locally?

for posterity, the output of reading a singer tap (tap-postgres) with 2 selected streams

https://github.com/airbytehq/airbyte/pull/479 going once 👀

the terminology we’re using is misaligned between the product/frontend and the backend. What do you think about the following renames on the data model:

Source

-->

SourceConnector

Destination

-->

DestinationConnector

SourceImplementation

-->

Source

DestinationImplementation

-->

Destination