does anyone else have issues compiling the project...
# contributing-to-airbyte
c
does anyone else have issues compiling the project locally. i’m getting c compiler issues when it attempts to run this task
Task :airbyte-integrations:connectors:source-postgres-singer:installReqs
u
which version of python are you running
u
Copy code
➜  airbyte git:(master) python --version
Python 2.7.16
➜  airbyte git:(master) python3 --version
Python 3.7.8
➜  airbyte git:(master)
u
is that wrong?
u
am i supposed to symlink python to point to python3 or something?
u
deleting this
airbyte-integrations/connectors/source-postgres-singer/.venv
seems to have fixed the issue.
u
hmm maybe we have to set the executable to the venvs to python3 binary instead of python
u
it seems possible that i wasn’t careful and created the venv in python 2.
u
maybe copy pasta-ed off of the tap-postgres website or something
u
the venv should be greated automatically by ./gradlew build
u
oh
u
i likely had an old one in there though.
u
that i had created manually.
u
oh ok
u
I noticed some issues with that one when building psycopg2 binaries too (it might require us to install postgres and python3 devel gcc etc to build properly?)
u
But havent been able to properly identify yet..
u
do you need to run the build multiple times to get the build to pass all of the python checks?
u
mypyCheck
has seemed very flakey.
u
do you have an example of a failure
u
haven't seen any mypy errors yet
u
it shouldn't require multiple builds, if it does things are happening in the wrong order
u
okay. i’m going to keep my fingers crossed that this is an issue with me switching back and forth between kinda old branches.
u
i’ll report back if i see this consistently on new branches.
u
nope. looks like still failing
u
Copy code
> Task :airbyte-integrations:bases:base-singer:mypyCheck FAILED
[python] .venv/bin/python -m mypy -m base_singer --config-file /Users/charles/code/airbyte/tools/python/.mypy.ini
         airbyte_protocol/__init__.py:27: note: In module imported here,
         base_singer/singer_helpers.py:33: note: ... from here,
         base_singer/__init__.py:25: note: ... from here:
         airbyte_protocol/models/__init__.py:2:1: error: Skipping analyzing
         'airbyte_protocol.models.airbyte_message': found module but no type hints or
         library stubs  [import]
             from .airbyte_message import *
             ^
         airbyte_protocol/models/__init__.py:2:1: note: See <https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports>
         Found 1 error in 1 file (checked 1 source file)
u
probably related. these two files are almost always dirty.
u
what's the diff?
u
there used to be license problems before
u
but I thought those were fixed
u
we probably need an additional gradle dependency to make sure generating the protocol files happens before spotlessapply
u
and that those happen before later formatting checks
u
it seems like specifically in the publish script it gets whack.y
u
./tools/integrations/manage.sh publish airbyte-integrations/connectors/source-exchangeratesapi-singer
u
after i run this script the diff happens.
u
so i can’t run it against for another integration unless i first remove the idff.
u
Copy code
→ ./gradlew integrationTest --dry-run | grep generate
:airbyte-integrations:bases:base-python:generateProtocolClassFiles SKIPPED
:airbyte-config:models:generateJsonSchema2Pojo SKIPPED
:airbyte-protocol:models:generateJsonSchema2Pojo SKIPPED

# jrhizor in ~/code/airbyte on git:master ✖︎ [9:15:52]
→ ./gradlew integrationTest --dry-run | grep spotless
u
looks like the call to integrationTest depends on
python:generateProtocolClassFiles
but not on
spotlessApply
, so it's getting strange results.
u
I guess
generateProtocolClassFiles
is probably in the wrong spot then?
u
adding this to my zshrc fixed the OP
Copy code
export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
u
the issue was
Copy code
ld: library not found for -lssl
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
which just meant the psycopg2 install couldn’t find ssl
u
on mac we probably want to add this option by default (since everyone is probably using openssl). Linux I’m not sure
u
wasn't necessary on my computer :/
u
We should at a minimum add these to a troubleshooting doc
u
which OS are you running? and what does
which openssl
return?
u
Copy code
→ which openssl
/usr/bin/openssl
u
Mac 10.15.4