Channels
c
charles
10/26/2020, 3:31 PM
does anyone else have issues compiling the project locally. i’m getting c compiler issues when it attempts to run this task
Task :airbyte-integrations:connectors:source-postgres-singer:installReqsu
user
10/26/2020, 3:31 PMwhich version of python are you running
u
user
10/26/2020, 3:32 PM Copy code
➜ airbyte git:(master) python --version
Python 2.7.16
➜ airbyte git:(master) python3 --version
Python 3.7.8
➜ airbyte git:(master)u
user
10/26/2020, 3:33 PMis that wrong?
u
user
10/26/2020, 3:33 PMam i supposed to symlink python to point to python3 or something?
u
user
10/26/2020, 3:35 PMdeleting this
airbyte-integrations/connectors/source-postgres-singer/.venvu
user
10/26/2020, 3:38 PMhmm maybe we have to set the executable to the venvs to python3 binary instead of python
u
user
10/26/2020, 3:39 PMit seems possible that i wasn’t careful and created the venv in python 2.
u
user
10/26/2020, 3:39 PMmaybe copy pasta-ed off of the tap-postgres website or something
u
user
10/26/2020, 3:39 PMthe venv should be greated automatically by ./gradlew build
u
user
10/26/2020, 3:39 PMoh
u
user
10/26/2020, 3:39 PMi likely had an old one in there though.
u
user
10/26/2020, 3:40 PMthat i had created manually.
u
user
10/26/2020, 3:40 PMoh ok
u
user
10/26/2020, 3:47 PMI noticed some issues with that one when building psycopg2 binaries too (it might require us to install postgres and python3 devel gcc etc to build properly?)
u
user
10/26/2020, 3:47 PMBut havent been able to properly identify yet..
u
user
10/26/2020, 3:55 PMdo you need to run the build multiple times to get the build to pass all of the python checks?
u
user
10/26/2020, 3:55 PMmypyChecku
user
10/26/2020, 4:00 PMdo you have an example of a failure
u
user
10/26/2020, 4:00 PMhaven't seen any mypy errors yet
u
user
10/26/2020, 4:01 PMit shouldn't require multiple builds, if it does things are happening in the wrong order
u
user
10/26/2020, 4:01 PMokay. i’m going to keep my fingers crossed that this is an issue with me switching back and forth between kinda old branches.
u
user
10/26/2020, 4:02 PMi’ll report back if i see this consistently on new branches.
u
user
10/26/2020, 4:06 PMnope. looks like still failing
u
user
10/26/2020, 4:06 PM Copy code
> Task :airbyte-integrations:bases:base-singer:mypyCheck FAILED
[python] .venv/bin/python -m mypy -m base_singer --config-file /Users/charles/code/airbyte/tools/python/.mypy.ini
airbyte_protocol/__init__.py:27: note: In module imported here,
base_singer/singer_helpers.py:33: note: ... from here,
base_singer/__init__.py:25: note: ... from here:
airbyte_protocol/models/__init__.py:2:1: error: Skipping analyzing
'airbyte_protocol.models.airbyte_message': found module but no type hints or
library stubs [import]
from .airbyte_message import *
^
airbyte_protocol/models/__init__.py:2:1: note: See <https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports>
Found 1 error in 1 file (checked 1 source file)u
user
10/26/2020, 4:07 PMprobably related. these two files are almost always dirty.
u
user
10/26/2020, 4:07 PMwhat's the diff?
u
user
10/26/2020, 4:07 PMthere used to be license problems before
u
user
10/26/2020, 4:07 PMbut I thought those were fixed
u
user
10/26/2020, 4:09 PMwe probably need an additional gradle dependency to make sure generating the protocol files happens before spotlessapply
u
user
10/26/2020, 4:09 PMand that those happen before later formatting checks
u
user
10/26/2020, 4:14 PMit seems like specifically in the publish script it gets whack.y
u
user
10/26/2020, 4:14 PM./tools/integrations/manage.sh publish airbyte-integrations/connectors/source-exchangeratesapi-singeru
user
10/26/2020, 4:14 PMafter i run this script the diff happens.
u
user
10/26/2020, 4:14 PMso i can’t run it against for another integration unless i first remove the idff.
u
user
10/26/2020, 4:16 PM Copy code
→ ./gradlew integrationTest --dry-run | grep generate
:airbyte-integrations:bases:base-python:generateProtocolClassFiles SKIPPED
:airbyte-config:models:generateJsonSchema2Pojo SKIPPED
:airbyte-protocol:models:generateJsonSchema2Pojo SKIPPED
# jrhizor in ~/code/airbyte on git:master ✖︎ [9:15:52]
→ ./gradlew integrationTest --dry-run | grep spotlessu
user
10/26/2020, 4:16 PMlooks like the call to integrationTest depends on
python:generateProtocolClassFilesspotlessApplyu
user
10/26/2020, 4:16 PMI guess
generateProtocolClassFilesu
user
10/26/2020, 6:22 PMadding this to my zshrc fixed the OP
Copy code
export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"u
user
10/26/2020, 6:23 PMthe issue was
Copy code
ld: library not found for -lssl
clang: error: linker command failed with exit code 1 (use -v to see invocation)u
user
10/26/2020, 6:23 PMon mac we probably want to add this option by default (since everyone is probably using openssl). Linux I’m not sure
u
user
10/26/2020, 6:23 PMwasn't necessary on my computer :/
u
user
10/26/2020, 6:24 PMWe should at a minimum add these to a troubleshooting doc
u
user
10/26/2020, 6:30 PMwhich OS are you running? and what does
which opensslu
user
10/26/2020, 6:31 PM Copy code
→ which openssl
/usr/bin/opensslu
user
10/26/2020, 6:31 PMMac 10.15.4