charles
Task :airbyte-integrations:connectors:source-postgres-singer:installReqs
charles
charles
2020-10-26 201430 ERROR i.a.w.p.a.DefaultAirbyteStreamFactory(internalLog):108 - * - CRITICAL Salesforce has reported 13060/15000 (87.07%) total REST quota used across all Salesforce Applications. Terminating replication to not continue past configured percentage of 80% total quota.
10106so another reason we may not want to run all integration integration tests with every build is that we’re going to run into quota issues. master build is failing currently because we’ve used up the quota from having built just a few times this morning.
charles
> Task :airbyte-integrations:connector-templates:singer-source:mypyCheck FAILED
[python] .venv/bin/python -m mypy -m template_singer_source --config-file /Users/charles/code/airbyte/tools/python/.mypy.ini
airbyte_protocol/__init__.py:27: note: In module imported here,
template_singer_source/source.py:27: note: ... from here,
template_singer_source/__init__.py:25: note: ... from here:
airbyte_protocol/models/__init__.py:2:1: error: Skipping analyzing
'airbyte_protocol.models.airbyte_message': found module but no type hints or
library stubs [import]
from .airbyte_message import *
^
airbyte_protocol/models/__init__.py:2:1: note: See <https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports>
Found 1 error in 1 file (checked 1 source file)
charles
charles
./gradlew :airbyte-integrations:connectors:source-github-singer:build
create a .venv
dir in airbyte-integrations/connectors/source-github-singer
or is there some other task that i should be running to do that?charles
charles
charles
> Task :airbyte-integrations:connectors:source-postgres-singer:installReqs
[python] .venv/bin/python -m pip install -r requirements.txt
Obtaining file:///Users/charles/code/airbyte/airbyte-integrations/bases/airbyte-protocol (from -r requirements.txt (line 1))
Obtaining file:///Users/charles/code/airbyte/airbyte-integrations/bases/base-python (from -r requirements.txt (line 2))
Obtaining file:///Users/charles/code/airbyte/airbyte-integrations/bases/base-singer (from -r requirements.txt (line 3))
Obtaining file:///Users/charles/code/airbyte/airbyte-integrations/connectors/source-postgres-singer (from -r requirements.txt (line 4))
Requirement already satisfied: PyYAML==5.3.1 in ./.venv/lib/python3.7/site-packages (from airbyte-protocol==0.0.0->-r requirements.txt (line 1)) (5.3.1)
Requirement already satisfied: pydantic==1.6.1 in ./.venv/lib/python3.7/site-packages (from airbyte-protocol==0.0.0->-r requirements.txt (line 1)) (1.6.1)
Expiring Daemon because JVM heap space is exhausted
Daemon is stopping immediately JVM garbage collector thrashing and after running out of JVM memory
anyone intermittently getting JVM heap space is exhausted
when building the project?charles
charles
Winar
10/30/2020, 9:33 AM:airbyte-integrations:connectors:source-postgres-singer:installReqs
, I will always get the error (#shown as the followed log file), that will led to the fail of the gradle build process. Is that cause of the version of Python or something? Thanks.Chris (deprecated profile)
airbyte-config/init/src/main/resources/config/STANDARD_SOURCE/778daa7c070c.json
with this content:
{
"sourceId": "778daa7c070c",
"name": "File",
"dockerRepository": "airbyte/source-file",
"dockerImageTag": "0.1.0",
"documentationUrl": "<https://hub.docker.com/r/airbyte/source-file>"
}
However when i start the docker-compose --env-file .env.dev -f docker-compose.yaml -f docker-compose.dev.yaml up
command and open the browser, i don't see my source-file in there, am i missing something?charles
Winar
11/02/2020, 1:38 PMMichel
Michel
charles
.dockerignore
files? I thought we had a pattern where we started with *
and then only surgically re-added the things we explicitly wanted to make available in the dockerfiles. Got some feedback asking to go the opposite way (e.g. just ignore build
). I don’t have a preference on this once, but I’d like us to coalesce around a standard that I will follow.Michel
charles
s
charles
setup.py
in how we refer to internal deps:
install_requires=["airbyte-protocol"],
install_requires=["airbyte_protocol"],
Both work. I think the -
version is more idiomatic in the python world. Please correct me if I’m wrong. I care most that we pick one way of doing it and stick with it.Hassan
11/05/2020, 12:27 PMChris (deprecated profile)
gradlew build -x spotlessSql -x spotlessSqlApply -x spotlessSqlCheck
to make it build successfully...
(Spotless is messing up the DBT SQL code turning into syntax errors otherwise...)
Thanks!Michel
Artem Astapenko
11/06/2020, 4:08 PMChris (deprecated profile)
Chris (deprecated profile)
charles
Jared Rhizor (Airbyte)
documentationUrl
in source/destination destinations?