Serhii Chvaliuk [GL]
11/02/2021, 9:38 PMcommit b1f5c23a6ffb11572a0b141f6aa93cbebc6c8a4a
Author: Jared Rhizor <jared@dataline.io>
Date: Tue Nov 2 09:45:54 2021 -0700
use ADD to reduce docker image sizes (#7537)
* use ADD to reduce docker image sizes
* switch to full paths
konrad schlatte
11/03/2021, 10:37 AMOleksandr Tsukanov [GL]
11/03/2021, 11:16 AMairbyte-config/init/src/main/resources/seed/destination_specs.yaml
or here airbyte-config/init/src/main/resources/seed/source_specs.yaml
as well? Thanks.Oleksandr Bazarnov [GL]
11/03/2021, 1:32 PMException in thread "main" java.lang.RuntimeException: Failed to fetch valid spec file for docker image airbyte/source-drift:0.2.4 from GCS bucket io-airbyte-cloud-spec-cache
at io.airbyte.config.specs.SeedConnectorSpecGenerator.lambda$fetchSpecFromGCS$4(SeedConnectorSpecGenerator.java:120)
at java.base/java.util.Optional.orElseThrow(Optional.java:401)
at io.airbyte.config.specs.SeedConnectorSpecGenerator.fetchSpecFromGCS(SeedConnectorSpecGenerator.java:120)
at io.airbyte.config.specs.SeedConnectorSpecGenerator.lambda$fetchUpdatedSeedSpecs$3(SeedConnectorSpecGenerator.java:113)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at io.airbyte.config.specs.SeedConnectorSpecGenerator.fetchUpdatedSeedSpecs(SeedConnectorSpecGenerator.java:114)
at io.airbyte.config.specs.SeedConnectorSpecGenerator.run(SeedConnectorSpecGenerator.java:82)
at io.airbyte.config.specs.SeedConnectorSpecGenerator.main(SeedConnectorSpecGenerator.java:72)
gunu
11/04/2021, 1:56 AM_AIRBYTE_AB_ID does not exist. The table needs to be rebuilt in full_refresh
but just want to confirm there’s no workaround as its a pretty big task
2. are the dbt logs removed?Davin Chia (Airbyte)
Vijay
11/04/2021, 9:51 PMVijay
11/05/2021, 4:18 AMJean-Mathieu Saponaro
11/05/2021, 10:14 AM[DEPRECATED] Augustin Lafanechere
SUB_BUILD=PLATFORM ./gradlew build
on the master branch. I'm on a Macbook M1 and I've set some specific env var as suggested in the doc (you can find it below). The build is failing on the airbyte-webapp:npmInstall
task with the error below. I installed node 16 and logged to npm with npm login
.
Env vars:
export DOCKER_BUILD_PLATFORM=linux/arm64
export DOCKER_BUILD_ARCH=arm64
export JDK_VERSION=17
export NODE_VERSION=16.13.0
export PATH="/opt/homebrew/opt/node@16/bin:$PATH"
export LDFLAGS="-L/opt/homebrew/opt/node@16/lib"
export CPPFLAGS="-I/opt/homebrew/opt/node@16/include"
Error:
npm ERR! code E401
npm ERR! Incorrect or missing password.
npm ERR! If you were trying to login, change your password, create an
npm ERR! authentication token or enable two-factor authentication then
npm ERR! that means you likely typed your password in incorrectly.
npm ERR! Please try again, or recover your password at:
npm ERR! <https://www.npmjs.com/forgot>
npm ERR!
npm ERR! If you were doing some other operation then your saved credentials are
npm ERR! probably out of date. To correct this please try logging in again with:
npm ERR! npm login
Welly Tambunan
11/07/2021, 12:44 AMDavin Chia (Airbyte)
Ben Rosen
11/08/2021, 7:40 PMPrateek Gupta
11/08/2021, 8:27 PMEthan Veres
11/09/2021, 12:00 AMMaxime Lavoie
11/09/2021, 8:00 PM"oneOf": [{…}, {…}]
type structure?
I’ve tried different approaches and the closest I got was an "anyOf": [{…},{…}]
structure using account_ids: Union[AccountConfigStrategy1, AccountConfigStrategy2] = Field(...)
Savanna Brombacher
11/10/2021, 7:06 AMZygimantas Vasiliauskas
11/10/2021, 4:06 PMOleh Zorenko
11/15/2021, 7:52 AM/test connector
command for google search console source in this PR and it fails:
• https://github.com/airbytehq/airbyte/actions/runs/1460805371
• https://github.com/airbytehq/airbyte/actions/runs/1460954249
I found this answer on stackoverflow and it looks like there is a shortage of virtual memory. Is it something that could be fixed by updating github workflows or the only way is to reduce amount of records?Ameya Bapat
11/16/2021, 4:53 AMSerhii Chvaliuk [GL]
11/18/2021, 8:35 PMoauthUserInputFromConnectorConfigSpecification
https://github.com/airbytehq/airbyte/blob/cb9c5123b1789310ee313c2823868ecba65aa35f/airbyte-api/src/main/openapi/config.yaml#L3259
on Python part we have snake_case oauth_user_input_from_connector_config_specification
https://github.com/airbytehq/airbyte/blob/cb9c5123b1789310ee313c2823868ecba65aa35f/airbyte-cdk/python/airbyte_cdk/models/airbyte_protocol.py#L122
I see that for old deprecated specification “authSpecification” we have camelCase for both python and javaThomas
11/19/2021, 8:39 PMThomas
11/19/2021, 8:39 PMgunu
11/23/2021, 1:29 AM0.30.23
(pre- new incremental normalization)
i’d like to upgrade to v0.32.25
to try out the recent changes made there, so i need to first upgrade to v0.32.0
to facilitate a safe migration.
question: if i encounter issues with v0.32.25
can i immediately revert back to 0.30.23
or will there be backward incompatible changes?Yurii Bidiuk [GL]
11/24/2021, 11:00 PMJsonNode
from AirbyteMessage
, reads the date/date-time value, converts it to a valid ISO-8601 format, and replaces the value in the JsonNode
.
Can you suggest a place in the code where we can do this job?
P.S. A possible place to use this solution is io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer#acceptTracked
method.Shubham Sinha
11/26/2021, 10:44 PMSUB_BUILD=CONNECTORS_BASE ./gradlew clean build
VERSION=dev docker-compose u
I notice in the airbyte logs that it still pulls the airbyte:normalization image from remote:
2021-11-26 22:37:46 INFO () DefaultNormalizationRunner(runProcess):135 - Running with normalization version: airbyte/normalization:0.1.60
2021-11-26 22:37:46 INFO () LineGobbler(voidCall):82 - Checking if airbyte/normalization:0.1.60 exists...
2021-11-26 22:37:47 INFO () LineGobbler(voidCall):82 - airbyte/normalization:0.1.60 not found locally. Attempting to pull the image...
Any ideas on this? Thanks!Jakub Novotný
11/29/2021, 12:07 PMMaxime edfeed
11/29/2021, 6:32 PMGCP_STORAGE_BUCKET: {{ default "" .Values.gcpLogs.bucket }}
GOOGLE_APPLICATION_CREDENTIALS: {{ default "" .Values.gcpLogs.credentials }}
Emin Can Oğuz (Hepsiburada)
11/30/2021, 8:45 AMJared Rhizor (Airbyte)