Caio César P. Ricciuti
10/07/2021, 1:04 PMNikhil Jain
10/08/2021, 5:40 AMDejan Antonic
10/08/2021, 11:21 AMChris (deprecated profile)
Martin Larsson
10/10/2021, 1:17 PMJeremy Branham
10/10/2021, 4:33 PM* What went wrong:
An exception occurred applying plugin request [id: 'airbyte-docker']
> Failed to apply plugin 'airbyte-docker'.
> A problem occurred starting process 'command 'docker''
docker
is on my path -
$ which docker
/usr/local/bin/docker
A little more stack trace -
Caused by: org.gradle.api.internal.plugins.PluginApplicationException: Failed to apply plugin 'airbyte-docker'.
at ...
... 250 more
Caused by: org.gradle.process.internal.ExecException: A problem occurred starting process 'command 'docker''
at ...
Caused by: net.rubygrapefruit.platform.NativeException: Could not start 'docker'
at ...
Caused by: java.io.IOException: Cannot run program "docker" (in directory "/Users/me/git/airbyte/airbyte/airbyte-cli"): error=2, No such file or directory
at net.rubygrapefruit.platform.internal.DefaultProcessLauncher.start(DefaultProcessLauncher.java:25)
... 7 more
Caused by: java.io.IOException: error=2, No such file or directory
... 8 more
Prithvi Pradip
10/11/2021, 6:30 AMAndrey Morskoy
10/11/2021, 9:50 AMsource-file
, as well as source-s3
. Seems that ~60% of time source spends converting data into AirbyteMessage (before transformers) and later making json.dump
. Are there any plans on making these conversions less painful? I would be happy to get any info to understand in which direction this architecture moves generally.
2. Are there any plans for scalability? At this moment conversions and transformations, performed in source
container, both are obvious subject to run in parallel. For me it looks pretty perspective to have source
responsible only for data fetch in some raw form (byte arrays?) and delegate or complex conversions, transformations/normalization to scalable middle layer (even naive Apache Spark Streaming would be good improvement I suppose). May I ask which direction does Airbyte follow to deal with scalability?Martin Larsson
10/11/2021, 8:14 PMAmeya Bapat
10/12/2021, 11:55 AMKyle Cheung
10/13/2021, 4:33 PMSunny J
10/13/2021, 9:33 PMCristiano Sarmento
10/15/2021, 2:49 PMSunny J
10/15/2021, 5:02 PMIshan Joshi
10/19/2021, 5:10 AMZak Keener
10/19/2021, 6:18 PMZak Keener
10/19/2021, 6:55 PMDavin Chia (Airbyte)
Caio César P. Ricciuti
10/20/2021, 12:42 PM401 Unauthorized
when trying to GET
``airbyte.mydomain.io/manifest.json` . Anyone had this problem and have any idea on how to fix? Thanks in advanceVijay
10/25/2021, 8:51 PMNikhil Jain
10/26/2021, 6:20 AMDavin Chia (Airbyte)
Error: Operation DescribeNamespace failed.
Error Details: rpc error: code = NotFound desc = Namespace default does not exist.
{"level":"error","ts":"2021-10-26T12:07:54.656Z","msg":"Internal service error","service":"frontend","error":"Not enough hosts to serve the request","logging-call-at":"workflowHandler.go:3406","stacktrace":"<http://go.temporal.io/server/common/log/loggerimpl.(*loggerImpl).Error\n\t/temporal/common/log/loggerimpl/logger.go:138\ngo.temporal.io/server/service/frontend.(*WorkflowHandler).error\n\t/temporal/service/frontend/workflowHandler.go:3406\ngo.temporal.io/server/service/frontend.(*WorkflowHandler).StartWorkflowExecution\n\t/temporal/service/frontend/workflowHandler.go:497\ngo.temporal.io/server/service/frontend.(*DCRedirectionHandlerImpl).StartWorkflowExecution.func2\n\t/temporal/service/frontend/dcRedirectionHandler.go:1105\ngo.temporal.io/server/service/frontend.(*NoopRedirectionPolicy).WithNamespaceRedirect\n\t/temporal/service/frontend/dcRedirectionPolicy.go:116\ngo.temporal.io/server/service/frontend.(*DCRedirectionHandlerImpl).StartWorkflowExecution\n\t/temporal/service/frontend/dcRedirectionHandler.go:1101\ngo.temporal.io/api/workflowservice/v1._WorkflowService_StartWorkflowExecution_Handler.func1\n\t/go/pkg/mod/go.temporal.io/api@v1.4.0/workflowservice/v1/service.pb.go:1047\ngo.temporal.io/server/common/authorization.(*interceptor).Interceptor\n\t/temporal/common/authorization/interceptor.go:136\ngoogle.golang.org/grpc.getChainUnaryHandler.func1\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1051\ngo.temporal.io/server/common/rpc.ServiceErrorInterceptor\n\t/temporal/common/rpc/grpc.go:100\ngoogle.golang.org/grpc.chainUnaryServerInterceptors.func1\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1037\ngo.temporal.io/api/workflowservice/v1._WorkflowService_StartWorkflowExecution_Handler\n\t/go/pkg/mod/go.temporal.io/api@v1.4.0/workflowservice/v1/service.pb.go:1049\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1210\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1533\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:871|go.temporal.io/server/common/log/loggerimpl.(*loggerImpl).Error\n\t/temporal/common/log/loggerimpl/logger.go:138\ngo.temporal.io/server/service/frontend.(*WorkflowHandler).error\n\t/temporal/service/frontend/workflowHandler.go:3406\ngo.temporal.io/server/service/frontend.(*WorkflowHandler).StartWorkflowExecution\n\t/temporal/service/frontend/workflowHandler.go:497\ngo.temporal.io/server/service/frontend.(*DCRedirectionHandlerImpl).StartWorkflowExecution.func2\n\t/temporal/service/frontend/dcRedirectionHandler.go:1105\ngo.temporal.io/server/service/frontend.(*NoopRedirectionPolicy).WithNamespaceRedirect\n\t/temporal/service/frontend/dcRedirectionPolicy.go:116\ngo.temporal.io/server/service/frontend.(*DCRedirectionHandlerImpl).StartWorkflowExecution\n\t/temporal/service/frontend/dcRedirectionHandler.go:1101\ngo.temporal.io/api/workflowservice/v1._WorkflowService_StartWorkflowExecution_Handler.func1\n\t/go/pkg/mod/go.temporal.io/api@v1.4.0/workflowservice/v1/service.pb.go:1047\ngo.temporal.io/server/common/authorization.(*interceptor).Interceptor\n\t/temporal/common/authorization/interceptor.go:136\ngoogle.golang.org/grpc.getChainUnaryHandler.func1\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1051\ngo.temporal.io/server/common/rpc.ServiceErrorInterceptor\n\t/temporal/common/rpc/grpc.go:100\ngoogle.golang.org/grpc.chainUnaryServerInterceptors.func1\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1037\ngo.temporal.io/api/workflowservice/v1._WorkflowService_StartWorkflowExecution_Handler\n\t/go/pkg/mod/go.temporal.io/api@v1.4.0/workflowservice/v1/service.pb.go:1049\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1210\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1533\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:871>"}
Davin Chia (Airbyte)
tharaka prabath
10/28/2021, 3:16 AMkonrad schlatte
10/28/2021, 9:27 AMVijay
10/28/2021, 3:13 PMVijay
10/28/2021, 9:43 PMOleksandr Tsukanov [GL]
10/29/2021, 10:38 AMresources/seed/destination_definitions.yaml
and airbyte-integrations/connectors/destination-bigquery-denormalized/Dockerfile
but not in resources/config/STANDARD_DESTINATION_DEFINITION/079d5540-f236-4294-ba7c-ade8fd918496.json
. Thanks.Davin Chia (Airbyte)
➜ ~ docker images | head -n 30 | grep 0.30.23-alpha
airbyte/webapp 0.30.23-alpha 705cee2dda23 6 days ago 53MB
airbyte/server 0.30.23-alpha fee3d39ea555 6 days ago 800MB
airbyte/worker 0.30.23-alpha 9f0cac3aa409 6 days ago 1.2GB
airbyte/scheduler 0.30.23-alpha 786de4d6c25a 6 days ago 759MB
airbyte/db 0.30.23-alpha 73344a9224f2 6 days ago 192MB
Vijay
11/02/2021, 7:55 PM