kz
03/22/2021, 6:10 PMuser
03/24/2021, 3:10 PMArtem Astapenko
03/24/2021, 4:31 PMairbyte-temporal | {"level":"error","ts":"2021-03-24T16:09:19.365Z","msg":"Operation failed with internal error.","service":"matching","error":"UpdateTaskQueue failed. Failed to start transaction. Error: context deadline exceeded","metric-scope":38,"logging-call-at":"persistenceMetricClients.go:804","stacktrace":"<http://go.temporal.io/server/common/log/loggerimpl.(*loggerImpl).Error\n\t/temporal/common/log/loggerimpl/logger.go:138\ngo.temporal.io/server/common/persistence.(*taskPersistenceClient).updateErrorMetric\n\t/temporal/common/persistence/persistenceMetricClients.go:804\ngo.temporal.io/server/common/persistence.(*taskPersistenceClient).UpdateTaskQueue\n\t/temporal/common/persistence/persistenceMetricClients.go:787\ngo.temporal.io/server/service/matching.(*taskQueueDB).UpdateState\n\t/temporal/service/matching/db.go:109\ngo.temporal.io/server/service/matching.(*taskReader).persistAckLevel\n\t/temporal/service/matching/taskReader.go:272\ngo.temporal.io/server/service/matching.(*taskReader).getTasksPump\n\t/temporal/service/matching/taskReader.go:160|go.temporal.io/server/common/log/loggerimpl.(*loggerImpl).Error\n\t/temporal/common/log/loggerimpl/logger.go:138\ngo.temporal.io/server/common/persistence.(*taskPersistenceClient).updateErrorMetric\n\t/temporal/common/persistence/persistenceMetricClients.go:804\ngo.temporal.io/server/common/persistence.(*taskPersistenceClient).UpdateTaskQueue\n\t/temporal/common/persistence/persistenceMetricClients.go:787\ngo.temporal.io/server/service/matching.(*taskQueueDB).UpdateState\n\t/temporal/service/matching/db.go:109\ngo.temporal.io/server/service/matching.(*taskReader).persistAckLevel\n\t/temporal/service/matching/taskReader.go:272\ngo.temporal.io/server/service/matching.(*taskReader).getTasksPump\n\t/temporal/service/matching/taskReader.go:160>"}
airbyte-temporal | {"level":"error","ts":"2021-03-24T16:09:19.365Z","msg":"Persistent store operation failure","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-tq-scanner-taskqueue-0/1","wf-task-queue-type":"Activity","store-operation":"update-task-queue","error":"UpdateTaskQueue failed. Failed to start transaction. Error: context deadline exceeded","logging-call-at":"taskReader.go:166","stacktrace":"<http://go.temporal.io/server/common/log/loggerimpl.(*loggerImpl).Error\n\t/temporal/common/log/loggerimpl/logger.go:138\ngo.temporal.io/server/service/matching.(*taskReader).getTasksPump\n\t/temporal/service/matching/taskReader.go:166|go.temporal.io/server/common/log/loggerimpl.(*loggerImpl).Error\n\t/temporal/common/log/loggerimpl/logger.go:138\ngo.temporal.io/server/service/matching.(*taskReader).getTasksPump\n\t/temporal/service/matching/taskReader.go:166>"}
airbyte-scheduler | 2021-03-24 16:09:23 INFO i.a.s.JobRetrier(run):53 - {workspace_app_root=/tmp/workspace/scheduler/logs} - Running Job Retrier...
maybe thats because my Internet stopped working, but the main problem is next: CPU usage increased a lot ( and probably memory leak too ), so my pc almost got freezed. Is it possible to limit usage somehow? I suppose I am not the only one who can face such issue.Micah
03/25/2021, 3:41 AMThomas
03/26/2021, 9:01 AMThomas
03/26/2021, 9:01 AMDavin Chia (Airbyte)
Davin Chia (Airbyte)
CORE_ONLY=1 ./gradlew build
2. ./gradlew :airbyte-integrations:connectors:destination-redshift:test
I’m on a 2017 Macbook pro while I wait for the M1 Pro to come out. Wondering if upgrading to a better Mac for half a year or so is worth its
2018-11-13T20:20:39+00:00
.
• `"time"`: Time, for example, 20:20:39+00:00
• `"date"`: Date, for example, 2018-11-13
https://json-schema.org/understanding-json-schema/reference/string.html#dates-and-timesJared Rhizor (Airbyte)
Davin Chia (Airbyte)
JDBCDestination
class for? Nothing seems to use itDavin Chia (Airbyte)
Jared Rhizor (Airbyte)
Davin Chia (Airbyte)
AbstractJdbcSourceStandardTest
in their intellij? works for me on gradle but not in intellijs
updated_at
, reading from the source records whose updated_at
is greater than the largest updated_at
from the previous sync.
So far I have only encountered instances where these cursor fields are at the “top-level” i.e: not nested within the record. But I’m curious to understand how prevalent nested cursor fields are e.g: date_info.updated_at
Davin Chia (Airbyte)
Harshith (Airbyte)
file
connection. There is a number
field in the csv data and when we are pushing to bigquery
this is taken as null. can someone help me with this ?Davin Chia (Airbyte)
Harshith (Airbyte)
fieldInclusions
this is not populating all important information. For reference link is the line where this is done. Any possible solutions for this ?Armand
04/10/2021, 8:58 PMDavin Chia (Airbyte)
airbyte-scheduler | 2021-04-12 12:21:46 ERROR i.t.i.w.Poller(lambda$new$0):70 - {} - Failure in thread Activity Poller taskQueue="SYNC", namespace="default": 1
airbyte-scheduler | io.grpc.StatusRuntimeException: INTERNAL: Not enough hosts to serve the request
airbyte-scheduler | at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262) ~[grpc-stub-1.34.1.jar:1.34.1]
airbyte-scheduler | at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243) ~[grpc-stub-1.34.1.jar:1.34.1]
airbyte-scheduler | at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156) ~[grpc-stub-1.34.1.jar:1.34.1]
airbyte-scheduler | at io.temporal.api.workflowservice.v1.WorkflowServiceGrpc$WorkflowServiceBlockingStub.pollActivityTaskQueue(WorkflowServiceGrpc.java:2696) ~[temporal-serviceclient-1.0.4.jar:?]
airbyte-scheduler | at io.temporal.internal.worker.ActivityPollTask.poll(ActivityPollTask.java:105) ~[temporal-sdk-1.0.4.jar:?]
airbyte-scheduler | at io.temporal.internal.worker.ActivityPollTask.poll(ActivityPollTask.java:39) ~[temporal-sdk-1.0.4.jar:?]
airbyte-scheduler | at io.temporal.internal.worker.Poller$PollExecutionTask.run(Poller.java:265) ~[temporal-sdk-1.0.4.jar:?]
airbyte-scheduler | at io.temporal.internal.worker.Poller$PollLoopTask.run(Poller.java:241) [temporal-sdk-1.0.4.jar:?]
airbyte-scheduler | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
airbyte-scheduler | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
airbyte-scheduler | at java.lang.Thread.run(Thread.java:832) [?:?]
happens when I put load on the my system. e.g. run now I'm running Acceptance Tests as well as a full ./gradlew build
s
Mason Wheeler
04/12/2021, 6:26 PMDavin Chia (Airbyte)
AbstractQueue
(struct.toString()
) without overriding toString
actually empties the queue.. thanks @charles for the assist!Davin Chia (Airbyte)
Vasily Safronov
04/15/2021, 7:48 AM2021-04-14 20:39:11 INFO (/tmp/workspace/8ea53a05-4867-4639-9f70-744cf07739ea/0) DefaultAirbyteStreamFactory(lambda$create$0):73 - org.postgresql.util.PSQLException: ERROR: permission denied for database test_db
but this message didn’t tell me anything about what specific query request was sent to the db when this error occurred.
During debugged I saw that the problem is not in the wrong password or the connector is not working correctly, but in the following query: CREATE SCHEMA IF NOT EXISTS public;
during this method call. My db user just doesn’t have permissions to create a schema.
So, if user will able to see specific queries logs he will have possibility to catching troubles like this with easy way and it will more friendly user experience.Harshith (Airbyte)
Roshan George
04/15/2021, 5:47 PM{"connection_id": 142, "status": "success"}
to some arbitrary web URL?Jared Rhizor (Airbyte)
Davin Chia (Airbyte)