Aditya
05/13/2021, 5:56 AMRahul Padhy
05/13/2021, 7:41 AMPing
05/13/2021, 10:34 PM[ec2-user@ip-172-31-45-53 ~]$ docker run --rm -v /home/ec2-user/0.14.1-alpha:/config airbyte/migration:0.22.3-alpha -- --input /config/airbyte_archive.tar.gz --output /config/airbyte_archive_migrated.tar.gz --target-version 0.22.3-alpha
2021-05-13 22:31:38 INFO i.a.m.MigrationRunner(parse):78 - {} - args: [--input, /config/airbyte_archive.tar.gz, --output, /config/airbyte_archive_migrated.tar.gz, --target-version, 0.22.3-alpha]
2021-05-13 22:31:38 INFO i.a.m.MigrationRunner(run):50 - {} - Unpacking tarball
2021-05-13 22:31:38 INFO i.a.m.MigrationRunner(run):67 - {} - Running migrations...
2021-05-13 22:31:38 INFO i.a.m.MigrationRunner(run):68 - {} - MigrateConfig{inputPath=/tmp/airbyte_migrate1347150974477562104/uncompressed, outputPath=/tmp/airbyte_migrate1347150974477562104/output, targetVersion='0.22.3-alpha'}
2021-05-13 22:31:39 INFO i.a.m.Migrate(run):88 - {} - Starting migrations. Current version: 0.14.1-alpha, Target version: 0.22.3-alpha
Exception in thread "main" java.lang.IllegalArgumentException: No migration found for target version: 0.22.3-alpha
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
at io.airbyte.migrate.Migrate.run(Migrate.java:101)
at io.airbyte.migrate.MigrationRunner.run(MigrationRunner.java:70)
at io.airbyte.migrate.MigrationRunner.main(MigrationRunner.java:108)
Ping
05/14/2021, 12:04 AMWaiting for logs...
The console output shows this error repeating over and over:
airbyte-scheduler | 2021-05-14 00:04:23 ERROR i.a.c.l.Exceptions(swallow):84 - {workspace_app_root=/tmp/workspace/scheduler/logs} - Swallowed error.
airbyte-scheduler | java.lang.IllegalArgumentException: Job type getSpec is not allowed!
airbyte-scheduler | at com.google.common.base.Preconditions.checkArgument(Preconditions.java:142) ~[guava-30.1-jre.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.persistence.job_tracker.JobTracker.lambda$trackSync$3(JobTracker.java:129) ~[io.airbyte.airbyte-scheduler-persistence-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.commons.lang.Exceptions.swallow(Exceptions.java:82) [io.airbyte-airbyte-commons-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.persistence.job_tracker.JobTracker.trackSync(JobTracker.java:126) [io.airbyte.airbyte-scheduler-persistence-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.app.JobSubmitter.trackSubmission(JobSubmitter.java:129) [io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.app.JobSubmitter.lambda$run$0(JobSubmitter.java:71) [io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at java.util.Optional.ifPresent(Optional.java:176) [?:?]
airbyte-scheduler | at io.airbyte.scheduler.app.JobSubmitter.run(JobSubmitter.java:70) [io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.app.SchedulerApp.lambda$start$0(SchedulerApp.java:137) [io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
airbyte-scheduler | at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
airbyte-scheduler | at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
airbyte-scheduler | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
airbyte-scheduler | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
airbyte-scheduler | at java.lang.Thread.run(Thread.java:832) [?:?]
airbyte-scheduler | 2021-05-14 00:04:23 ERROR i.a.s.a.JobSubmitter(run):78 - {workspace_app_root=/tmp/workspace/scheduler/logs} - Job Submitter Error
airbyte-scheduler | java.lang.IllegalArgumentException: Does not support job type: GET_SPEC
airbyte-scheduler | at io.airbyte.scheduler.app.worker_run.TemporalWorkerRunFactory.createSupplier(TemporalWorkerRunFactory.java:78) ~[io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.app.worker_run.TemporalWorkerRunFactory.create(TemporalWorkerRunFactory.java:55) ~[io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.app.JobSubmitter.submitJob(JobSubmitter.java:84) ~[io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.app.JobSubmitter.lambda$run$0(JobSubmitter.java:72) ~[io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at java.util.Optional.ifPresent(Optional.java:176) ~[?:?]
airbyte-scheduler | at io.airbyte.scheduler.app.JobSubmitter.run(JobSubmitter.java:70) [io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at io.airbyte.scheduler.app.SchedulerApp.lambda$start$0(SchedulerApp.java:137) [io.airbyte.airbyte-scheduler-app-0.22.3-alpha.jar:?]
airbyte-scheduler | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
airbyte-scheduler | at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
airbyte-scheduler | at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
airbyte-scheduler | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
airbyte-scheduler | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
airbyte-scheduler | at java.lang.Thread.run(Thread.java:832) [?:?]
Aleksandr
05/14/2021, 8:44 AMAdhithya Ravichandran
05/14/2021, 4:11 PMDefaultAirbyteStreamFactory(lambda$create$0):73 - W, [2021-05-14T16:07:49.870271 #8] WARN -- : MONGODB | Error running ismaster on <redacted server name>.<http://mongodb.net:27017|mongodb.net:27017>: SocketError: getaddrinfo: Name does not resolve
2021-05-14 16:08:50 INFO (/tmp/workspace/c0bdcfee-0a4d-48c0-8de1-560e9908d59b/0) DefaultAirbyteStreamFactory(internalLog):110 - [2021-05-14 16:07:50 +0000] WARN : | "MONGODB | Error running ismaster on <redacted server name>.<http://mongodb.net:27017|mongodb.net:27017>: SocketError: getaddrinfo: Name does not resolve"
2021-05-14 16:08:50 INFO (/tmp/workspace/c0bdcfee-0a4d-48c0-8de1-560e9908d59b/0) TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...
Help is greatly appreciated 🙂Matt Fleshman
05/17/2021, 3:40 PMMané Rom
05/17/2021, 3:53 PMAnton Nosovitsky
05/18/2021, 8:55 AMAnton Nosovitsky
05/18/2021, 10:02 AMRahul Padhy
05/18/2021, 12:25 PMPriyanka MCP
05/19/2021, 3:48 PMA problem occurred starting process 'command 'E:\Priyanka\workspace\Airbyte\airbyte\tools\bin\build_image.sh''Any pointers please
Warren Gaspay
05/20/2021, 8:08 AMRahul Padhy
05/20/2021, 9:45 AMAlex Koshterek
05/21/2021, 9:58 AMpacker
script building a VM and switching to new Airbyte is a matter of 1 line change.
Also are there e2e examples of API calls for setting up connections?Jeff Crooks
05/21/2021, 2:18 PMJohn Gaspar
05/22/2021, 6:11 AMJohn Gaspar
05/22/2021, 6:25 AMJohn Gaspar
05/22/2021, 7:26 AMUrbunMe
05/22/2021, 10:15 AMRahul Padhy
05/25/2021, 11:41 AMimport requests
BASE_URL = '<http://localhost:8001/api>'
get_source = '/v1/sources/get'
file_source = {"sourceDefinitionId": "778daa7c-feaf-4db6-96f3-70fd645acc77"}
full_url = BASE_URL + get_source
response = <http://requests.post|requests.post>(full_url, json=file_source)
print (response.text)
And here’s the output that I’m getting :-
{"message":"Invalid JSON","details":"Unrecognized field \"sourceDefinitionId\" (class io.airbyte.api.model.SourceIdRequestBody), not marked as ignorable"}
I’d gotten the sourceDefinitionId for source type Files from the response of the list-all-sourceDefinitions endpoint -> https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#post-/v1/source_definitions/list
Please do tell me where I’m going wrong.Rahul Padhy
05/25/2021, 2:12 PMKRISHNA MALLIK
05/25/2021, 2:57 PMIevgenii Tsybaiev [GL]
05/25/2021, 7:10 PMDan Spohn
05/26/2021, 8:15 PMRahul Padhy
05/27/2021, 8:12 AMRonen Yaari
05/27/2021, 9:19 AMRahul Padhy
05/27/2021, 10:10 AMDaniel Mateus Pires (Earnest Research)
05/27/2021, 10:24 AM2021-05-27 10:22:09 ERROR (/tmp/workspace/2/0) LineGobbler(voidCall):69 - data
2021-05-27 10:22:09 ERROR (/tmp/workspace/2/0) LineGobbler(voidCall):69 - value is not a valid dict (type=type_error.dict)
2021-05-27 10:22:10 INFO (/tmp/workspace/2/0) DefaultReplicationWorker(run):134 - Source thread complete.
2021-05-27 10:22:10 INFO (/tmp/workspace/2/0) DefaultReplicationWorker(run):135 - Waiting for destination thread to join.
2021-05-27 10:22:11 INFO (/tmp/workspace/2/0) DefaultReplicationWorker(run):137 - Destination thread complete.
2021-05-27 10:22:11 ERROR (/tmp/workspace/2/0) DefaultReplicationWorker(run):141 - Sync worker failed.
in the logs on version 0.24.1-alpha
Steve
05/27/2021, 5:08 PM