https://linen.dev logo
Join Slack
Powered by
I’m having this same issue when upgrading from `v0...
# troubleshooting
a
I’m having this same issue when upgrading from 
v0.35.3-beta
to 
v0.35.46-beta
upgrading is fine but after that, export and import the same file fails with this message. Going 
v0.35.30-beta
first does not help.
Copy code
airbyte-server      | java.lang.RuntimeException: io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema.
airbyte-server      | Errors: $.id: is missing but it is required, $.catalog: is missing but it is required, $.catalogHash: is missing but it is required
airbyte-server      | Schema:
airbyte-server      | {
airbyte-server      |   "$schema" : "<http://json-schema.org/draft-07/schema#>",
airbyte-server      |   "$id" : "<https://github.com/airbytehq/airbyte/blob/master/airbyte-config/models/src/main/resources/types/AttemptFailureSummary.yaml>",
airbyte-server      |   "title" : "ActorCatalog",
airbyte-server      |   "description" : "Catalog of an actor.",
airbyte-server      |   "type" : "object",
airbyte-server      |   "additionalProperties" : false,
airbyte-server      |   "required" : [ "id", "catalog", "catalogHash" ],
airbyte-server      |   "properties" : {
airbyte-server      |     "id" : {
airbyte-server      |       "type" : "string",
airbyte-server      |       "format" : "uuid"
airbyte-server      |     },
airbyte-server      |     "catalog" : {
airbyte-server      |       "type" : "object",
airbyte-server      |       "existingJavaType" : "com.fasterxml.jackson.databind.JsonNode"
airbyte-server      |     },
airbyte-server      |     "catalogHash" : {
airbyte-server      |       "type" : "string"
airbyte-server      |     }
airbyte-server      |   }
airbyte-server      | }
airbyte-server      | 	at io.airbyte.server.ConfigDumpImporter.lambda$readConfigsFromArchive$2(ConfigDumpImporter.java:203) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
airbyte-server      | 	at java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:1003) ~[?:?]
airbyte-server      | 	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845) ~[?:?]
airbyte-server      | 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
airbyte-server      | 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
airbyte-server      | 	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
airbyte-server      | 	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
airbyte-server      | 	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
airbyte-server      | 	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
airbyte-server      | 	at io.airbyte.config.persistence.DatabaseConfigPersistence.lambda$replaceAllConfigs$89(DatabaseConfigPersistence.java:1585) ~[io.airbyte.airbyte-config-persistence-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.db.Database.lambda$transaction$0(Database.java:31) ~[io.airbyte.airbyte-db-lib-0.35.46-alpha.jar:?]
airbyte-server      | 	at org.jooq.impl.DefaultDSLContext.lambda$transactionResult0$0(DefaultDSLContext.java:564) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at org.jooq.impl.Tools$12$1.block(Tools.java:4926) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463) ~[?:?]
airbyte-server      | 	at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434) ~[?:?]
airbyte-server      | 	at org.jooq.impl.Tools$12.get(Tools.java:4923) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at org.jooq.impl.DefaultDSLContext.transactionResult0(DefaultDSLContext.java:616) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at org.jooq.impl.DefaultDSLContext.transactionResult(DefaultDSLContext.java:533) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at io.airbyte.db.Database.transaction(Database.java:31) ~[io.airbyte.airbyte-db-lib-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.db.ExceptionWrappingDatabase.transaction(ExceptionWrappingDatabase.java:31) ~[io.airbyte.airbyte-db-lib-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.config.persistence.DatabaseConfigPersistence.replaceAllConfigs(DatabaseConfigPersistence.java:1493) ~[io.airbyte.airbyte-config-persistence-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.config.persistence.ValidatingConfigPersistence.replaceAllConfigs(ValidatingConfigPersistence.java:100) ~[io.airbyte.airbyte-config-persistence-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.config.persistence.ConfigRepository.replaceAllConfigs(ConfigRepository.java:851) ~[io.airbyte.airbyte-config-persistence-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.ConfigDumpImporter.importConfigsFromArchive(ConfigDumpImporter.java:185) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.ConfigDumpImporter.importDataWithSeed(ConfigDumpImporter.java:130) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.handlers.ArchiveHandler.lambda$importData$0(ArchiveHandler.java:102) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.handlers.ArchiveHandler.importInternal(ArchiveHandler.java:133) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.handlers.ArchiveHandler.importData(ArchiveHandler.java:102) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.apis.ConfigurationApi.lambda$importArchive$75(ConfigurationApi.java:711) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.apis.ConfigurationApi.execute(ConfigurationApi.java:735) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.apis.ConfigurationApi.importArchive(ConfigurationApi.java:711) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
airbyte-server      | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
airbyte-server      | 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
airbyte-server      | 	at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
OK seems like it’s related to files 
ACTOR_CATALOG.yaml ACTOR_CATALOG_FETCH_EVENT.yaml
after deleting those files the exported configuration can be imported again.
@Marcos Marx (Airbyte) Is this the correct thread to speak about those issues or should I open a new one?
l
@Malik Diarra can you think of a temp workaround in the meantime? Running into this on 
0.35.51
, too
m
Hi, probably the correct thread indeed. Let me take a look at that this morning. We indeed had a bug related to those tables that we corrected a couple of weeks ago.
The fix mentioned above by Andres is a good workaround. You can safely remove those entries from the exported archive and run the import again.
The export function was bugged for some older version (0.35.31 to 0.35.56) of Airbyte, it is fixed now but if you try to import export from those in one of the latest version you’ll run into those issues.
l
Thanks Malik. Unfortunately removing those two files, re-zipping, and importing didn’t work. Now hitting this issue:
Copy code
airbyte-server-7c755b75b9-xbn7d airbyte-server-container 2022-04-08 15:39:57 ERROR i.a.s.h.ArchiveHandler(importInternal):136 - Import failed                                                 airbyte-server-7c755b75b9-xbn7d airbyte-server-container java.nio.file.NoSuchFileException: /tmp/airbyte_archive9599233141101451698/VERSION
I confirmed the 
VERSION
file is in the correct spot in the newly zipped file, so that doesn’t appear to be the issue…
But it sounds like if we update both instances to 0.35.57+ then we shouldn’t hit this (initial) issue?
m
Yes, please try upgrading both instance.
l
I spoke too soon, I think the mac default unzip -> tar zip caused some issues in how airbyte read the compressed file, but we got it working now! Will try the upgrade if there are still any issues, though. Thanks for the input!
m
Great! Thanks for the feedback and sorry for the poor experience here.
4 Views
Open in Slack
PreviousNext
Powered by