https://linen.dev logo
#troubleshooting
Title
# troubleshooting
a

Andrés Bravo

04/05/2022, 9:17 PM
I’m having this same issue when upgrading from
v0.35.3-beta
to
v0.35.46-beta
upgrading is fine but after that, export and import the same file fails with this message. Going
v0.35.30-beta
first does not help.
Copy code
airbyte-server      | java.lang.RuntimeException: io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema.
airbyte-server      | Errors: $.id: is missing but it is required, $.catalog: is missing but it is required, $.catalogHash: is missing but it is required
airbyte-server      | Schema:
airbyte-server      | {
airbyte-server      |   "$schema" : "<http://json-schema.org/draft-07/schema#>",
airbyte-server      |   "$id" : "<https://github.com/airbytehq/airbyte/blob/master/airbyte-config/models/src/main/resources/types/AttemptFailureSummary.yaml>",
airbyte-server      |   "title" : "ActorCatalog",
airbyte-server      |   "description" : "Catalog of an actor.",
airbyte-server      |   "type" : "object",
airbyte-server      |   "additionalProperties" : false,
airbyte-server      |   "required" : [ "id", "catalog", "catalogHash" ],
airbyte-server      |   "properties" : {
airbyte-server      |     "id" : {
airbyte-server      |       "type" : "string",
airbyte-server      |       "format" : "uuid"
airbyte-server      |     },
airbyte-server      |     "catalog" : {
airbyte-server      |       "type" : "object",
airbyte-server      |       "existingJavaType" : "com.fasterxml.jackson.databind.JsonNode"
airbyte-server      |     },
airbyte-server      |     "catalogHash" : {
airbyte-server      |       "type" : "string"
airbyte-server      |     }
airbyte-server      |   }
airbyte-server      | }
airbyte-server      | 	at io.airbyte.server.ConfigDumpImporter.lambda$readConfigsFromArchive$2(ConfigDumpImporter.java:203) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
airbyte-server      | 	at java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:1003) ~[?:?]
airbyte-server      | 	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845) ~[?:?]
airbyte-server      | 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
airbyte-server      | 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
airbyte-server      | 	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
airbyte-server      | 	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
airbyte-server      | 	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
airbyte-server      | 	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
airbyte-server      | 	at io.airbyte.config.persistence.DatabaseConfigPersistence.lambda$replaceAllConfigs$89(DatabaseConfigPersistence.java:1585) ~[io.airbyte.airbyte-config-persistence-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.db.Database.lambda$transaction$0(Database.java:31) ~[io.airbyte.airbyte-db-lib-0.35.46-alpha.jar:?]
airbyte-server      | 	at org.jooq.impl.DefaultDSLContext.lambda$transactionResult0$0(DefaultDSLContext.java:564) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at org.jooq.impl.Tools$12$1.block(Tools.java:4926) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463) ~[?:?]
airbyte-server      | 	at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434) ~[?:?]
airbyte-server      | 	at org.jooq.impl.Tools$12.get(Tools.java:4923) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at org.jooq.impl.DefaultDSLContext.transactionResult0(DefaultDSLContext.java:616) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at org.jooq.impl.DefaultDSLContext.transactionResult(DefaultDSLContext.java:533) ~[jooq-3.13.4.jar:?]
airbyte-server      | 	at io.airbyte.db.Database.transaction(Database.java:31) ~[io.airbyte.airbyte-db-lib-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.db.ExceptionWrappingDatabase.transaction(ExceptionWrappingDatabase.java:31) ~[io.airbyte.airbyte-db-lib-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.config.persistence.DatabaseConfigPersistence.replaceAllConfigs(DatabaseConfigPersistence.java:1493) ~[io.airbyte.airbyte-config-persistence-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.config.persistence.ValidatingConfigPersistence.replaceAllConfigs(ValidatingConfigPersistence.java:100) ~[io.airbyte.airbyte-config-persistence-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.config.persistence.ConfigRepository.replaceAllConfigs(ConfigRepository.java:851) ~[io.airbyte.airbyte-config-persistence-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.ConfigDumpImporter.importConfigsFromArchive(ConfigDumpImporter.java:185) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.ConfigDumpImporter.importDataWithSeed(ConfigDumpImporter.java:130) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.handlers.ArchiveHandler.lambda$importData$0(ArchiveHandler.java:102) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.handlers.ArchiveHandler.importInternal(ArchiveHandler.java:133) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.handlers.ArchiveHandler.importData(ArchiveHandler.java:102) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.apis.ConfigurationApi.lambda$importArchive$75(ConfigurationApi.java:711) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.apis.ConfigurationApi.execute(ConfigurationApi.java:735) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at io.airbyte.server.apis.ConfigurationApi.importArchive(ConfigurationApi.java:711) ~[io.airbyte-airbyte-server-0.35.46-alpha.jar:?]
airbyte-server      | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
airbyte-server      | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
airbyte-server      | 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
airbyte-server      | 	at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
OK seems like it’s related to files
ACTOR_CATALOG.yaml ACTOR_CATALOG_FETCH_EVENT.yaml
after deleting those files the exported configuration can be imported again.
@Marcos Marx (Airbyte) Is this the correct thread to speak about those issues or should I open a new one?
l

Luke Olson

04/07/2022, 3:54 PM
@Malik Diarra can you think of a temp workaround in the meantime? Running into this on
0.35.51
, too
m

Malik Diarra (Airbyte)

04/07/2022, 3:58 PM
Hi, probably the correct thread indeed. Let me take a look at that this morning. We indeed had a bug related to those tables that we corrected a couple of weeks ago.
The fix mentioned above by Andres is a good workaround. You can safely remove those entries from the exported archive and run the import again.
The export function was bugged for some older version (0.35.31 to 0.35.56) of Airbyte, it is fixed now but if you try to import export from those in one of the latest version you’ll run into those issues.
l

Luke Olson

04/08/2022, 3:51 PM
Thanks Malik. Unfortunately removing those two files, re-zipping, and importing didn’t work. Now hitting this issue:
Copy code
airbyte-server-7c755b75b9-xbn7d airbyte-server-container 2022-04-08 15:39:57 ERROR i.a.s.h.ArchiveHandler(importInternal):136 - Import failed                                                 airbyte-server-7c755b75b9-xbn7d airbyte-server-container java.nio.file.NoSuchFileException: /tmp/airbyte_archive9599233141101451698/VERSION
I confirmed the
VERSION
file is in the correct spot in the newly zipped file, so that doesn’t appear to be the issue…
But it sounds like if we update both instances to 0.35.57+ then we shouldn’t hit this (initial) issue?
m

Malik Diarra (Airbyte)

04/08/2022, 3:54 PM
Yes, please try upgrading both instance.
l

Luke Olson

04/08/2022, 4:07 PM
I spoke too soon, I think the mac default unzip -> tar zip caused some issues in how airbyte read the compressed file, but we got it working now! Will try the upgrade if there are still any issues, though. Thanks for the input!
m

Malik Diarra (Airbyte)

04/08/2022, 4:10 PM
Great! Thanks for the feedback and sorry for the poor experience here.
4 Views