https://linen.dev logo
Join Slack
Powered by
you can’t get the API to return them right now bec...
# dev-frontend
c
you can’t get the API to return them right now because are hiding them since they don’t work in the UI, but you can see an example of one on master here: https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-hubspot-singer/resources/spec.json
a
yeah, thats what I am trying to test it on. Just wanted to test with less hardcoded values
c
oh
you know what the file source actually has a one of
and is expose in the ui (by accident 🤷 )
🙄 1
but i guess it’s there so you can use it. 🤣
a
Are those changes in master?
c
sorry, which ones?
a
you know what the file source actually has a one of
File source 🙂
c
yeah! this one ^
and it should be available in the ui
a
hm, probably I am running old backend. I will check
c
kk.
just realized the docker image wasn’t pushed for it, bu that shoudln’t prevent it from appearing the ui.
(pushing the docker image now)
everything working okay for you?
a
I have not checked yet
hmm nope. I have file source, but it doesn’t have 
oneOf
Can I hack it a bit somehow? I really need it to test on real oneOf source … Maybe I can just change schema some way, so it will return oneOf, but backend will still consume it correctly
@s not quite sure whether Charles will be available today, so if you can guide me a bit how to resolve the last question - it would be really awesome 🙂
s
charles is out
let me catch up
@Artem Astapenko I’m looking at the File Source right now — it looks like it contains a oneOf, although it’s using the “dependencies” attribute of JSON schema. is that supported on the frontend?
I’m not caught up on exactly which variations of OneOf are supported
c
Use the one in source hubspot.
I think it is more standard.
We can adjust the one in source file if it is non considered standard.
Though i think both are valid.
a
I do not think oneOf in dependencies will help me as we look into properties field. I can’t see hubspot source type. Is it in master?
s
i’m adding it to a branch
one second
it’s available on this branch 
sherif/register-hubspot2
💪 1
if you merge it into your branch then 
docker-compose down -v
and up it should appear in the UI
a
awesome thank you! I will check a bit later today
I have
Copy code
javax.ws.rs.NotFoundException: HTTP 404 Not Found
airbyte-server | 	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:250) [jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [jersey-server-2.31.jar:?]
should I rebuild something or just 
up
is enough?
s
1. in the file you merged from 
sherif/register-hubspot2
change the version from 
<http://0.1.to|0.1.0
to> 
dev
2. 
docker-compose down -v
and rebuild 3. run 
./tools/integrations/manage.sh build airbyte-integrations/connectors/source-hubspot
4. VERSION=dev docker-compose up
a
By rebuild you mean 
./gradle clean build
right?
s
yes
./gradlew
a
yeah exactly. Ok thanks. Btw, why is it required to call 
./tools/integrations/manage.sh build airbyte-integrations/connectors/source-hubspot
?
s
this rebuilds the integration image
we don’t build all integration images by default in the build since it would make it very long
a
oh ok. Thank you a lot for explanation.
s
np
how is it looking?
works?
a
hehe, gradle build just finished 🙂 It takes almost 10 minutes on my PC 🙂 Also I suppose you mentioned 
source-hubspot-singer
right?
🤯 1
s
yes
a
ok. Waiting the build to finish
Copy code
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

tap-hubspot 2.8.1 requires requests==2.20.0, but you'll have requests 2.24.0 which is incompatible.
WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
but build passes. Should I ignore that?
s
yes
a
Copy code
airbyte-server | 2020-11-05 23:50:38 DEBUG i.a.s.e.UncaughtExceptionMapper(toResponse):42 - {} - Uncaught exception
airbyte-server | java.lang.RuntimeException: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "sourceId" (class io.airbyte.config.StandardSourceDefinition), not marked as ignorable (5 known properties: "sourceDefinitionId", "dockerRepository", "name", "documentationUrl", "dockerImageTag"])
airbyte-server |  at [Source: (String)"{
airbyte-server |   "sourceId": "57eb1576-8f52-463d-beb6-2e107cdf571d",
airbyte-server |   "name": "Hubspot",
airbyte-server |   "dockerRepository": "airbyte/source-hubspot-singer",
airbyte-server |   "dockerImageTag": "0.1.0",
airbyte-server |   "documentationUrl": "<https://https>://docs.airbyte.io/integrations/sources/hubspot"
airbyte-server | }
airbyte-server | "; line: 2, column: 16] (through reference chain: io.airbyte.config.StandardSourceDefinition["sourceId"])
airbyte-server | 	at io.airbyte.commons.json.Jsons.deserialize(Jsons.java:52) ~[io.airbyte-airbyte-commons-0.2.1-alpha.jar:?]
airbyte-server | 	at io.airbyte.config.persistence.DefaultConfigPersistence.getConfigInternal(DefaultConfigPersistence.java:98) ~[io.airbyte.airbyte-config-persistence-0.2.1-alpha.jar:?]
airbyte-server | 	at io.airbyte.config.persistence.DefaultConfigPersistence.getConfig(DefaultConfigPersistence.java:65) ~[io.airbyte.airbyte-config-persistence-0.2.1-alpha.jar:?]
airbyte-server | 	at io.airbyte.config.persistence.DefaultConfigPersistence.listConfigsInternal(DefaultConfigPersistence.java:119) ~[io.airbyte.airbyte-config-persistence-0.2.1-alpha.jar:?]
airbyte-server | 	at io.airbyte.config.persistence.DefaultConfigPersistence.listConfigs(DefaultConfigPersistence.java:74) ~[io.airbyte.airbyte-config-persistence-0.2.1-alpha.jar:?]
airbyte-server | 	at io.airbyte.config.persistence.ConfigRepository.listStandardSources(ConfigRepository.java:102) ~[io.airbyte.airbyte-config-persistence-0.2.1-alpha.jar:?]
airbyte-server | 	at io.airbyte.server.handlers.SourceDefinitionsHandler.listSourceDefinitions(SourceDefinitionsHandler.java:66) ~[io.airbyte-airbyte-server-0.2.1-alpha.jar:?]
airbyte-server | 	at io.airbyte.server.apis.ConfigurationApi.execute(ConfigurationApi.java:349) ~[io.airbyte-airbyte-server-0.2.1-alpha.jar:?]
airbyte-server | 	at io.airbyte.server.apis.ConfigurationApi.listSourceDefinitions(ConfigurationApi.java:158) ~[io.airbyte-airbyte-server-0.2.1-alpha.jar:?]
airbyte-server | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
airbyte-server | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
airbyte-server | 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
airbyte-server | 	at java.lang.reflect.Method.invoke(Method.java:564) ~[?:?]
airbyte-server | 	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) [jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [jersey-common-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [jersey-server-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) [jersey-container-servlet-core-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) [jersey-container-servlet-core-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) [jersey-container-servlet-core-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) [jersey-container-servlet-core-2.31.jar:?]
airbyte-server | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) [jersey-container-servlet-core-2.31.jar:?]
airbyte-server | 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) [jetty-servlet-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569) [jetty-servlet-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) [jetty-servlet-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.Server.handle(Server.java:501) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) [jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) [jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) [jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) [jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) [jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723]
airbyte-server | 	at java.lang.Thread.run(Thread.java:832) [?:?]
Copy code
airbyte-server | Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "sourceId" (class io.airbyte.config.StandardSourceDefinition), not marked as ignorable (5 known properties: "sourceDefinitionId", "dockerRepository", "name", "documentationUrl", "dockerImageTag"])
airbyte-server |  at [Source: (String)"{
airbyte-server |   "sourceId": "57eb1576-8f52-463d-beb6-2e107cdf571d",
airbyte-server |   "name": "Hubspot",
airbyte-server |   "dockerRepository": "airbyte/source-hubspot-singer",
airbyte-server |   "dockerImageTag": "0.1.0",
airbyte-server |   "documentationUrl": "<https://https>://docs.airbyte.io/integrations/sources/hubspot"
airbyte-server | }
airbyte-server | "; line: 2, column: 16] (through reference chain: io.airbyte.config.StandardSourceDefinition["sourceId"])
airbyte-server | 	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:843) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1206) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1610) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1588) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:294) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4218) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3214) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3182) ~[jackson-databind-2.10.4.jar:2.10.4]
airbyte-server | 	at io.airbyte.commons.json.Jsons.deserialize(Jsons.java:50) ~[io.airbyte-airbyte-commons-0.2.1-alpha.jar:?]
airbyte-server | 	... 60 more
airbyte-server | Nov 05, 2020 11:50:38 PM org.glassfish.jersey.logging.LoggingInterceptor log
airbyte-server | INFO: 18 * Server responded with a response on thread qtp1234905692-19
Thats what I see when I try to call
Copy code
<http://localhost:8001/api/v1/source_definitions/list>
What I have done - switched to your branch, build everything, called manage.sh. So far nothing failed and I started frontend
s
gaaah
I did a silly mistake
related to the naming
can you change the field 
sourceId
to 
sourceDefinitionId
in the json you merged from my branch?
you’ll have to rebuild the image with the 
./tools/integrations/manage.sh
command
a
ok
Is it possible things are cached/stored somewhere?
I rebuilt the image with manage.sh and still have
Copy code
at [Source: (String)"{
airbyte-server |   "sourceId": "57eb1576-8f52-463d-beb6-2e107cdf571d",
airbyte-server |   "name": "Hubspot",
airbyte-server |   "dockerRepository": "airbyte/source-hubspot-singer",
airbyte-server |   "dockerImageTag": "0.1.0",
airbyte-server |   "documentationUrl": "<https://https>://docs.airbyte.io/integrations/sources/hubspot"
airbyte-server | }
even though I changed sourceId to sourceDefinitionId like in other files.
s
docker-compose down -v
and up again
ah crap you might need to build again
a
thats exactly what I have done 2 times 🙂 Lets try one more time
yay. ok
Ok, so I was able to list hubspot. Though I still have next error 
Error while getting spec from image airbyte/source-hubspot-singer:0.1.0:
but I suppose thats just because I forgot to change 0.1.0 to dev. Thank you for help!
s
a quick way to fix this locally is to run 
docker build . -t airbyte/source-hubspot-singer:0.1.0
from 
airbyte-integrations/connectors/source-hubspot-singer
this will just override v0.1.0
and you don’t need to change to dev
and rebuild all of gradle again
Open in Slack
PreviousNext
Powered by