Hi I m a developer working on adding a new Python HTTP API s

Question

Hi I m a developer working on adding a new Python HTTP API source connector following the guide I ve got things working up to <https docs airbyte com connector development tutorials cdk tutorial python http 7 use connector in airbyte|this step of adding the connector to UI> and somehow can t add the source connector locally following the instruction From the docker server log it seems like it cannot get spec Can someone help

Phoebe Yang · Answer

```java lang IllegalStateException Get Spec job failed at com google common base Preconditions checkState Preconditions java 502 ~ guava 31 0 1 jre jar at io airbyte server converters SpecFetcher getSpecFromJob SpecFetcher java 14 ~ io airbyte airbyte server 0 35 11 alpha jar at io airbyte server handlers SourceDefinitionsHandler getSpecForImage SourceDefinitionsHandler java 174 ~ io airbyte airbyte server 0 35 11 alpha jar at io airbyte server handlers SourceDefinitionsHandler updateSourceDefinition SourceDefinitionsHandler java 137 ~ io airbyte airbyte server 0 35 11 alpha jar at io airbyte server apis ConfigurationApi lambda$updateSourceDefinition$9 ConfigurationApi java 306 ~ io airbyte airbyte server 0 35 11 alpha jar at io airbyte server apis ConfigurationApi execute ConfigurationApi java 733 ~ io airbyte airbyte server 0 35 11 alpha jar at io airbyte server apis ConfigurationApi updateSourceDefinition ConfigurationApi java 306 ~ io airbyte airbyte server 0 35 11 alpha jar at jdk internal reflect NativeMethodAccessorImpl invoke0 Native Method ~ at jdk internal reflect NativeMethodAccessorImpl invoke NativeMethodAccessorImpl java 77 ~ at jdk internal reflect DelegatingMethodAccessorImpl invoke DelegatingMethodAccessorImpl java 43 ~ at java lang reflect Method invoke Method java 568 ~ at org glassfish jersey server model internal ResourceMethodInvocationHandlerFactory lambda$static$0 ResourceMethodInvocationHandlerFactory java 52 ~ jersey server 2 31 jar at org glassfish jersey server model internal AbstractJavaResourceMethodDispatcher$1 run AbstractJavaResourceMethodDispatcher java 124 ~ jersey server 2 31 jar at org glassfish jersey server model internal AbstractJavaResourceMethodDispatcher invoke AbstractJavaResourceMethodDispatcher java 167 ~ jersey server 2 31 jar at org glassfish jersey server model internal JavaResourceMethodDispatcherProvider$TypeOutInvoker doDispatch JavaResourceMethodDispatcherProvider java 219 ~ jersey server 2```

Phoebe Yang · Answer

server error log and the docker image was built locally with command `docker build t airbyte source example dev`

Daniel Eduardo Portugal Revilla · Answer

Hi < Phoebe Yang> what s the output of `docker run airbyte source example dev spec`

Daniel Eduardo Portugal Revilla · Answer

``` type SPEC spec documentationUrl connectionSpecification $schema title Test Source Spec type object required client id client secret access token additionalProperties false properties client id type string title Client Id description Client Id for Basic Auth Header when sending requests airbyte secret true order 1 client secret type string title Client Secret description Client Secret for Basic Auth Header when sending requests airbyte secret true order 2 access token type string title Access Token description The access token associated with the data being requested for airbyte secret true order 3 start date type string title Start Date pattern ^ 0 9 4 0 9 2 0 9 2 T 0 9 2 0 9 2 0 9 2 0 9 3 Z$ description UTC date and time in the format 2017 01 25T00 00 00 000Z Any data before this date will not be replicated examples 2017 01 25T00 00 00Z order 4 ```

Daniel Eduardo Portugal Revilla · Answer

I also saw this in the worker error log

Daniel Eduardo Portugal Revilla · Answer

```airbyte worker | Caused by io temporal failure ApplicationFailure message= Cannot run program tmp scripts17475122944268089990 image exists sh error=0 Failed to exec spawn helper pid 130 exit value 1 type= java io IOException nonRetryable=false airbyte worker | at java base java lang ProcessBuilder start ProcessBuilder java 1143 ~ airbyte worker | at java base java lang ProcessBuilder start ProcessBuilder java 1073 ~ airbyte worker | at io airbyte workers process DockerProcessFactory checkImageExists DockerProcessFactory java 173 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers process DockerProcessFactory create DockerProcessFactory java 92 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers process AirbyteIntegrationLauncher spec AirbyteIntegrationLauncher java 44 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers DefaultGetSpecWorker run DefaultGetSpecWorker java 48 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers DefaultGetSpecWorker run DefaultGetSpecWorker java 23 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers temporal TemporalAttemptExecution lambda$getWorkerThread$2 TemporalAttemptExecution java 174 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at java base java lang Thread run Thread java 833 ~ airbyte worker | Caused by io temporal failure ApplicationFailure message= error=0 Failed to exec spawn helper pid 130 exit value 1 type= java io IOException nonRetryable=false airbyte worker | at java base java lang ProcessImpl forkAndExec Native Method 0 ~ airbyte worker | at java base java lang ProcessImpl lt init gt ProcessImpl java 314 ~ airbyte worker | at java base java lang ProcessImpl start ProcessImpl java 244 ~ airbyte worker | at java base java lang ProcessBuilder start ProcessBuilder java 1110 ~ airbyte worker | at java base java lang ProcessBuilder start ProcessBuilder java 1073 ~ airbyte worker | at io airbyte workers process DockerProcessFactory checkImageExists DockerProcessFactory java 173 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers process DockerProcessFactory create DockerProcessFactory java 92 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers process AirbyteIntegrationLauncher spec AirbyteIntegrationLauncher java 44 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers DefaultGetSpecWorker run DefaultGetSpecWorker java 48 ~ io airbyte airbyte workers 0 35 11 alpha jar airbyte worker | at io airbyte workers DefaultGetSpecWorker run DefaultGetSpecWorker java 23 ~ io airbyte airbyte workers 0 35 11 alpha jar ```

Daniel Eduardo Portugal Revilla · Answer

seems like an error with worker finding the docker image

Daniel Eduardo Portugal Revilla · Answer

I guess you are on Apple M1 computer smile

Daniel Eduardo Portugal Revilla · Answer

If you are indeed working on M1 we usually solve this by adding the following environment variable to the airbyte worker in the docker compose file `JAVA OPTS Djdk lang Process launchMechanism=vfork `

Daniel Eduardo Portugal Revilla · Answer

yes its M1

Daniel Eduardo Portugal Revilla · Answer

ahh i see i ll give it a try

Daniel Eduardo Portugal Revilla · Answer

this is the docker compose under the root directory

Daniel Eduardo Portugal Revilla · Answer

You ll find updates about our current M1 support here <https github com airbytehq airbyte issues 2017>

Daniel Eduardo Portugal Revilla · Answer

thanks I got it working now slightly smiling face

Daniel Eduardo Portugal Revilla · Answer

a follow up question if I deploy airbyte on ec2 can I run my test connector just by pulling the docker image from a public docker repository