Hi All I have recently deployed the airbyte solution on my l

Question

Hi All I have recently deployed the airbyte solution on my local machine Mac OS M1 while trying to connect to a Postgres Source or Destination I m getting following error ``` 2022 01 19 18 03 25 INFO i a w t TemporalAttemptExecution get 118 Docker volume job log path tmp workspace ee299288 f423 45c1 a9d4 b3d678894763 0 logs log 2022 01 19 18 03 25 INFO i a w t TemporalAttemptExecution get 123 Executing worker wrapper Airbyte version version not set 2022 01 19 18 03 27 INFO i a w t TemporalAttemptExecution lambda$getWorkerThread$2 177 Completing future exceptionally io airbyte workers WorkerException Error while getting checking connection at io airbyte workers DefaultCheckConnectionWorker run DefaultCheckConnectionWorker java 84 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers DefaultCheckConnectionWorker run DefaultCheckConnectionWorker java 27 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers temporal TemporalAttemptExecution lambda$getWorkerThread$2 TemporalAttemptExecution java 174 ~ io airbyte airbyte workers 0 35 5 alpha jar at java lang Thread run Thread java 833 Caused by java lang RuntimeException java io IOException Cannot run program tmp scripts11060788679633185033 image exists sh error=0 Failed to exec spawn helper pid 129 exit value 1 at io airbyte workers process DockerProcessFactory checkImageExists DockerProcessFactory java 205 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers process DockerProcessFactory create DockerProcessFactory java 96 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers process AirbyteIntegrationLauncher check AirbyteIntegrationLauncher java 60 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers DefaultCheckConnectionWorker run DefaultCheckConnectionWorker java 53 ~ io airbyte airbyte workers 0 35 5 alpha jar 3 more Caused by java io IOException error=0 Failed to exec spawn helper pid 129 exit value 1 at java lang ProcessImpl forkAndExec Native Method ~ at java lang ProcessImpl lt init gt ProcessImpl java 314 ~ at java lang ProcessImpl start ProcessImpl java 244 ~ at java lang ProcessBuilder start ProcessBuilder java 1110 ~ at java lang ProcessBuilder start ProcessBuilder java 1073 ~ at io airbyte workers process DockerProcessFactory checkImageExists DockerProcessFactory java 193 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers process DockerProcessFactory create DockerProcessFactory java 96 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers process AirbyteIntegrationLauncher check AirbyteIntegrationLauncher java 60 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers DefaultCheckConnectionWorker run DefaultCheckConnectionWorker java 53 ~ io airbyte airbyte workers 0 35 5 alpha jar 3 more 2022 01 19 18 03 27 INFO i a w t TemporalAttemptExecution get 144 Stopping cancellation check scheduling 2022 01 19 18 03 27 WARN i t i s POJOActivityTaskHandler activityFailureToResult 363 Activity failure ActivityId=12a45b30 5c21 336e 8a78 dd18f28ee557 activityType=Run attempt=1 java util concurrent ExecutionException io airbyte workers WorkerException Error while getting checking connection at java util concurrent CompletableFuture reportGet CompletableFuture java 396 ~ at java util concurrent CompletableFuture get CompletableFuture java 2073 ~ at io airbyte workers temporal TemporalAttemptExecution get TemporalAttemptExecution java 142 ~ io airbyte airbyte workers 0 35 5 alpha jar at io airbyte workers temporal check connection CheckConnectionActivityImpl run CheckConnectionActivityImpl java 81 ~ io airbyte airbyte workers 0 35 5 alpha jar at jdk internal reflect NativeMethodAccessorImpl invoke0 Native Method ~ at jdk internal reflect NativeMethodAccessorImpl invoke NativeMethodAccessorImpl java 77 ~ at jdk internal reflect DelegatingMethodAccessorImpl invoke DelegatingMethodAccessorImpl java 43 ~ at java lang reflect Method invoke Method java 568 ~ at io temporal internal sync POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor execute POJOActivityTaskHandler java 286 ~ temporal sdk 1 6 0 jar at io temporal internal sync POJOActivityTaskHandler$POJOActivityImplementation execute POJOActivityTaskHandler java 252 ~ temporal sdk 1 6 0 jar at io temporal internal sync POJOActivityTaskHandler handle POJOActivityTaskHandler java 209 ~ temporal sdk 1 6 0 jar at io temporal internal worker ActivityWorker$TaskHandlerImpl handle ActivityWorker java 193 ~ temporal sdk 1 6 0 jar at io temporal internal worker ActivityWorker$TaskHandlerImpl handle ActivityWorker java 151 ~ temporal sdk 1 6 0 jar at io temporal internal worker PollTaskExecutor lambda$process$0 PollTaskExecutor java 73 ~ temporal sdk 1 6 0 jar at java util concurrent ThreadPoolExecutor runWorker ThreadPoolExecutor java 1136 at java util concurrent ThreadPoolExecutor$Worker run ThreadPoolExecutor java 635 at java lang Thread run Thread java 833 ``` I have checked this issue online so I got some discussion on github <https github com airbytehq airbyte issues 2017> there was a comment ```h7kanna commented on 27 Nov 2021 the above error is because of the airbyte worker not able to create a process on Mac M1 Add this to the worker environment in docker compose and try once JAVA OPTS Djdk lang Process launchMechanism=vfork ``` I have tried this to fix the issue but the issue didn t solve now the server is restarting many times Configuration on my machine are following Exported env variables ```export DOCKER BUILD PLATFORM=linux arm64 export DOCKER BUILD ARCH=arm64 export ALPINE IMAGE=arm64v8 alpine 3 14 export POSTGRES IMAGE=arm64v8 postgres 13 alpine export JDK VERSION=17``` Java version ```kshitijchaurasiya ckshitij ~ SandboxDev new airbyte airbyte$ java version java 17 0 2 2022 01 18 LTS Java TM SE Runtime Environment build 17 0 2+8 LTS 86 Java HotSpot TM 64 Bit Server VM build 17 0 2+8 LTS 86 mixed mode sharing ``` Any help will be appreciated slightly smiling face

Gorkem Yurtseven · Answer

< Augustin Lafanechere Airbyte > could you help here smile

charles · Answer

Hi < kshitij chaurasiya> adding ` JAVA OPTS= Djdk lang Process launchMechanism=vfork ` in the environment variable list of `airbyte worker` in `docker compose yaml` worked for me to fix the same problem you mentioned Did you set the `JAVA OPTS` in ` env` or specifically in `docker compose yaml` as I suggest

Guido Turturici · Answer

Hi < DEPRECATED Augustin Lafanechere> Thank you so much for the replying I have added the JAVA OPTS in docker compose yaml under airbyte worker section only but after that I m getting the server issue as below and not able to access the UI ```airbyte server | 2022 01 23 17 00 05 ERROR i a s ServerApp main 277 Server failed airbyte server | java util NoSuchElementException No value present airbyte server | at java util Optional orElseThrow Optional java 377 ~ airbyte server | at io airbyte server ServerApp getServer ServerApp java 179 ~ io airbyte airbyte server 0 35 7 alpha jar airbyte server | at io airbyte server ServerApp main ServerApp java 275 io airbyte airbyte server 0 35 7 alpha jar ``` ```airbyte scheduler | 2022 01 23 17 10 46 INFO i a s a SchedulerApp waitForServer 210 Waiting for server to become available airbyte scheduler | 2022 01 23 17 10 48 INFO i a s a SchedulerApp waitForServer 210 Waiting for server to become available airbyte webapp | 172 27 0 1 23 Jan 2022 17 10 49 +0000 GET HTTP 1 1 200 4506 Mozilla 5 0 Macintosh Intel Mac OS X 10 15 7 AppleWebKit 537 36 KHTML like Gecko Chrome 97 0 4692 71 Safari 537 36 airbyte webapp | 2022 01 23 17 10 50 error 92 92 8 connect failed 111 Connection refused while connecting to upstream client 172 27 0 1 server localhost request POST api v1 workspaces list HTTP 1 1 upstream host localhost 8000 referrer airbyte webapp | 172 27 0 1 23 Jan 2022 17 10 50 +0000 POST api v1 workspaces list HTTP 1 1 502 494 Mozilla 5 0 Macintosh Intel Mac OS X 10 15 7 AppleWebKit 537 36 KHTML like Gecko Chrome 97 0 4692 71 Safari 537 36 airbyte webapp | 2022 01 23 17 10 50 error 92 92 8 connect failed 111 Connection refused while connecting to upstream client 172 27 0 1 server localhost request POST api v1 workspaces list HTTP 1 1 upstream host localhost 8000 referrer airbyte webapp | 172 27 0 1 23 Jan 2022 17 10 50 +0000 POST api v1 workspaces list HTTP 1 1 502 494 Mozilla 5 0 Macintosh Intel Mac OS X 10 15 7 AppleWebKit 537 36 KHTML like Gecko Chrome 97 0 4692 71 Safari 537 36 airbyte webapp | 172 27 0 1 23 Jan 2022 17 10 50 +0000 GET HTTP 1 1 200 4506 Mozilla 5 0 Macintosh Intel Mac OS X 10 15 7 AppleWebKit 537 36 KHTML like Gecko Chrome 97 0 4692 71 Safari 537 36 airbyte scheduler | 2022 01 23 17 10 51 INFO i a s a SchedulerApp waitForServer 210 Waiting for server to become available airbyte webapp | 2022 01 23 17 10 51 error 92 92 8 connect failed 111 Connection refused while connecting to upstream client 172 27 0 1 server localhost request POST api v1 workspaces list HTTP 1 1 upstream host localhost 8000 referrer airbyte webapp | 172 27 0 1 23 Jan 2022 17 10 51 +0000 POST api v1 workspaces list HTTP 1 1 502 494 Mozilla 5 0 Macintosh Intel Mac OS X 10 15 7 AppleWebKit 537 36 KHTML like Gecko Chrome 97 0 4692 71 Safari 537 36 airbyte webapp | 2022 01 23 17 10 51 error 92 92 8 connect failed 111 Connection refused while connecting to upstream client 172 27 0 1 server localhost request POST api v1 workspaces list HTTP 1 1 upstream host localhost 8000 referrer airbyte webapp | 172 27 0 1 23 Jan 2022 17 10 51 +0000 POST api v1 workspaces list HTTP 1 1 502 494 Mozilla 5 0 Macintosh Intel Mac OS X 10 15 7 AppleWebKit 537 36 KHTML like Gecko Chrome 97 0 4692 71 Safari 537 36 airbyte scheduler | 2022 01 23 17 10 53 INFO i a s a SchedulerApp waitForServer 210 Waiting for server to become available airbyte scheduler | 2022 01 23 17 10 55 INFO i a s a SchedulerApp waitForServer 210 Waiting for server to become available ``` I have tried many times to redeploy as well but getting the same issue Do you have face this kind of issue on your setup

Guido Turturici · Answer

Hi < kshitij chaurasiya> no I don t have this problem I suggest you to try a fresh redeploy again after cleaning volumes `docker compose down` `docker compose rm v` `docker compose up`

Ariyo Kabir · Answer

Hi < DEPRECATED Augustin Lafanechere> Actually I got the issue it was due to docker resource management so by increasing the docker memory it start working slightly smiling face Thank you so much for the help slightly smiling face

Arian Giles Garcia · Answer

Hi This is my first day in the Airbyte open source community I think it s a great tool If possible can someone please name one connector on M1 that s working One source to destination So far I have tried Csv and API as a source they both failed Workaround to run connectors would be appreciated Additionally can we create our connectors using python or JS Any links would be helpful Thank you

Arian Giles Garcia · Answer

Hi < karan jagota> welcome did you tried what I mentionned

Arian Giles Garcia · Answer

Yes The server still may be starting up

Arian Giles Garcia · Answer

Okay Thank you It worked But I have to docker compose thrice It somehow lost connection in between Don t know why But it s a solid workaround Thank you I have successfully transferred data from CSV to Postgres the exchange rate API to Postgres and the JSON file