https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
c

Chris P

11/16/2021, 11:14 PM
Is there any way to set up an airbyte instance on Google Compute Engine entirely using terraform? I’m new to terraform and I’ve been using airbyte for a bit but I wasn’t sure if there’s a way to automate some of the steps in this setup guide: https://docs.airbyte.io/deploying-airbyte/on-gcp-compute-engine
1
I’m able to set up a compute engine VM instance in Terraform pretty easily but I still have to SSH into it to install docker, docker-compose, and then airbyte
f

fisher

11/24/2021, 12:02 AM
Hi! We are doing this in GCP GCE using container-optimized OS and cloud-init. Docker comes installed already on cos, so it's just a matter of using cloud-init to grab the Docker-compose and .env file and then running Docker-compose up
We technically create a system init file that handles starting the service, stopping it, etc.
c

Chris P

11/29/2021, 6:54 PM
Oh that’s cool. Thanks for getting back to me!
Hey @fisher just curious. When you ssh into your GCE VM after your init script runs, is there an airbyte folder somewhere on the virtual machine? So far I haven’t been able to prove that airbyte actually gets installed on my VM, and I haven’t been able to connect to the airbyte frontend over SSH
u

[DEPRECATED] Marcos Marx

12/07/2021, 12:45 AM
@Chris P did you check this? https://github.com/airbytehq/airbyte/tree/master/terraform have one example for AWS but should be eaasy to change to GCP
f

fisher

12/07/2021, 2:08 AM
yeah, this is essentially what we do: https://github.com/airbytehq/airbyte/blob/master/terraform/aws/demo/core/init.sh#L21 We’re using container-optimized OS so not all of the disk is writable, so we just run this script from the context of the home folder for our automation user and that does the trick. Since we’re using cloud-init, it looks like this:
Copy code
- [wget, "<https://raw.githubusercontent.com/airbytehq/airbyte/${airbyte_release_version}/.env>", -O, /home/airbyteuser/.env]
- [wget, "<https://raw.githubusercontent.com/airbytehq/airbyte/${airbyte_release_version}/docker-compose.yaml>", -O, /home/airbyteuser/docker-compose.yaml]
- [bash, /home/airbyteuser/setup_airbyte_env.sh]
15 Views