Is there any way to set up an airbyte instance on ...
# ask-community-for-troubleshootingc
Chris P
11/16/2021, 11:14 PMIs there any way to set up an airbyte instance on Google Compute Engine entirely using terraform? I’m new to terraform and I’ve been using airbyte for a bit but I wasn’t sure if there’s a way to automate some of the steps in this setup guide: https://docs.airbyte.io/deploying-airbyte/on-gcp-compute-engine
✅ 1
Chris P
11/16/2021, 11:15 PMI’m able to set up a compute engine VM instance in Terraform pretty easily but I still have to SSH into it to install docker, docker-compose, and then airbyte
f
fisher
11/24/2021, 12:02 AMHi! We are doing this in GCP GCE using container-optimized OS and cloud-init. Docker comes installed already on cos, so it's just a matter of using cloud-init to grab the Docker-compose and .env file and then running Docker-compose up
fisher
11/24/2021, 12:03 AMWe technically create a system init file that handles starting the service, stopping it, etc.
c
Chris P
11/29/2021, 6:54 PMOh that’s cool. Thanks for getting back to me!
Chris P
12/06/2021, 11:17 PMHey @fisher just curious. When you ssh into your GCE VM after your init script runs, is there an airbyte folder somewhere on the virtual machine? So far I haven’t been able to prove that airbyte actually gets installed on my VM, and I haven’t been able to connect to the airbyte frontend over SSH
u
[DEPRECATED] Marcos Marx
12/07/2021, 12:45 AM
@Chris P did you check this? https://github.com/airbytehq/airbyte/tree/master/terraform have one example for AWS but should be eaasy to change to GCP
f
fisher
12/07/2021, 2:08 AMyeah, this is essentially what we do: https://github.com/airbytehq/airbyte/blob/master/terraform/aws/demo/core/init.sh#L21
We’re using container-optimized OS so not all of the disk is writable, so we just run this script from the context of the home folder for our automation user and that does the trick. Since we’re using cloud-init, it looks like this:
Copy code
- [wget, "<https://raw.githubusercontent.com/airbytehq/airbyte/${airbyte_release_version}/.env>", -O, /home/airbyteuser/.env]
- [wget, "<https://raw.githubusercontent.com/airbytehq/airbyte/${airbyte_release_version}/docker-compose.yaml>", -O, /home/airbyteuser/docker-compose.yaml]
- [bash, /home/airbyteuser/setup_airbyte_env.sh]15 Views