What is the right CPU, RAM configuration to use vi...
# replication-ask-ai
c
What is the right CPU, RAM configuration to use virtual machine to run AirByte on GCP? I have about 5 apps with the frequency of operation is once every hour?
k
A message from kapa.ai
To run Airbyte on a Google Cloud Platform (GCP) Compute Engine instance, you need to follow several steps to set up the environment, install and start Airbyte, and connect it to the GCP instance. Here are the steps: 1. Set up the environment - Create a new GCP instance. You can follow the instructions [here](https://cloud.google.com/compute/docs/instances/create-start-instance). - Set variables in your local terminal for your project ID and instance name. - Install Google Cloud SDK and initialize the gcloud command-line tool. - Verify that you can see your instance. - Connect to your instance. - Install Docker on your VM instance. - Install
docker-compose
on your VM instance.
Copy code
bash # Set variables PROJECT_ID=PROJECT_ID_WHERE_YOU_CREATED_YOUR_INSTANCE   INSTANCE_NAME=airbyte # or any other name that you've used    # Install Google Cloud SDK and initialize gcloud brew install --cask google-cloud-sdk   gcloud init    # Verify you can see your instance   gcloud --project $PROJECT_ID compute instances list    # Connect to your instance gcloud --project=$PROJECT_ID beta compute ssh $INSTANCE_NAME    # Install Docker sudo apt-get update   sudo apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common   curl -fsSL <https://download.docker.com/linux/debian/gpg> | sudo apt-key add --   sudo add-apt-repository "deb [arch=amd64] <https://download.docker.com/linux/debian> buster stable"   sudo apt-get update   sudo apt-get install -y docker-ce docker-ce-cli <http://containerd.io|containerd.io>   sudo usermod -a -G docker $USER    # Install docker-compose sudo apt-get -y install docker-compose-plugin   docker compose version
2. Install and launch Airbyte - In your local terminal, connect to your Google Cloud instance. - In your VM terminal, install Airbyte.
Copy code
bash # Connect to your Google Cloud instance gcloud --project=$PROJECT_ID beta compute ssh $INSTANCE_NAME    # Install Airbyte mkdir airbyte && cd airbyte   wget <https://raw.githubusercontent.com/airbytehq/airbyte/master/run-ab-platform.sh>   chmod +x run-ab-platform.sh   ./run-ab-platform.sh -b
3. Connect to Airbyte - In your local terminal, create an SSH tunnel to connect the GCP instance to Airbyte. - Verify the connection by visiting http://localhost:8000 in your browser.
Copy code
bash # Create an SSH tunnel gcloud --project=$PROJECT_ID beta compute ssh $INSTANCE_NAME -- -L 8000:localhost:8000 -N -f
For the VM instance type, if you're just testing Airbyte, an
e2.medium
instance with at least 30GBs of disk per node is recommended. For a production environment, a
n1-standard-2
instance is recommended. Please note that for security reasons, it's strongly recommended not to expose Airbyte publicly. Sources: - [Deploy Airbyte on GCP](https://docs.airbyte.com/deploying-airbyte/on-gcp-compute-engine) - [Airbyte Security](https://docs.airbyte.com/operator-guides/security)
n
@AJ Steers (Airbyte) can you help me anout this