Hi! We’re planning on deploying Airbyte on-premise...
# replication-ask-ai
m
Hi! We’re planning on deploying Airbyte on-premise on a machine that does not have an internet connection. Hence, we’ll need to pre-package all docker images that we need to use beforehand. Is there any documentation of which docker images to pull, and how we best navigate this? 😇
s
Hey Magnus, Is it not possible to build and pull the docker images and then retag or push those images to whatever repository you plan on using to host packages internally?
c
if it’s acceptable to do this with docker compose, you can also get the canoncial list/behavior by running run-ab-platform.sh (file in airbytehq/airbyte) to a non-airgapped machine and seeing the images that get pulled. What airbyte needs to run locally is a constellation of 5-6 files, plus the images themselves, plus environment configuration (which is the .env file run-airbyte-platform.sh pulls) A straightforward way to achieve this, that isn’t super friendly, is to take the images you pulled on the non airgapped machine, tar them up using
docker save -o <path-to-save-image.tar> <image-name>:<tag>
, tarring up the 5-6 files that the shellscript pulls as well, and then on the airgapped machine use
docker load -i <path-to-image.tar>
This wouldn’t require an internal repository and could theoretically be wrapped up into a single scriptable command that could be loaded onto the airgapped machine alongside the tarred up images and files. The ux of that isn’t super great though, and if you do have an internal repository I would recommend mirroring those images to it instead and just using a modified version of run-ab-platform.sh for UX
m
Thank you both! 🙏 This should be more than sufficient 😍
@Conor Barber (Airbyte) I am experiencing an error on an airgapped machine that whenever I click a connection, I get “unknown error occurred”. All logs I can find from the webapp and the server show 200 status codes, so I’m wondering what the issue could be?
c
does the client (e.g. the browser via developer tools) show an error code on the network tab?
d
Hello everyone, is it best practice to deploy Airbyte using docker on an on-premise server machine? We don’t want to use any cloud services.
m
@David Effiong We’re using Docker-Compose (using the
run-ab-platform.sh
script as a scaffold) and it works reasonably well. Be aware that if the machine is airgapped, you will need to use Airbyte’s API to create the connections. You can edit them using the webapp but creating them won’t work since the webapp relies on some external API calls that seem to be hardcoded.
d
Thank you