I have an Ubuntu 22.04.2 LTS Jammy ARM64. I have ...
# replication-ask-ai
p
I have an Ubuntu 22.04.2 LTS Jammy ARM64. I have docker, docker compose, and docker-compose higher than v2. But not Docker Desktop. I have a problem running Airbyte when executing the shell script "sudo ./run-ab-platform.sh". It returns back: docker compose v2 not found! please install docker compose!" This is weird because "docker compose" is greater than v2.
k
Based on the information provided, it seems like you have Docker and Docker Compose installed, but the script is not recognizing Docker Compose V2. This could be due to a path issue or a version mismatch. Airbyte version 0.40.27 or later requires Docker Compose V2 to be installed before upgrading as mentioned in the [Airbyte documentation](https://docs.airbyte.com/operator-guides/upgrading-airbyte). Please ensure that you have the correct version of Docker Compose installed. You can check your Docker Compose version by running the command
docker-compose --version
in your terminal. If you have the correct version installed and you're still facing the issue, it could be due to the snap version of Docker. Some users have reported issues with the snap version of Docker on Ubuntu 22.04, as discussed in the [Airbyte forum](https://discuss.airbyte.io/t/3583). The solution was to completely remove the snap version and install the apt version of Docker. If the issue persists, I would recommend reaching out to the Airbyte community on their [forum](https://discuss.airbyte.io/) or [Slack](https://airbytehq.slack.com) for further assistance.
p
I didn't install it using snap
I ran: docker-compose --version and it returned: Docker Compose version v2.20.2
c
you need to be able to run docker space compose
not docker dash compose
in other words, this command must succeed
docker compose
, not this command
docker-compose
p
Thank you for the respons. Yes, I can:
When I run the shell script, I get this:
c
this is the check the shell script runs, do you also get the same result when you run
Copy code
docker compose version
p
No... that is successful
c
can you add these two lins above where it says
########## Dependency Check ##########
in the script, and see what the result is
Copy code
version=$(docker compose version 2>&1)
echo "Docker Compose version: $version"
(i’m guessing is somehow v1 is getting invoked in your shell env somehow?)
p
Thanks... I'll try it right now. Hold on please.
I saved it and it looks like this now:
c
nice, what does it print before you see the failure?
it looks like this on my machine:
Copy code
█████╗ ██╗██████╗ ██████╗ ██╗   ██╗████████╗███████╗
██╔══██╗██║██╔══██╗██╔══██╗╚██╗ ██╔╝╚══██╔══╝██╔════╝
███████║██║██████╔╝██████╔╝ ╚████╔╝    ██║   █████╗  
██╔══██║██║██╔══██╗██╔══██╗  ╚██╔╝     ██║   ██╔══╝  
██║  ██║██║██║  ██║██████╔╝   ██║      ██║   ███████╗
╚═╝  ╚═╝╚═╝╚═╝  ╚═╝╚═════╝    ╚═╝      ╚═╝   ╚══════╝
                                            Move Data

Docker Compose version: Docker Compose version v2.19.1
p
But when I execute the script, I get this:
🤔 1
It just dies
c
can you cat the full contents of run-ab-platform.sh and paste it here
e.g
cat run-ab-platform-sh
p
sure
c
(hopefully it’s small enough to fit in slack chat, if not paste it as a file)
p
#!/bin/bash VERSION=0.50.11 # Run away from anything even a little scary set -o nounset # -u exit if a variable is not set set -o errexit # -f exit for any command failure" # text color escape codes (please note \033 == \e but OSX doesn't respect the \e) blue_text='\033[94m' red_text='\033[31m' default_text='\033[39m' # set -x/xtrace uses a Sony PS4 for more info PS4="$blue_text""${BASH_SOURCE}:${LINENO}: ""$default_text" ############################################################ # Help # ############################################################ Help() { # Display Help echo -e "This Script will download the necessary files for running docker compose" echo -e "It will also run docker compose up" echo -e "Take Warning! These assets may become stale over time!" echo # $0 is the currently running program echo -e "Syntax: $0" echo -e "options:" echo -e " -d --download Only download files - don't run docker compose" echo -e " -r --refresh ${red_text}DELETE${default_text} existing assets and re-download new ones" echo -e " -h --help Print this Help." echo -e " -x --debug Verbose mode." echo -e " -b --background Run docker compose up in detached mode." echo -e "" } ########## Declare assets care about ########## docker_compose_yaml="docker-compose.yaml" docker_compose_debug_yaml="docker-compose.debug.yaml" dot_env=".env" dot_env_dev=".env.dev" flags="flags.yml" temporal_yaml="temporal/dynamicconfig/development.yaml" # any string is an array to POSIX shell. Space seperates values all_files="$docker_compose_yaml $docker_compose_debug_yaml $dot_env $dot_env_dev $flags $temporal_yaml" base_github_url="https://raw.githubusercontent.com/airbytehq/airbyte-platform/v$VERSION/" ############################################################ # Download # ############################################################ Download() { ########## Check if we already have the assets we are looking for ########## for file in $all_files; do # Account for the case where the file is in a subdirectory. # Make sure the directory exists to keep curl happy. dir_path=$(dirname "${file}") mkdir -p "${dir_path}" if test -f $file; then # Check if the assets are old. A possibly sharp corner if test $(find $file -type f -mtime +60 > /dev/null); then echo -e "$red_text""Warning your $file may be stale!""$default_text" echo -e "$red_text""rm $file to refresh!""$default_text" else echo -e "$blue_text""found $file locally!""$default_text" fi else echo -e "$blue_text""Downloading $file""$default_text" curl --location\ --fail\ --silent\ --show-error \ ${base_github_url}${file} > $file fi done } DeleteLocalAssets() { for file in $all_files; do echo -e "$blue_text""Attempting to delete $file!""$default_text" if test -f $file; then rm $file && echo -e "It's gone!" else echo -e "$file not found locally. Nothing to delete." fi done } dockerDetachedMode="" # $0 is the currently running program (this file) this_file_directory=$(dirname $0) # Run this from the / directory because we assume relative paths cd ${this_file_directory} for argument in $@; do case $argument in -d | --download) Download exit ;; -r | --refresh) DeleteLocalAssets exit ;; -h | --help) Help exit ;; -x | --debug) set -o xtrace # -x display every line before execution; enables PS4 ;; -b | --background) dockerDetachedMode="-d" ;; *) echo "$argument is not a known command." echo Help exit ;; esac shift done ########## Pointless Banner for street cred ########## # Make sure the console is huuuge if test $(tput cols) -ge 64; then # Make it green! echo -e "\033[32m" echo -e " █████╗ ██╗██████╗ ██████╗ ██╗ ██╗████████╗███████╗" echo -e "██╔══██╗██║██╔══██╗██╔══██╗╚██╗ ██╔╝╚══██╔══╝██╔════╝" echo -e "███████║██║██████╔╝██████╔╝ ╚████╔╝ ██║ █████╗ " echo -e "██╔══██║██║██╔══██╗██╔══██╗ ╚██╔╝ ██║ ██╔══╝ " echo -e "██║ ██║██║██║ ██║██████╔╝ ██║ ██║ ███████╗" echo -e "╚═╝ ╚═╝╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝ ╚══════╝" echo -e " Move Data" # Make it less green echo -e "\033[0m" sleep 1 fi version=$(docker compose version 2>&1) echo "Docker Compose version: $version" ########## Dependency Check ########## if ! docker compose version >/dev/null 2>/dev/null; then echo -e "$red_text""docker compose v2 not found! please install docker compose!""$default_text" exit 1 fi Download ########## Source Envionmental Variables ########## for file in $dot_env $dot_env_dev; do echo -e "$blue_text""Loading Shell Variables from $file...""$default_text" source $file done ########## Start Docker ########## echo echo -e "$blue_text""Starting Docker Compose""$default_text" docker compose up $dockerDetachedMode # $? is the exit code of the last command. So here: docker compose up if test $? -ne 0; then echo -e "$red_text""Docker compose failed. If you are seeing container conflicts""$default_text" echo -e "$red_text""please consider removing old containers""$default_text" fi ########## Ending Docker ########## if [ -z "$dockerDetachedMode" ]; then docker compose down else echo -e "$blue_text""Airbyte containers are running!""$default_text" fi
c
hmm, it works on my machine, but mine is a macos. Let’s try isolating just that piece of code. Save that blurb as test.sh and run it and see what you get
Copy code
version=$(docker compose version 2>&1)
echo "Docker Compose version: $version"
########## Dependency Check ##########
if ! docker compose version >/dev/null 2>/dev/null; then
 echo -e "$red_text""docker compose v2 not found! please install docker compose!""$default_text"
 exit 1
 fi
you probably will need to change execute permissions like this
Copy code
touch test.sh
copy contents in
chmod +x test.sh
./test.sh
p
I did that... now it's doing a bunch of thins 🙂
standby
downloading all kinds of stuff
c
it sounds like it’s working…. did you simply remove the version check (the piece of code I mentioned above?)
p
no... I just made it executable
Copy code
chmod +x run-ab-platform.sh
actually:
Copy code
sudo chmod +x run-ab-platform.sh
c
interesting. That file should have been checked in as executable already so you shouldn’t have needed to do that
I wonder if it’s some idiosyncracy of how linux executes shell of shell
p
and ran it as:
Copy code
./run-ab-platform.sh
instead of:
Copy code
sudo ./run-ab-platform.sh
c
i would be curious what the permissions looked like before and after you ran that command. This is what they look like on my machine
Copy code
-rwxr-xr-x   1 conor  staff   6.1K Jul 28 09:25 run-ab-platform.sh
the reason I ask is if those defaults aren’t good enough for linux we probably need to change permissions on the file that gets checked in
p
Last message I got is: "Started","logging-call-at":"taskQueueManager.go:292"}
I guess it will not stop until I CTRL-T it
hold on let me do ln -l on it
c
I think that is a normal message when running the platform, it sounds like you are off to the races. Yes it will run in the foreground unless it’s run as a daemon
if you run it with run-ab-platform.sh -b it will run in background e.g. detached mode, and you’ll have to
docker compose down
if you want to kill it if you do that
p
It is now: -rwxr-xr-x 1 root root 6214 Jul 28 10:32 run-ab-platform.sh
c
looks right. Perhaps a docs update to not run it as the root user is in order
p
did you mean.. run-ab-platform.sh -d?
c
Copy code
# Display Help
   echo -e "This Script will download the necessary files for running docker compose"
   echo -e "It will also run docker compose up"
   echo -e "Take Warning! These assets may become stale over time!"
   echo
   # $0 is the currently running program
   echo -e "Syntax: $0"
   echo -e "options:"
   echo -e "   -d --download    Only download files - don't run docker compose"
   echo -e "   -r --refresh     ${red_text}DELETE${default_text} existing assets and re-download new ones"
   echo -e "   -h --help        Print this Help."
   echo -e "   -x --debug       Verbose mode."
   echo -e "   -b --background  Run docker compose up in detached mode."
   echo -e ""
we use -b, because -d was already taken when we put -b in 😄
and rather than break existing users we chose -b
p
cool
so do I simply go to url: localhost:8080 to launch it?
c
i was going to say, as per the readme, but then I realized you are probably looking at airbytehq/airbyte readme. It says in the airbyte-platform readme.md https://github.com/airbytehq/airbyte-platform
oh it also does say in the airbyte readme, link to the docs here: https://docs.airbyte.com/quickstart/deploy-airbyte/
p
Well I tried localhost:8080 and it is giving me an "unaable to connect" 😢
c
as per the docs, it’s
<http://localhost:8000>
p
oh thanks!
what is username and password?
nermind... i got it
I'm in... thanks for your help!
Support is great
c
no problem!
p
I've been slated to mostly see if support is there... so it's definiately something we will look at
currently we are using FiveTran and we don't.mind giving you our business if this all works out
c
Happy to be of assistance! let us know if you have other further questions 🙂
p
You been fantastic Conor!
airbyte heart 1
Our company loves great support. It's critical in a business. We don't care how great your product is, if support is not excellent we don't consider it. We got rid of a few vendors because of this
c
Happy to be part of an org that takes support seriously! We love helping out 🙂
❤️ 1