https://linen.dev logo
s

Samuel Gordalina

03/06/2021, 9:49 PM
I'm trying to run migrations but can't seem to pass arguments to the main class - I know am doing something wrong. I'm running:
Copy code
./gradlew :airbyte-migration:run --target-version=0.17.0
u

user

03/07/2021, 8:19 AM
@Samuel Gordalina the migration is typically run from docker as demonstrated in this tutorial https://docs.airbyte.io/tutorials/upgrading-airbyte
u

user

03/07/2021, 8:19 AM
does this help with the issue?
u

user

03/08/2021, 5:36 PM
I'm writing my own migration, let me try building the docker image. This is super helpful, thank you
u

user

03/08/2021, 5:57 PM
When i run build, i see a bunch of stuff in the
build
folder, but not a docker image. I'm not sure how to build it. (I'm a java noob)
u

user

03/09/2021, 2:26 AM
Samuel did you figure this out?
u

user

03/09/2021, 2:51 AM
I wasn't.
u

user

03/09/2021, 2:52 AM
I ended up creating a test for the migration, but couldn't really test it with local data
u

user

03/09/2021, 3:03 AM
I see. You can pass args into as such
./gradlew :airbyte-migration:run --args="--target-version=0.17.0 --output=/tmp --input=/tmp"
(my paths aren’t right)
u

user

03/09/2021, 3:04 AM
although I’m not sure I understand what you are trying to do - like Sherif pointed out, we usually run migrations in Docker. do you mind restating what you are trying to do for me?
u

user

03/09/2021, 3:06 AM
Absolutely. I'm adding workspace creation/deletion (code is here: https://github.com/airbytehq/airbyte/pull/2325) and that required me to add
tombstone
to the workspace definition. That requires me to write a migration (which i have) to migrate the default airbyte workspace to have tombtstone set to false.
u

user

03/09/2021, 3:14 AM
I think my main issue is that I could find a way to build the migrations docker image to test
u

user

03/09/2021, 3:16 AM
I was able to test with your command and it worked 🙂 thank you
u

user

03/09/2021, 3:21 AM
Thanks for the concise summary! Glad I was able to help!
u

user

03/09/2021, 3:26 AM
if you are wondering about Docker, I believe something like
./gradlew :composeBuild
will build all the relevant Docker images.
Copy code
$ ./gradlew :composeBuild
> Task :composeBuild
The GIT_REVISION variable is not set. Defaulting to a blank string.
Building init      ... done
Building db        ... done
Building seed      ... done
Building scheduler ... done
Building server    ... done
Building webapp    ... done
Building migration ... done
we can check this via:
Copy code
$ docker images
REPOSITORY                               TAG            IMAGE ID       CREATED             SIZE
airbyte/scheduler                        dev            ff66289384e8   3 seconds ago       1.01GB
airbyte/server                           dev            a17366c4fd42   3 seconds ago       596MB
airbyte/migration                        dev            abcfe3536b90   26 seconds ago      436MB
(the migration image was recently generated)
u

user

03/09/2021, 4:21 AM
Oh, I see! I'll test now
u

user

03/09/2021, 4:22 AM
I think my lack of knowledge of the java ecosystem didn't help
u

user

03/09/2021, 4:27 AM
👍 can confirm it worked, thank you all for helping
4 Views