I assume it's enough if i update the devDependency and also update the base docker image in my docker-compose to the same version?

Upgrading the CLI & then upgrading the docker image in the compose file is the way to go. I would recommend initializing a new project with the new CLI and diffing the generated docker compose files for major differences, e.g. in Prisma configuration. Just to be sure.