The "Quick start with yarn dx" is only meant to be like a temporary trail? If I want to actually deploy the non docker version of Cal.com, I will need to use something like Heroku?

you need a place to host your postgres db, heroku, aws, railway, ..