Knowing that bigger web apps, usually aren't build like this... it feels kinda weird doing it this way

Not sure thats correct. Most sites I've worked on have used the same tech stack across the whole site and react works great with content management systems. Just hook up whichever content API you are using like you would any other and go to town. The downside is very very slim in my experience, if at all. If SSR is not needed I recommend hosting it on S3.