Running into a situation where Google search is unable to index the site running as pages. Robots.txt renders as index.html for some reason. I did create explicit robots.txt in my repo and I'm pretty sure it's propagating to pages - anyone know what might be happening?

Do you have a 404.html? If not, any page that 404s will render the index.html, for SPA behaviour. That won't prevent Pages from server a file though. Where is your robots.txt? Can you provide a link to it and/or your repo for us to see?

Thanks

Your robots.txt should be in your output folder. I imagine that’s public or dist?

From what it looks like you're running Vite (or a framework that uses Vite). The way Vite works is that it copies everything from public -> dist at build time so your robots.txt needs to go in public/ as James says. Side note - I'd recommend against committing dist (add it to your .gitignore), those generated files change often and will slow down git in the long run.

Thanks, you’re right - robots doesn’t seem to make it to public or dist. If I remove dist from the github, will it get generated by the cf pages?

Yes, it should.

Got it, I'll try- thanks!