It's a folder not a file. I'm not super familiar with windows, but

rm -rf .next

should probably work