I have a Mac Mini with M1 at home. At the beginning I faced some difficulties with some packages, since not all work with the architecture. Fortunately, a time has passed and there are fewer issues now (but they still exist!).
In my case, I'd still opt for M1/M2 too.