I made the swap from a C63 507 that was moderately tuned. Obviously had quite a bit more power and poke than the M2 but have been craving a manual gearbox with RWD for years.
Although the M2's engine is a bit "weak" in comparison it is just so much more fun to drive and as you point out ALL the power can be used more of the time. In fact the M2 is least powerful car I have owned in years but just provides more smiles per miles
A test drive would be the obvious thing to do but I doubt you would be disappointed.