Residency is purely about taxes, health insurance and right to work. You e.g. don't get voting rights in national elections.

EU citizens automatically get permanent residency in any EU country regardless of language.

E.g. I'm a permanent resident in Germany as a Dutchie.

(However I did end up picking up German and speak it now. But never had to do a language test)

Yea free movement and choice of residency is one of the main points of the EU