Well to be clear, it's not "just because he says so." It's because Elon personally had an experience (the death of his child) that is salient enough for him to supersede free speech. This is very different from the personal experiences of other people regarding other contested topics/personalities primarily in that those are the experiences of people not named Elon.
In those cases, free speech wins open and close, very simple, and if you are upset about it it's because you're part of the cabal that's trying to take down Elon/Twitter/free speech.
It's not valid, that's how. He's using "free speech" as a way to hype up all the right winger fanboys and to unban people who will cause incredible drama (i.e. engagement) on Twitter. Hopefully all it causes is online drama, but the barrier between real world actions and online information sphere is way more permeable than the "it's just a website!" people would have you believe.