But if they have my number, why would I be worried that they know my username? The username is there so I can avoid sharing my number, not the other way around.
Ah, that's what you mean. Yeah, if you want to be anonymous to Signal itself, I don't think that's possible. If you want to be anonymous to people, I think you can delete and recreate your account. I think that might do the trick.