I don't know what you mean; uBlock Origin Lite blocks every ad on any page I visit in Safari and Chrome, even YouTube ads. Safari also blocks tracker cookies by default, and is significantly faster than Firefox in my use.

The average user does not care about ads being present. Speed is paramount.

Shouldn't getting rid of all the ads make things notably faster?

It does. Fundamentally.

I disagree. I have installed adblockers on different relatives machines. Without fail, they mentioned how much better surfing now is.

They did not not mind the ads, they just didn‘t know there is an easy way to get rid of them

You are radically underestimating how much slow web performance is caused by the massive volume of ads and user tracking data attached to each site.