I generally agree, but some people really are allergic to some vaccinations. They present the same exposure risk as somebody who chooses not to be vaccinated for stupid reasons. Should they also not be allowed in?
That would be a viable medical reason, and could probably be exempted with a doctor's note or some other documentation of a medical condition (at least it should be). That's not comparable to deliberately denying otherwise-healthy children the means to stay healthy and not die of horrible and easily preventable diseases.
As a population, the people who legitimately cannot be vaccinated -- or who get vaccinated but a certain vaccination just doesn't take -- are small enough that the rest of us protect them. When people start voluntarily adding themselves to that group without medical reason things start to break.