There is another catch. Sun's surface is about 6000K. A light with such color temperature would be considered bluish by most people if they looked directly at a lamp with such a spectrum. Rayleigh scattering lowers it to 5600-5800K at noon time, but still most humans would call a moderately bright lamp of this color "bluish".

The eye has a built-in white balance correction, and it moves the white point towards longer waves as the brightness lowers. This is why a single 5600K lamp in a room makes bluish light, while 10 of the same lamps make it lit by bright white. Same, a single 2500K lamp in a room makes a calming sort-of-white light. but 10 such lamps make a unmistakably yellow light.

The sunlight outside at a cloudless summer noon is very bright (one usually even wants to wear sunglasses), so the sun looks white or yellowish, especially in contrast to the blue sky.