For any given hue, you can pick a range of colors. There are an infinite number of neutral red/green/blue/yellow colors within sRGB. However, for blue and green especially, these unfortunately happen to be less colorful than you might achieve with paint or see in nature.
(Note, the “hue” measure used in spaces like HSL/HSV is non-uniform, especially in the blue–purple range. So that explains part of the effect you see when you “desaturate” sRGB #0000FF. Your “desaturate” operation is also shifting the hue, if you define hue based on human perception.)
The lessons from scratchapixel.com were for me the most helpful, they really made all the stuff about colour-spaces "click" for the first time. Mainly this chapter, explaining about the tristimulus, CIExyz and how they transform into sRGB and what is up with that horseshoe shape:
Further, if you click around there's a few more chapters relevant to colours that put this theory into practice (in the context of raytracing and rendering):
(Note, the “hue” measure used in spaces like HSL/HSV is non-uniform, especially in the blue–purple range. So that explains part of the effect you see when you “desaturate” sRGB #0000FF. Your “desaturate” operation is also shifting the hue, if you define hue based on human perception.)