Neat suggestion. I too am seeing shades of brown for most queries. Maybe this would help.

I heartily agree. For example, searching kiwi should result in roughly the color of the inside part. Instead it's a light muted green, probably taking into account a white background that it's usually featured on. Taking the mode by clustering instead of the mean would improve results.

Agreed. A flower should not be represented by a brown color.

Yes, this would be much better. Additionally it could return a pallet of N colors instead of just one.

Using a better algorithm makes sense, though a single color seems to be the point of the "product" in this case.

There's a paper which does something similar including an analysis of different statistical models on top of a CIELAB color space.

Estimating Color-Concept Associations from Image Statistics. Ragini Rathore et al., IEEE VIS InfoVis 2019. https://arxiv.org/pdf/1908.00220.pdf

I wrote a tool that explores different ways to cluster in the RGB space - https://github.com/mattnedrich/palette-maker

It compares k-means, median-cut, and simple RGB space quantization.

mean-shift is great, but it is unbelievably slow (n^2)

I thought you can make mean shift go faster with the Fast Multipole Method.

See: https://home.cscamm.umd.edu/programs/fam04/fgt_duraiswami_fa... (PDF file)