Perception and interpretation can very much be influenced by language (Sapir-Wharf hypothesis), so to the extent that perception and interpretation influence intelligence, it's not clear that the relationship is only in one direction.

"It would be naïve to imagine that any analysis of experience is dependent on pattern expressed in language."

- Sapir

It's hard to take these discussions on cognition and intelligence seriously when there is so much lossy compression going on.

Sapir-Whorf was named after, but not postulated as a single theory by Sapir or Whorf. It's just a colloquialism for Linguistic Relativity (vs Universality). In its weak form, there are many examples of Linguistic Relativity.

Am I the exception? When thinking I don't conceptualize things in words - the compression would be too lossy. Maybe because I'm fluent in three languages (one germanic, one romance, one slavic)?

Our brains reason in many domains depending on the situation.

For domains built primarily on linguistic primitives (legal writing), we do often reason through language. In other domains (i.e spatial) we reason through vision or sound.

We experience this distinction when we study the formula vs the graph of a mathematical function, the former is linguistic, the latter is visual-spatual.

And learning multiple spoken languages is a great way to break out of particularly rigid reasoning patterns, and as important, countering biases that are influenced by your native language.