Same here. 2012 Macbook Pro, 16GB Ram, SSD etc. Pretty sure the issue relates to thermal throttling.

It's particularly bad when in clamshell mode, plugged into the Thunderbolt display.

kernel_task CPU usage shoots up when CPU throttling kicks in. I suspect however GPU throttling also is affecting performance.

At times the GUI just feels really slow and laggy. Seriously considering ditching OS X altogether.