Not sure about OSX, but in Windows you can. This is a huge miss in Linux I find (all distributions I've tried so far), you need the mouse to get anything done without resorting to a terminal.