It works the same way as Windows Explorer and can't be turned off. One way to mitigate is to click on the icon instead of the text.
http://www.ghacks.net/2011/01/02/lef...gs-you-can-do/
Or even better, use the keyboard instead of the mouse to select items.