I tested this earlier inside the tower in Dorn's Keep, or however this is called.
On HoF difficulty, I use a rogue, Bounty Hunter in my case, any class with Hide in Shadows will do just fine.
I enable Stealth and move towards a room full of enemies (5-6 close to each other).
Then I step away carefully so as to leave only one enemy out of the fog of war.
I disable hide in shadows.
Only this specific enemy follows me.
Traps and the rest of the group take good care of him.
Obviously the game is not meant to be played like this, at least this is what I personally think.
Should this change?