I did manage to lure them one by one thou.
Once the dungeon is cleared of all the shadows, I set Hirge at the hall of the dragon, and tell her to wait.
Then I go to the throne room and literally advance step by step. It takes time, a lot of patience and careful watch over the crowd, but eventually one (and only one) enemy is closer to me while the rest are farther. That is the moment when I get 1 or 2 steps closer to trigger his/her reaction. Then I bravely (but also slowly) retreat to the safety of Hirge, then we kill the enemy.
Rinse and repeat. I mean, including the pets (tiger and golem iirc)
Some may think it is tedious, but I love the thrill of the lurking. Besides, I can only handle Blafarne if she is absolutely alone.