Boss will not attack you if you stand outside of melee range.
However, you can still attack her.

http://www.wowhead.com/npc=20885/dalliah-the-doomsayer