Mine doesnt hurt anything. I think if you have larger hermits, or any type of stinging coral that will teach them to stay away from everything. I have tons of polyps and zoos and it doesnt touch a thing.
However, he is very aggressive towards new additions to my tank. Ive only recently had success with adding a mandarin and a spotted hawk.