I had a green stripe mushroom attach itself to a hermit one time. It's wasn't long before it released in poor health. I think he just spent to much time on the dark side.
I use to be able to keep xenia and it grew like weeds when I had a skimmer setup. When I switched to a refugium w/caulerpa and my NO3 and PO4 dropped to 0, it got to where it slowly dies now.
I know xenia requires more nutrients than other corals because it dosen't feed like most corals. Xenia don't eat(capture food and ingest it) they absorb nutrients from the water. I just don't think my tank has enough of them to keep it alive anymore and if your nutrients(NO3 and or PO4) are to low it may be your problem as well.