If it is not retreating totally into the rock I would suggest to you that it is not an anemone. If it is really bothering you though, take the rock out and scrub them off. But seriously these are not a problem from your description and the pictures.
Let's see if others chime in here to give you a better feeling about it all.