that depends entirely of what it is. size and color barely narrows it down at all. i would say that it is most likely a harmless bristle worm. is it burried in the skeleton at all? like did it make a large hole in the actual CaCO3 structure or is it jsut kinda hanging out in a hole in the rock?
you can remove it if it would make you feel better, but i really cant say if you should or not since im not sure what it is. you might want to use tweezers jsut in case it can sting.