How about a photo?
There are so many critters that fit the description "brown worm" that I'd be dead before I could give you all the possibiities. FWIW, it's likely some type of flatworm or bristleworm tho.
Try finding your critter here:
http://www.lionfishlair.com/hitchhiker/hitchhiker.shtml
Just scroll down and match a pic to your critter.
