The majority of polychaete worms are considered beneficial. everyone with live rock as bristleworms.
There are some species that become destructive, but it is very very very hard to identify polychaetes.
However there is a lot of hype and fear promoted. Most are harmless...and you would miss them if they were gone. Might you get a nasty one? Maybe, but it is pretty rare all considering. If you try and remove all thing things in your tank where you might get a bad species, there would be little joy in the hobby, and few things "natural" about the tank.
So offhand, I would not worry, personally. If you have a worm that is 6 or more inches long, that is where I would consider doing something but even then a large number will not cause trouble. But most are doing a very important job as members of your clean up crew. There are bad one's, definitely.. But most are not bad.