well overflow boxes are cheap, if you wanted to upgrade just throw the old one out, make a new little feature on the face of the new one.
You can still take it out and clean it, it would just have rocks and foam attached to it to make it look real.
Ive never even considered cleaning mine come to think of it. the Utube gets cleaned but not the box, not sure whats even in there LOL.
Overflow boxes are just ugly is all, anything to spruce them up is a win in my book
or how about just epoxy and sand on the face of it? that wouldnt be bad either.