I had 3 damsels when cycling and they swam around quite abit. Every fish is different, but my damsels would hide behind the LR if I would go up to the tank, but not during feeding time. I have a firefish now and all he does is hide, he wont even come out for feeding time, he is gonna starve.
Just check you water, if everything is fine don't sweet it.