You should be alright. Though most seem to suggest using no more than 2 or 3 fish during the cycling process, Damsels are just about bulletproof! They seem to be able to take some pretty rough water conditions without breaking a sweat.
I don't know too many people whose Damsels died of...