not necessarily, it wouldnt be a great place for the diamond goby but you shouldnt have problems.
IME with fuzzy's they dont eat anything in the tank that was in there before them. lots of people say that you should never keep them with shrimp, however my fuzzy is doing fine with my CBS. they also say that you shouldnt keep them with fish that they could fit in their mouth and my neon goby is doing fine.
i have had 2 different fuzzy dwarf lions and this has been the case with both of them. the only reason that i had to get another one is because my other fuzzy jumped out of the tank.
But IMO you should be fine with a Fuzzy dwarf, a dwarf angel, and a diamond goby.