That's what I do! I don't even linger long enough to shut the cover. I put the food in the tank then slink away stealthily, go to the other side of the room, and watch. I'm telling you, I can be practically inside the tank and they still swim around normally, not bothered by my presence. It's only when food is in the tank that they act all schizo. Yesterday I place a small piece of purple seaweed in the tank and they continued acting normal for about 20 seconds until they noticed it, then they fled. I was across the room. Eventually they ate the whole thing.
And I don't know how many times I have to say this, but it's EVERY fish in the tank, not the older ones or smaller ones cowering before the bigger, newer fish.