If that pump is pumping 4000 gph and all of it is going through the refugium, then yes that is too much flow. A fuge should have about 10 times turnover an hour, so that would mean if you are doing half of that 20L as a fuge, that would mean you only want 100 gph flowing through the fuge side. Remember, a fuge should work something like a tide pool (calm, place for space without predation) so having that much flow will make it more like rapids.