Well the dirt in the bio-balls is detritus, which is a breeding ground for nitrates.
Perhaps phosphates aren't your issue but rather nitrates.
Wet dry filters need to be kept clean. Gunk building up in the balls, below them, or in the filter pads will cause problems.
The best thing to do is do a water change and keep all of the water you remove from the tank. Use that water to rinse off your bio-balls and get any built up gunk in the filter removed (a wet/dry vac works very well for this), and change any filter pads.