It's Diatoms. How long does it take before you notice the buildup?
Diatoms are incredibly efficient at gathering food so there's no way to prevent them just by nutrient deprivation.
You can reduce the Diatom population by keeping Silicates very low but you can't really prevent them completely.