I have a few suggestions, but it's up to you do actually do something about it.
1. If you don't have corals, which you didn't mention, simply turn your lights off for about two weeks and replace your phosphate removal media.
2. Buy a quality protein skimmer - your sump is a bit small for most skimmers, so you may have to look into a hang on back skimmer. I really like Octopus brand skimmers. They are quality built and perform very efficiently.
3. Because your algae is brown, it means that your tank has a very high amount of nutrients in it. You need to start stripping the water of these nutrients.
3A. Buy and use an RO Unit. Do not use tap water in your tank at all. When you use your RO Unit, make sure you change the filters out every three months and regularly flush the membrane and replace the membrane every year to two years. Buy and use a TDS meter - and make sure your TDS reading is always less than 3.
3B. Feed quality foods like frozen mysis and brine shrimp. Also, you can make your own home made foods and save yourself a lot of money. Since you have tangs, you can feed Nori - which is seaweed. Don't overfeed your tank.
3C. Do proper water changes - Remove all of the detritus out of the sump, clean your bioballs in the old aquarium water. Use a powerhead to blow off all of the rocks, inside and out and siphon out as much gunk as possible and let the rest get caught by the mechanical filtration. At the next proper water change, you can gravel vac the top 1" of sand to remove built up detritus and waste.
4. Remove your canister filter completely. If you are not regularly maintaining it every two weeks to thirty days, it is doing nothing but contributing to high nitrates and phosphates. Canister filters, if not maintained - if they are ever turned off for any period of time and then turned back on, they can crash a saltwater tank really easily.
5. Once youve turned your lights out for two weeks, fed cleaner foods, done four or five proper water changes and made sure you are using 100% clean RO water you can turn your lights back on and put them on a timer for 6 hours a day. Change your light bulbs - update them if they are out of date. Old bulbs shift their spectrum to the yellow side which can cause major algae outbreaks.
6. Increase your water flow - This keeps detritus from building up on the sandbed and rocks and move more of it down your overflow to be caught by your mechanical filtration.
7. Use your mechanical filtration in pairs of twos. Replace the top filter one week, place a new filter under the old filter and keep it going like that. This helps to constantly keep some bacteria in your tank - and prevents any kind of biological filtration mishaps.
8. If, after doing this for the next two months and you are not seeing any results at all - and you still have algae problems, consider keeping the schedule that you have been on and simply remove all of your rocks and spray hydrogen peroxide all over the algae invested rocks - and then put it back into your tank. The hydrogen peroxide does not affect the living stuff on the live rock, but somehow kills only the green and brown algae on the rocks.
9. If your algae starts going away and you are seeing good results physically but your test kit still tests high nitrates and phosphates, but a better quality test kit from lets say... Seachem or Salifert. Test kits do occasionally go bad and give bad test results. But, at this point, that is not the case because you are obviously having a serious algae problem.
Something that you can consider setting up around / on / in your tank or sump would be an algae scrubber. Algae scrubbers remove algae, cyano and other undesirable algae by growing it on a screen instead of your tank. They are very effective and very efficient. I have proven for myself that I can eradicate an algae problem in a tank within a month with just the use of a properly built and maintained algae scrubber.
If you just start doing a combination of all of these steps, you will be well on your way to a better looking tank.
