In my opinion using the rock as is will create a tank full of Phosphate loaded rocks what will be a constant source of filament algae for at least a year.
You could spend some time cooking the rocks (high flow, low light) until bacteria pushes out and consumes the majority of the PO4.