There are legal considerations depending on where you are, that is a big one to start. A potentially very very big one.
If you are not in Florida, then odds are the rock is granite or similar, which is no help to your tank really. It is not going to be particularly porous. If you were doing a regional tank then it would be ok.
Run off, pollution, etc is another big issue with just taking rock from whereever.