I'm PADI certified. Got my certs here in NY about 12 or so years ago. Your Open Water certification allows you to dive to 60 feet and will consist of classroom, pool sessions and then a check out dive in open water. After that you can get your Advanced Open Water which lets you dive to 120 feet. I think that this is where most people stop their official training since with these 2 certs you can pretty much do any type of recreational diving that you want. I am also Wreck Diver certified and there are tons of other certs you can take like Underwater Naturalist, Photography, Search and Rescue, etc.
Like was already said, you will need at least a mask, fins and snorkel for the classes. After that, if you really like it and think you will go often you can get into buying the other gear like regulators and a BCD (the vest you wear that your tank attaches to). I think I spent about $200 or so on my original mask, fins and snorkel. After that I dropped close to $600 on regulators and about $500 on a BCD. Add onto that a wetsuit or drysuit and you can easily spend upwards of $2000 or more on your gear.