Depends on how often you want to go. If it's a couple times a year thing and you don't want the hassle of renting your own gear, then you can go rent it. Most people will start off by buying their own goggles and fins.
If you want to buy your gear, then it adds up. Can run you into the thousands for a dry suit + BCD depending on what you get.
You will always need to rent tanks. Pricing when I dived was $10 a tank, but it's been awhile since i've dived.
Go take your basic padi. Buy goggles and snorkel.
If you like it, take your advanced and then decide what gear you want to buy from there.
http://www.scubagearcanada.ca/ to get a feel for pricing