My personal favorite place to find board shorts would be a place called "Fun in the Sun" on West 4th. It used to be a tiny place before they moved locations but generally speaking, the prices there are really good.
http://www.yelp.ca/biz/fun-in-the-sun-vancouver