Canadian1, the situation is not as bad as some of the previous posters make it out to be. Fish are generally hardier than we give them credit for, so a little chlorine isn't going to kill them -- as you seem to know since you haven't been using conditioners yet. Chorine will diffuse out of the water in about a day anyway, so if your water company was just using chlroine none of that was in there anyway. The cycling bacteria we culture in out tanks can actually take care of the chloramines, too. Many of the strains out there today, can actually take the amine part of the the chloramine, amine=ammonia, and use that just like they use the ammonia in our tanks. Then the clorine part gases off just like above. Water companies are actually quite concerned about these strains since they limit the chloramine's effectiveness in killing the really important bacteria like E. coli and whatnot. And, you can't just put more chroramines in there since the cycling bacteria will just grow more in responce. In fact, if those bacteria were growing at your water company, chances are really, really good that you have grown the exact same strains in your tank -- that is most likely where the cycling bacteria came from in the first place!
Heavy metals are a concern. Several heavy metals are at perfectly safe levels for humans, but at deadly levels for fish. Copper is the best example I can think of. By the time copper levels are high enough for the average human to tate them, the copper levels are still perfectly safe, but are over 20 times the average deadly limit for fish. The toxic levels for people are 100-1000 times higher than the toxic levels for fish. Some of the other heavy metals are similar. But, chemicals like dissolved organics (DOCs) can bind and protect fish, so if you have live plants, they will protect the fish. The DOCs bind up the heavy metal, make them much less toxic to fish, and make them easier for plants to take up -- so plants excrete DOCs naturally, and also when plants decay lots of DOCs are released.
All that said, I still use conditioner. I've used one bottle for the entire last year. It's not as if it's going to break the bank, and frankly I just think that it is better safe than sorry. The conditioners do no harm, and again, I just like the extra protection.