Yes, yes I totally believe it's expensive. But it's relative to your enjoyment, if it's overly so or not. I can't afford a larger tank, and a smaller tank would probably stress myself and my fish too much. It's taken me literally months and months to save up enough money for all the things I need to start a 20g. I inherited my 10g. I got lucky with my two 5gs.
If you're too poor to jump into the game, though, don't. You'll wind up being cruel by accident, unable to afford to give the fish the food, home, and care they require. The usual, general, answer, is always the most obvious and most frustrating -- research before you buy! Look at what you're able to do, be real about it, and then figure it into your budget. Resist the urge to buy over it, or buy fish who won't fit. Aquarium keeping requires gobs of patience, and IMHO, that's part of what makes it grand.