I'd elaborate on this point a little further. What matters is not just how much you feed, but how much protein it contains. This is because it is protein that forms the nitrogen-bearing waste producs that tax the filter. Fats, ash, water, and sugars do not have this effect.
If you look at dried food like flake or catfish pellets, you'll see that the protein level per unit weight is typically around 30%. But frozen and fresh foods typically have far, far less. A slice of courgette, for example, that you would feed to a plec contains less than 2% protein. Frozen (not freeze-dried) bloodworms are under 5% protein. Fish actually need tiny amounts of food, and for this reason it is probably best to use frozen, low-protein foods as the staple and relegate flake foods to a supplement. That way, the fish are "filled up" without getting overfed. (Think of it in human terms, we may like the meat, but the vegetables are far more important for health and are the bits that make us feel satisfied.)
Often overlooked is the point that many aquarium fish are omnivores, not carnivores. Many tetras, most catfish, almost all cichlids and barbs, and all the common livebearers feed at least in part on plant material in the wild. Visiting Fishbase and reading over the "diet" section of species entries there is fascinating. Did you know kribs eat algae in the wild? Or that Corydoras eat decaying plant material? Mollies and platies feed almost entirely on algae in the wild. Violet gobies also eat algae despite their huge mouths, and even halfbeaks feed on pollen, in the case of some species more often than they eat insects!
For this reason, I highly recommend using vegetarian rather than regular flake. Because it is made from algae it provides a better staple for many species than carnivore flake, especially if you are keeping things like livebearers. Most fish will eat it readily. Also offer slices of soft vegetables such as cucumber; you'll be surprised how many fish will come along and nibble on it. Cooked peas are another useful food. Even some of my pufferfish like these, and they are excellent source of fibre.
In other words, look at the selection of fish you have, research their habits in the wild, and extend the range of foods you offer them accordingly.
Cheers, Neale
Gatorbait has a very good point, it's not about the number of times, but about how much you feed in total.