Don't think of it as "changing" a filter. You are absolutely right that the filter and the materials or "media" inside are the surfaces that the beneficial bacteria like. So you don't change them if you can help it. What most forum members say is that you should rinse out the media in old tank water every couple of months. Only really change a sponge or other media if it is falling apart. So that means some of your media will last years. The manufactures want you buying the stuff regularly, so they recommend replacing the material but it is unnecessary and could cause mini cycling because you would be throwing out your bacteria.
Also most people recommend using charcol only to remove medications. If you have medicate the tank, then when the medication needs to be removed use the charcol. Charcol is only good for 3-5 days, so save your charcol and replace that in your filter with another sponge or better yet, buy some ceramic rings (available in the filter section of your LFS), both of which are good materials for your beneficial bacteria.
Hope that helps.