Bacteria (as was mentioned) live on every surface in your aquarium. And IMO, just hovering over a gravel substrate while doing a gravel vac will do more to harm the tnak than help it. The bacteria are not going to be "sucked up" in any great numbers. Gravel will however hold a lot of "crap" becuase it is easy for the detrius and waste to fall between the cracks, as it were. If you do not "deep clean" (plunge the vac deep into the gravel bed) your gravel (IMO, at least once per month) you are leaving a lot of waste and detrius to rot and add to ammonia problems later down the road. Skimming over the surface of the substrate is great for sand, and it "works" because most of the detrius/waste is too large to fit between the grains of sand and thus will not fall between them and pollute your tank, like it will with gravel.
Short version: You will not suck up all your bacteria. You should "deep clean" your gravel at least once per month. You will not have any mini-cycle if you do this.
\Dan