If you are not overfeeding, or over stocked, then you could vaccum one half the gravel once each week and the other half the following week. This way you will not disturb too much of the bacteria present.
while it is true that the majority of bacteria will be found in the filter, it will establish itself on all surfaces inside the aquarium and in newly set up tanks,,it is best not to remove any more than is necessary to keep water parameter's within non toxic levels.
Would also not clean filter material in anything but dechlorinated water or old aquarium water you take out during changes for same reason.
As mentioned,,if you are not overfeeding,, or overstocked,there is little need to worry about vaccuming the bottom or cleaning the filter while the tank is cycling