Its in the tap water, and any of the decor you added. Nothing kills ALL the bacteria, some always manage to survive... So, that's where they actually come from. Its just a matter of getting them to grow.
Imagine that they double in number every 24 hours. Having a single cell to start means that it would take 8 days before you end up with 256 cells (which is a minute number really). Meanwhile, if you have 20 to start, it would take only 4 days to get to 320 cells. The key is that the starting number more than anything else, will really determine the amount of time it takes.
You can adjust their growth rate by tweaking the temp, the oxygen level, the pH, etc. But, ultimately, it all comes back to what you start with.