It sounds like your tank is still cycling. When you cycle with fish it takes longer for it to get fully cycled and you need to be doing water changes everytime the ammonia or nitrites get to .25.
When you do a water change you aren't loosing any bacteria as there is very little in the water...