Fish readily absorb nitrIte from the water and it combines with the hemoglobin in their blood, forming methaemoglobin. As a consequence, the blood cannot transport oxygen as easily and this can become fatal. At 0.25 ppm nitrite begins to affect fish after a short period; at 0.5 ppm it becomes dangerous; and at 1.0 ppm it is often fatal.
Fish that survive nitrite increases can be permanently affected. Different species can seem to manage this better than others, but the effect is still there. Like nitrate issues, it is partly a matter of the fish being weakened, and that has consequences.
Provided you are using a conditioner for water changes, and assuming there is nothing toxic in your source water aside from chlorine/chloramine that will be handled by the conditioner, water changes should not be detrimental.