In order to keep media viable it needs to have water flowing through it.
When I want to replace a sponge, either in a filrer or as a stand alone sponge filter, I normatlly remove the old sponge, drop it into the tank and put the new sponge in its place. I know that the the old sponge just sitting in the tank will lose bacteria over time as it gets no circulation throught it. What does happen is as the colonies in the old sponge are dying back, new colonies are growing in the replacement sponge due to it having flow through it.
At the end of a couple of weeks I remove the old sponge and throw it out.
Bear in mind that even though the bacteria are able to survive loss of essential nutirents for quite some time, reducing the nutrients is not the same thing. So if one cuts the nutrients in half, about half the bacteria will die off over time. That is because the bacteria as a whole are not being starved. There are merely insufficient nutrients to support them all. When they go dormant it is because there is an absence of one or more essential needs- for example- this can be ammonia, oxygen, inorganic carbon or a combination of them.
The longer your sponges with no circulation through them sit in the tank, then less viable bacteria they will have inside.