I concur on the tap water. Retest, but also check with the water authority and see what they give as the GH and KH (Alkalinity), and pH while you're at it. It is always possible that your tests are off for some reason so this will confirm or not.
I have always read that insufficient water changes are more likely to result in a diminishing of minerals, not increasing. Water changes replenish the minerals, here calcium and magnesium, that fish and plants are using. I have always though too much was made of this, because the amount of calcium and magnesium used by even a tank full of fish and plants will not be much, though with low levels to begin with I suppose they might become depleted. And while it is true that evaporating water will leave behind the calcium and magnesium, with water this soft I can't see that really being an issue. I would not insist upon this though, should others have a contrary view. But regardless, the only explanation of increasing GH with minimal water changes would be a source of calcium and/or magnesium from somewhere.
You said you are not adding any substances; my reason for asking this was that years ago I added magnesium sulphate, epsom salts, just a spoon or so for the plants. This was back in the 1980'sand the author of the planted tank column in FAMA recommended this. It certainly did improve the plants' growth some, though at the time I was ignorant of the possible effect on my soft water fish. I had water out of the tap that was zero GH/KH and pH was below 5.