i think it's really weird that my tank water is 4 to 6 degrees warmer than room temp. so basically for the fish to be happy, i have to freeze. is this normal or is there something wrong with the tank?
I'm sure there's nothing wrong with it, assuming you have a heater in your tank? The water in your tank is circulating around the tank carrying the heat from your heater. The heat won't transfer to the air and heat your whole room, because of laws of physics which I can't explain, ask a physicist I just assume that most of the heat is lost to the glass-which is cooled to the temperature of the room- before it can get to the other side.
What temperature is the water at just out of interest?
Yeah the general jist of my post was that the hardly any or none of the heat will transfer between the glass either way because it is lost on the cold surface of the glass which is cooled to the temperature of the cooler environment, in your case, the water in the tank In aquarius's case, the air in the room