I wash my hand with hot water prior to entering my tanks and have never had adverse effects from doing it that way. I think introducing a glove that has who knows what on it (powder, etc,) pose more of a threat than a clean hand.You should try using gloves next time. I use the disposable gloves that you can buy these on ebay or amazon with good results. It's not really safe to work in the tank without gloves anyway...
The glove is specifically designed to be used in aquarium and I give it a wash before going into tank so gloves are safe for the tank. You can get non-disposable gloves for working in pond which may be cheaper in long run. These are also safe for the tank.
It is not safe for humans to work in the tank without gloves. Even thought the threat level is not hight it is not zero nevertheless.