Yes, it is. The rubberband itself isn't harmful (at least from what I know) but if you leave your stem plants in a bunch, they will eventually rot at the bottom, and lose the lower leaves. So yes, remove the rubberband!
If you have fish that dig the plants up, you can use little peices of lead weights and small bunches of 2-3 stems max.
Also, try not to talk in caps!
~Robin