The main issue from my experience is the potential for alergic reaction and rash, not the pain from the actual event. Pain from getting the bristles in your skin will vary depending on how far they go in (e.g. if it's a baby worm or a big one). When I've gotten small spines in areas with thick skin, I haven't even been able to feel it, but elsewhere on my hand/arm it's about like stinging nettles at worst. Removal is pretty easy using tape to pull the spines out (don't use tweezers - it will break them off), and if any get left in the skin you'll just get little blisters there that will go away pretty quickly. The sting will die off within a few minutes, but as mentioned you can have an alergic reaction that lasts longer. As for being "bitten," I've had that happen too, but only with a couple really big ones (12"+ in length). First and worst time I had that happen it took a really long time to heal because of the reaction my skin had to it. I'm not entirely sure what the worm got me with since they don't have "jaws" in the typical sense, but the worm got a pretty good hold on me and when I finally got its mouth off of my arm I had puncture marks all the way through the skin. The good news is that gloves seem to work against being bitten, as I've never had a worm interested in nibbling on rubber. I've never had a worm smaller than about 10-12" try to eat my arm either, so it's probably not something to worry about with average size worms.
If you have small (<6" or so) worms and they aren't crawling everywhere, you are probably safe to grab rock with thick rubber gloves when you can SEE where your fingers are going. Digging into the sand and such is a risk, even with gloves. If you pick the rock up from a visible section and inspect it before handling it elsewhere, you should be fine.
Latex (doctors) type gloves are less susceptible to water ingress but are a lot thinner. I think they would still protect you from bristles though.
I'd never use those in a tank with bristleworms. Those spines go through rubber just fine, it's just a matter of whether they break off long enough to punch through all the way. Latex surgical gloves wouldn't offer any protection against anything but perhaps the really small young worms. Rubber gloves only protect against average to small bristleworm spines. When bigger worms have been involved (admittedly larger than most people ever have to deal with), I have gotten spines into my fingers right through rubber gloves that would normally protect against just about everything else. The gloves will work against smaller worms and will work against most accidental encounters brushing against the worms.