I don't know how big the pellets are, but try putting one in a cup of water and seeing if it swells up to look like the stuff in your tank. If it is indeed left over pellets then you are feeding too much. If one of those balls of stuff is one pellet then I would only be feeding 2-3 pellets a day.
You can supplement the shrimps and snails diet with some veggies like cucumber or squash slices. I freeze the slices and then thaw them out (makes them a bit softer) and then stick them on the handle of a spoon. My snails and plecos love it. Just take the left overs out after a day so it doesn't dirty the water.