NO, NO! NOT a caterpillar! Definitely not a caddisfly. And certainly not something that you can find in your garden. It appears to be some sort of nymph, probably a cranefly; I will try to look it up for you to be sure. (I'm into aquatic entomology, so I would know a bit about this.)