Gravel is horrible stuff, the waste falls between the cracks and you are horrified when you gravel vac. So does a lot of uneaten food, and then the fish can't get to it.
With sand, and a few bottom feeders, I never have a problem. The bottom feeders keep shifting the fish waste until the filter gets it, and they also clean up any uneaten food, so the bottom of my tank is permanently pristine. I don't even have to "sand vac", I can just do water changes! (although I usually do stir the top layer of sand around a bit to keep things interesting).