I'm going against you all and say leave the fish in the tank. Just gravel clean the substrate the day or two before. Then scoop it out with a net. When all the gravel is out, add the sand.
Try to clean the sand the day before so you don't have to spend hours washing it before adding it to the tank.
If you remove the gravel from one side of the tank and then go to the opposite end and do the same, the fish can hang out at whichever side you aren't working in.
The fish will get stressed but they also get stressed when you chase and catch them, then move them into a holding tank for a few hours, then chase and catch them again to put back in the main tank. However, the choice is yours, but unless you're proficient at catching fish and have a spare container, I would just leave them in the tank and remove the gravel with a net.
Don't keep the gunky water from a gravel clean, that stuff is just nasty and contains way more harmful micro-organisms than good, and you can add a couple of aquarium plant fertiliser tablets to compensate for the lack of gunk in the new sand.