The colour of the substrate does have an effect on the fish, either positive or negative depending. The extent depends upon the species. White sand is absolutely detrimental to most of the fish we maintain because they come from habitats that are dimly lit and have a dark substrate, and a white reflective substrate is highly stressful. The dark substrate such fish require may be dark due to leaves, wood, and little or no light penetration, even if the sand is buff toned. To my knowledge there is no white sand in Amazonia for instance, and this is where many of our fish originate.
The other factor is the overhead tank light. This can cause the lighter-coloured substrate to be even brighter, and this definitely unsettles fish that "expect" dark. The opposite can occur too; Ian Fuller has more than once written of the harm both white and black sands cause for Corydoras catfish. I have had this play out in my own tanks.
The lightest natural sand in Amazonia is the buff colour of the overall body tone of so many Corydoras, which is why from above they blend in and are unseen until they move. As one of many examples, some 12 new species of Corydoras were discovered by the British ichthyologist David Sands during the two decades from 1980 to 2000, each species endemic to specific southerly-flowing tributaries of the upper Rio Negro, and having very similar cryptic (camouflaging) colour patterns. When viewed from above in the habitat, the colour pattern of all these fish cause them to blend in with the twigs and branches that litter the buff-coloured sandy streambeds. The patterns include a buff-coloured body with a black dorsolateral band (that may only be partial on some species) which appears as sand and twigs.
If we care about the well-being of the fish we keep, then we must do what we can to replicate the habitat conditions. This eliminates much stress, which means healthier fish.