Your answers are always so helpful and thorough! When it comes to a small tank, would they prefer a sufficient group size at the expense of what little space they have?
Yes, if I'm understanding the question.
Each species of fish has evolved over thousands of years to function in a very specific environment. By "environment" we mean the water parameters, the habitat conditions, the numbers of the species, and the other species that may share that habitat. These things matter because they are part of the genetic blueprint of the individual species. The individual fish in a species "expects" these things, it is their inherent nature; and when they do not have what they expect around them, stress occurs, leading to all sorts of problems.
But we now know, as I mentioned previously, that there is much more to this. Fish kept in a group of five showed increased aggression compared to a group of 10 of the same species. In the case of the neon tetra, one of the species studied, the normally peaceful fish became aggressive. The Tiger Barb was another species studied, and here it was profound; the group of 10 behaved normally, some interaction but nothing too serious. But the group of five literally tore into each other to such an extent that the scientists removed them from the study because they viewed this as inhumane and cruel. Forcing a fish into unexpected situations will never end well.
In addition to the aggression issue, the fish in the groups of five showed a latency to feed, compared to the groups of ten. In other words, the detriment being done to the fish by denying it what it expects in terms of numbers even causes it to be less willing to eat. That says a lot about the stress.
So a group of ten "tetras" will always be better than smaller groups. It is up to the aquarist to ensure the group has the necessary space to act normally. Here again, we have other studies showing that aggression is increased by small spaces--small to the fish.
We cannot mess with nature.