There is a very interesting game theory description of communication that fish exhibit that I read about a while back. Consider a school of smaller fish that wants to know information about a larger fish. They would want to know, if the larger fish is a predator and then how aggressive or hungry it may be.
Following game theory, we consider every member of the smaller fish to be one of two groups, a brave group or a cowardly group. Well, if a fish were brave, it would swim right up to larger fish, and frankly, probably be eaten very quickly. All that remains are the cowards to pass on their cowardly genes, and make an entire population of cowards. But then, no information about predators could be passed back to the group, as no fish would leave the school to get information.
What does happen is that small fish approach larger fish is short bursts. That is, a group of smaller fish swim forward a few centimeters then wait. On the whole, each individual fish wants to hang back, that is the cowardliness is dominant, but the information potential is so great that each fish takes its share of the risk by going a short distance away from the main group.
In 1987, Milinski published a really neat paper confirming these observations. He used sticklebacks and two different kinds of mirrors. A straight reflecting mirror and a reversing mirror. When a lone stickleback was swimming next to a straight reflecting mirror, it saw its reflection in the mirror, and felt that it was not alone, so fish placed in front of the reflecting mirror swam pretty much right up to the larger fish knowing that other fish were also taking on a share of the risk. On the flip side, when a fish swam past the reversing mirror, it saw its reflection swimming backward, so fish placed in front of the reversing mirror almost never took on all the risk on itself. In game theory, this is called a tit-for-tat strategy, or in other words, "I know you took some risk, so now I will take my share of the risk".
I think that many different kinds of animals display this behavior. Another rather famous game theory example are vampire bats. Vampire bats' metabolism runs so quickly and they are such a small animal, that if a vampire bat does not get food for three staight days it will die. Consequently, they have developed a system where if one bat comes back to the roost very well fed, it will regurgiate some of its food to help the other bats that were unsuccessful at finding food. The bats learned which members would share and which were greedy. Over time, it was observed that a bat that had shared in the past was much more likely to be shared with in the future, and a bat that was greedy was almost never allowed to share if it came back hungry. Again, tit-for-tat. "you shared in the past, so I'll share with you now".
While this may not be the direct communication the OP was talking about, it certainly describes some very sophisticated communications occuring by fish, and how the dynamics of a population plays itself out.