There are some great colorful fish, but keeping them is very dependent on tank size, water parameters and tank mates.
Discus are mentioned are probably some of the most beautiful fish in the freshwater trade. But, they are finicky in terms of their needs in both temp and water parameters.
Barbs - cherry males are brilliant red, as are rosy barbs.
Bettas both male and female are beautiful fish. There are different style of fins.
Livebearers - these can be quite brilliantly colored. Mollies (black are my favorite), Swordtails and platies (come in a variety of colors from red to blue, as well as yellows and oranges), guppies (especially the males, are brilliantly colored) and endlers can be very interestingly colored as well.
Rainbowfish are some brilliantly colored fish.
Even tetras have some very brilliantly colored fish: cardinals, neons, black neons are nearly iridescent, red phantoms, black phantoms, black skirts look very good.
Microrasboras are very interesting colors and the Trigonostigma genus is interesting... like the espei, hengeli and heteromorpha are very interesting.
Dwarf gouramis are excellently colored as well.
All of these are colorful, but your tank size and water parameters are where you need to start this discussion. While some fish can work well in certain situations, they simply can't work at all in others. Gouarmis are great, as are bettas, but they aren't suited to be with each other. Livebearers are also great, but they need very different conditions than many of these others. Rainbowfish are great too, but they need a lot of room to swim. If you only have a 10 gallon tank, forget about it.