the general rule you'll hear is one inch of fish per gallon.
my ten gallon tank has 3 platies, 3 small cory catfish and an aquatic frog. I suppose with a lot of work you could have some more, but I probably wouldn't want to add any more.
Remember to make sure you have 2 females per each male. Otherwise they get really stressed out. To avoid this I bought all females. You could always buy two of each fish for a 10 gallon tank (making 6), providing you dont mind having all females.
You could probably keep anywhere from 5-9 in a 10 gallon tank, depending on size. Mollies are larger than guppies and platies aren't they?
It may be a good idea to wait for more replies, as im no expert and speak from personal experience.