There is a huge problem when thinking of 2.5 gallon tanks. That is, I don't know of any filtration that will work in one other than a custom undergravel provided with the tank or possibly a Penguin "Mini."
As for what to put in there, I don't know of many for that either. I would assume either one male betta or a couple of small livebearers. Fancy guppies would make for a very bright and colorful aquarium that would otherwise be seen as a glass of drinking water. As for cycling, I would recommend either the temporary use of striped dianos borrowed from an LFS or friend or to be later placed in a larger, more appropriate aquarium. Striped dianos are very hardy, much like the goldfish you had earlier, and a relativly small fish at trade size. This fish is very popular for cycling, and use as dither in larger community tanks. As far as an alternative goes, "fishless" cycling is an option, but I don't know how well it would work with an undergravel filter, since it relys on the build-up of "mulm" or unused food and poo. It would however work well with a "mini," if you could fit it on the tank, and you weren't keeping bettas (they like calmer waters).
Now, as far as hardware. The tank is needed of course. If I were you, I would prefer one that is shaped more like a cube than a coke bottle so you get a little more surface area. However, the taller ones "can" make viable aquariums. Second, you will need a light. This can either be a desk lamp, the ceiling lights or one provided in a hood. Also, you will need a thermometer to check the temp of the water. There are small strip ones available that are acurate enough for this set-up. If temp is a problem, ie too cold, you may want to invest in a small heater. You can either get one that goes in the tank, thus wasting precious tank space or one for reptiles. I would assume that the reptile heaters would work best if you could protect them from dripping water. I would wrap it in a small thin towel except under the tank. And remember, this is only necessary if cold water is a problem.
The second most important thing is filtration. If a mini fits on the tank, and the fish that you keep don't mind slightly turbulent water, by-all-means by the "mini." There cost around $10-15, but are well worth the investment. If keeping a betta, or if an undergravel filter is provided and you want to use it, you will also need an air pump. This will be the drive of the undergravel lift tube, as I don't know of any powerheads that won't throw fish out of 2.5 gallon tanks.
Next, substrate can be important. If useing an undergravel filter, you need to stick with typical aquarium gravel. However, if keeping small plants, you may want to consider a little different substrate. Sand makes a great substrate, and you may mix a small bit of laterite or clay in the bottom of it. Good plants would be dwarf sag. or anubias nana or maybe some baby tears for a little height in the back.
So, all that to say this, set up the tank how you want, put ONE striped diano in there (you may see if the LFS will let you return it if you don't have at least a 10 gallon and 5 buddies to go with it), let it set for 2 weeks, take a water sample to your LFS. Have them check for ammonia and nitrIte. You should be right in the middle of the cycle by now. Continue to take samples each week, until both ammonia and nitrIte are zero. Do a water change (about 1 gallon worth) and the next day go get your planned fish. Remember, only one male betta, or only 3 fancy guppies.
Questions you need to answer-
1. What filtration?
2. What water temp is norm in the office? (actually check it out, don't rely on air temp)
3. How to solve # 2? (come tell us the temp., we'll help solve)
4. What fish are you going to keep?
5. What plants, if any?
