to the OP-
there are certain things to keep in mind if you are going to keep him in a smaller tank until you get your bigger one. dont feel put off by this forum where folks just rant and rave that the tanks too small, you are killing your fish,NEEDS to be a 5 gal or you are a cruel torturous person(not saying this as a quote folks-taking it as intent), etc. bigger tanks are almost always better for anyfish.
to the OP-
most important thing that you can do RIGHT NOW is to change his water frequently. half everyday would be best, but everyother day would be fine if you feed him every other day. reed less than you normally would (less food in=less waste out) be sure to clean up any uneaten food and poo any time you see it(use a turkey baster to suck things off the bottom).
definitely get more plants! some taller ones would be best as the tank is tall. it will give him something to rest on and help a bit with water quality. look for easy to care for plants like elodea (also called anacharis) camboba, water wisteria.
if you have an airpump and an old,.small fishfood container (or similar container) you can quickly,easily and cheaply make your own small filter that works VERY well. ammo-lock is also another good suggestion (As mentioned above. it does bind the ammonia, and needs to be replaced everyso often. you can "recharge" ammo chips if you buy aquarium salt (recharging instructions are on the box of ammolock). then you can rotate the bags. by binding the ammonia, it takes it out of the water column where it is "stuck" to the chips (sort of like meds do when running black carbon). if you change your water every/other day and use ammo-lock, i would worry very little about the "poisons" getting him.
a good liquid test kit is also essential!! personally, i feel it is the second thing a fishkeeper should buy second only to the tank itself. will help you figure out when to replace the ammo-lock and do water changes (or cycle your filter if you get one). but do NOT use ammo-lock if you get a filter. will not allow it to cycle.
a heater is also very important. they are tropical fish. if the room temp is very warm (75F or higher, which unless you live in a warm place or keep your home ridiculously warm is unlikely). you can move him on top of the fridge until you get a heater. the kitchen is usually the warmest room in the house, and hot air rises.
basically, right now you will need- test kit, heater, and either airpumpand container OR some ammo lock. plants would be good too. if buying ammolock, it would be best to get some aquarium salt at the same time (not marine salt) as you will want to "recharge" the chips.
i know nearly all the "regular" poster opinions about this here, so im not looking to get into a big "thing". i just think that all too often new folks come here, from all sorts of backgrounds (could be a kid spending weeks of saved up allowance on a betta who has no CLUE what they need, and lets face it most likely didnt get very good advice at the lfs. then they are curious about their new fish, find us and are proud of their new fish! to say that basically, its dead already unless you go spend probably on the order of 10X plus what they spent on the fish is ludicris! they see this, add up their "allowance" and how many weeks it will take to save the $ for it, etc and either say, nope not worth it and neglect the fish thinking its dead anyway, or certain people might go into a panic thinking there is no way to help their fish stay healthy as the simply cannot afford it. do we see many of these people come back. in a word no. they probably dont feel welcomed and feel belittled. this case is a bit different in that they stated they will be getting a bigger home soon (and should pinch every single fishy-penny until then to get the best possible set up they can at that time). but it is the same attitude that is getting really old. i know it has also driven some very knowldgeable folks away from replying to be constructive and help with the current situation, as they dont want to deal with the backlash that may ensue. (not to say very knowlegable folks ARENT replying but still., this entire website is meant ot be a community of fish lovers and supportive community. a lot of times it doesnt seem very supportive to me
too often there are no solutions happening, only criticizm and sometimes personal attack (on the OP or other). that tank isnt the worst ive seen. not by a longshot, and there are certain things the OP can do in the meantime (until they get the larger tank as stated) to keep that lovely boy perfectly healthy.
*braces for the inevitable*
good luck with your fish. hes lovely
cheers
I know quite a few people who keep Bettas in Fluval Edge tanks; they'll be fine. 
They make good ( but expensive ) shrimp tanks if nothing else.
