Waterdrop and I are writing an article that deals with this very issue. It's not completely finished yet however.
As OldMan47 has stated, crushed coral would probably be your best bet. I am willing to bet that your tap water has a low KH. I have to deal with this problem where I live as our tap water is so soft. I made the decision to put crushed coral in my filters, and have been experiencing stable conditions.
Here's the short version of the technique that I use: You need to buy a small bag of "crushed coral" or aragonite substrate. You will also need the foot off of a pair of nylons or a tuxedo sock or other suitable mesh material. Take a handfull of the crushed coral and put it in the mesh bag and rinse it out in old tank water thouroghly. Take this bag and put it somewhere in your filter where there is alot of flow. Test your tap's pH, and test everyday until you get a rise of no more than .5 of a pH unit. After you have reached .5 over your tap's ph, do a large waterchange, at least 50%. Note how many days it too reach .5 over your tap's pH. Whith further testing you can work out how many days you can go between waterchanges, and only need to test periodicailly. You also need to "recharge" your bag of media. Every regular waterchange you need to squeeze the bag around so that media in the bag turns white again, making the calcium available to the water.
Now if you miss a waterchange and your pH goes up further than .5 you would do a few smaller waterchanges to get the level down slowly enough as to not shock the fish. It's better not to miss a waterchange unless you really need to miss it.
Hope that helps!!