Soft water can cause pH to crash. I've read that it's not really dangerous 'til it's below I think 5.5 or so, maybe lower, but (w/ my kit anyway) 6.0 is lowest, so I figure ya don't know how low it really might be.
It is suggested to alter the pH
no more than 0.2 in a 24hr period, maybe 48, but I think it's 24.
I would suggest a water change (what's your tap's pH?). Maybe 20-30%--trying not to be drastic--and see how it measures.
Once it's back where you want it, there's a few things you can do to stabilize the KH. Personally, I use crushed coral in a mesh bag in the external filter.
Might wanna check your water hardness, I could be wrong...but it sounds familiar.