Your old gravel is doing no harm and may be helping a bit. Leave it in place until things get stable. Big water changes are your friend until the last of the ammonia is no longer showing up in your tests. After that you can start worrying about things like nitrates instead.