Brineshrimp didn't cause this.
When you say gravel rake the substrate, do you actually use a gravel cleaner like the one in the following link, or do you just suck the water out and not clean the gravel?
Aquarium cleaning should take 30 minutes every two weeks. Learn how the professionals do it in 18 easy steps.
www.about-goldfish.com
These types of problems are usually cased by dirty gravel, a dirty filter, lack of water changes and too much food. You can have clean water but if the filter or gravel is full of gunk, it will be a breeding ground for disease organisms that infect the fish.
---------------------
You mention changing the filter when this started.
What sort of filter do you have?
How do you normally clean it?
If you have a power filter, you should wash the filter media/ materials in a bucket of tank water at least once a month. You can re-use the media. If you have filter pads/ cartridges, these can be washed in a bucket of tank water and re-used too. If they start to fall apart, you can throw them away and replace them with a sponge. However, only replace one piece of media at a time and wait a couple of months before changing another piece of media.
If you replace all the filter media at the same time, you get rid of the beneficial filter bacteria and end up with ammonia and nitrite problems.