What should I do?
Try to pin down the source. Keep the filter rinsed, especially the finest media (the sponge/foam type which catches the smallest particles). Check a clear jar/glass of tap water to see if that is the source. Does the substrate stir up gunk easily?