Yea, I've never understood the whole 3 second memory thing. I've never believed it either.
There's many things we can all name that our fish do that they wouldn't be able to if they had such a short memory...
Go to the same exact spot daily for their food.
Recognizing individual people that are their owners and shying away from anyone else. (cichlids especially do this)
When chasing away intruders from their eggs, know to hurry back because they have eggs to protect. (Studies were done with fishermen years ago that when a fish is caught sitting on a nest, when released it immediately goes right back to the same spot to protect them)
That's just a couple that prove they have intelligence and a memory. At least, imo.