I think I saw a mention of ADHD in this thread. I don't know if your kid actually has that diagnosis, or if he's just a typical 5-year-old with a typical 5-year-old immature brain. 5-year-olds aren't generally known for having great impulse control.
I don't have kids of my own, but have taught, mentored, and quasi-fostered kids with an alphabet soup of developmental atypicalities. With attention deficits and/or impulse control disorders, or just typical immaturity at your kid's age, there is really nothing you can do after the fact to change their behavior in the future. However remorseful he may be about what he's already done, he still liable to do something similar in the future if the impulse strikes him. He'll remember after the fact that he shouldn't have done it, but by then it will be too late.
All you can do is set and maintain and supervise and enforce consistent routines (wash hands immediately upon entering the house, interact with fish only with parental assistance), and HOPE that in the inevitable eventuality that no one is there to supervise, they will stick to the familiar routine. It doesn't always work.
I once spent months consistently teaching a 12-year-old that he needed to call and inform his mother each and every time he was going to enter my vehicle or my home. It didn't matter that he came over almost every day after school. Each and every time, I required him to call his mom and let her know where he was before I let him through the door. And I explained why this was important, and he seemed to understand.
And then he got himself into an exceedingly dangerous situation by cheerfully getting into a car in the middle of the night with strangers who offered to bring him along for a ride to a city 40 miles away. Kids like this are nerve-wracking!
I think your idea of childproofing aquarium covers is a good one.
And now you're aware that he doesn't independently remember to wash his hands right away, so he needs adult supervision to make sure he does it.