A lot of it was undoubtedly due to my inexperience at the time, but angelfish, rams and weirdly, even livebearers I've had a bad history with in terms of survival rate. Special mention to livebearers in particular, I remember the likes of platies and mollies never lasting long, which is weird because the natural water around here is the hard, high PH stuff they prefer.
And kind of a half regret, clown loaches. Got one that came with a 30 gallon, over the years upgraded to a 55 gallon and 3 more, and eventually rehomed them due to the fact that even a further upgrade to a 125 gallon wasn't going to be enough long-term. That and they single-handedly changed the community type of my setup once they figured out that they didn't have to wait for small fish to die to be considered edible...
I did find them enjoyable for the years I had them, but I don't regret the decision to ultimately rehome them and have the chance to completely redo the makeup of my new system.