Cardinals are in danger if you buy them tiny. Adults are fine. I had to give the fish away for a move, but I recently had 2 very large wild caught angels in with 40+ 5 year old cardinals, and there was zero danger of predation.
Where the danger lies with angels is they can quickly make you get more than one tank. They are hard to sex with 100% accuracy, and if you get two males, yikes. But if the 5 were together, and 4 are still available, they might have sorted their social issues. A 55 is tight, but a lot depends on whether it's a 3 footer or a 4 footer. You should be fine.
I'll respectfully disagree with
@anewbie and say with 2, you have a 50% chance of trouble. 4 is a low number, and I would go for the lot. Even if they fight, if they are evenly matched things will spread around.
But if mayhem breaks out (it could with two as well), then they have to be separated, and that means another tank.