I think that's not a very good diet...you should also feed some pellets as well.
It may make him bloated if you only feed them. So, just stick to regular pellet (or flake) food, but feed the shrimp & bloodworms as well, just sparingly so you don't upset his stomach.