Algae will not harm your fish. It is simply a visual impact on the fish keeper. Most fish will even benefit from some algae in their tank. All fish need small amounts of greens in their diet, some need far more than others, and nothing is fresher than algae for them. Many fish will also constantly graze at the algae picking off micro-organisms that grow there. If it does not bother you to have some algae on the glass, leave it and let the fish benefit from it in some small ways. As Fluttermoth said, it will even absorb some nitrogen as fertilizer, it is after all a plant.
Be aware that the slimy looking stuff called blue-green algae is not an algae at all. It is a fast growing cyano-bacteria that can indeed be harmful to fish.