When biologists talk about "primitive" and "advanced" taxa (taxa = groups of organisms) they mean something very specific. A primitive taxon (the singular of taxa) is one that retains many features in common with its ancestors. An advanced one retains few characteristics in common with its ancestors.
So if you look at bony fish generally, a sturgeon would be "primitive" and a cichlid "advanced". For example, a sturgeon has an heterocercal* tail; a swim bladder connect to the gut; pectoral and pelvic fins that are far apart and held horizontally to the body primarily to generate lift and act as brakes. These are features that are (more or less) shared with the ancestors of the bony fish and can, for the same reason, be seen in other primitive bony fish-like groups, such as bichirs, lungfish and coelacanths.
A cichlid, by contrast, has an homocercal* tail; a swim bladder that is not connected to the gut; and pectoral and pelvic fins that are close together and do not act as hydroplanes, instead being used to generate slow movement and to adjust poise and position.
While "primitive" taxa tend to be older than "advanced" ones for obvious reasons, this isn't always the case. Cichlids and bichirs both appear to have their origins in the Cretaceous. We have fossil bichirs from the Cretaceous so know that fact directly, and while there are no fossil cichlids from the Cretaceous, the fact they are distributed across "Gondwanaland" (i.e., Africa, India, Madagascar and South America) implies that they were living at that time.** So one has to be very careful about directly linking primitive or advance qualities to the age of the taxon in question.
Similarly, a primitive taxon need not be "simpler" than its advanced relative. Classic examples are things like cave-dwelling and deep sea fish that have lost their eyes. Despite being blind, these fish are more advanced than their ancestors. This argument is an issue with Creationists, who assume (falsely) that Evolutionists imply Darwinian evolution is a trend from simple to more complex. It is actually nothing of the sort. There are plenty of parasites that are taxonomically advanced relative to their ancestors but very simple anatomically. The parasite barnacle Sacculina is a extreme example: its ancestors were regular barnacles with recognisable (of odd) crustacean bodies; Sacculina is little more than a mass of threads running through its host (usually a crab) and looks like fungal mold. The only time Sacculina betrays its crustacean origins is as a larva, when it looks just like any other barnacle larva.
So to get to the main point: if you broadly define fish as including jawless fish, cartilaginous fish, and bony fish, then yes, this group has more in common with the ancestors of the vertebrates than do land vertebrates (amphibians, reptiles, birds and mammals). If nothing else, fish lack all those advanced characteristics connected with living on land. So in that sense, yes, they are "lower vertebrates".
But being a lower vertebrate doesn't imply some degree of clunkiness or old-fashioned physiology. For one thing, it depends on perspective. Tetrapods (i.e., amphibians, reptiles, birds and mammals) are closer relatives to the lungfish than the lungfish is to, say, a pufferfish. So if pufferfish were writing the taxonomy books, they might consider mammals to be part of a "stem group" that branched off from the main line of bony fish evolution. Lungs, for example, are PRIMITIVE characteristic within fishes. Lungfish have them, coelacanths have them, and basal bony fish have them (things like bichirs). Tetrapods retain them because they work as well on land as they do for air-breathing in swamps. But bony fish, as a whole, have lost them. The lung became the swim bladder, and where advanced bony fish needed to breathe air, they evolved alternative organs (e.g., the labyrinth organ in gouramis).
Similarly, our jointed arms and legs are (slightly) modified lobe fins: the fins are our fingers, and the lobe is the arm and shoulder (more or less). Again, these are characteristics of primitive fish. Once you get about the bichir level though, lobe fins are done away with, replaced with the much more effective rayed fins. If you look at how different fish use their fins, you can see that rayed fins are FAR better. Watch cichlids swim, for example: much more control and precision than anything a lobe fin can offer. And again, where advanced fish have had to evolve mechanisms for moving on land, they have evolved entirely new systems (e.g., the pectoral fins of mudskippers).
In some ways, immunologists have an interesting way at looking at "advanced" versus "primitive", by seeing how sophisticated the immune system is. When you look at the vertebrate tree like this, you find there are actually multiple peaks, with different groups having "advanced" immune systems (though often in different ways). This is all being dragged up from memory of lectures in the early 1990s, so I may not be up to speed here, but as I recall you have peaks with the mammals, birds, crocodiles, anuran amphibians (frogs and toads) and bony fish.
Be that as it may, perhaps the last word should be a quote from 'Life on Earth'. As David Attenborough put it so memorably, "75% of the word is covered with water; 75% of the word belongs to the fish".
Cheers, Neale
* Heterocercal tails are asymmetrical, with the upper lobe bigger than the lower lobe; homocercal tails are essentially symmetrical with the upper and lower lobes similar in size.
** Specifically, South America and Africa split apart during the Cretaceous. Since cichlids are unlikely to have swum across the Atlantic Ocean from one place to colonise the other, the assumption is they spread across these two continents when they were still one land mass.
