I can relate to this in the sense that I worked away from home for about 4 months when my wife got our Ginger cat.
In the hierarchy of their domain, the cats also include us humans in their ranking orders, which I observed very prominently during my SPCA days when I had 22 stray cats in my house.
In all instances, I was accepted as the dominant figure of this "clan", but with some of them my wife and kids fitted in somewhere down the order. If any of the cats assumed that they were slightly higher in ranking than the other, the "hissing" would start to provoke reaction and if the reaction is not submissive, the fight will ensue to settle the matter for once and all.
Getting back to my ginger, well, he had just settled in comfortably accepting his superior roll in life (his first
) when out of the blue, I came into the picture being back home so in trying to maintain his "throne", a lot of hissing and blowing was the order of the day whenever we crossed paths.
Obviously, there was no submissive reaction from me, so, to this day, we have this love/hate relationship, where he would blow vigorously at me when we stumble accross one another and I would just merely walk up to him and scratch his back (as vigorously)
You seem to have a very dominant cat (who considers himself to be "in charge" around there) which is extremely frustrated by getting no submissive responses.
Fighting with him is not going to help any as it would indicate to him that you are not sure of your position in life (and has to fight to prove your stance).... When he confronts you, just walk up to him directly and talk (long sentences) in a normal tone of voice and (with no hesitation - he'll pick that up), rub his back or scratch behind the ear (if he does not run away during your approach)
I doubt if he will actually hurt anyone. No need to put him down.