I am a tall man, much taller than the average man so I have some insight from my perspective. First I am 6'5" or about 1.95 meters. I am not giant but significantly taller than average here in the U.S., where not all of us dislike George W. Bush! Anyway getting back to my point, I feel that my height gives me a certain distinction in people's mind. I honestly feel that I generally am ganted an initial higher level of respect due to my height than a shorter man. This is not pompous but my general feeling.
It really is an initial impression though and after that I need to show the personality to match the initial impression. Now from a tall man's perspective when I do chance upon somebody who is taller than I, it is a rare occurance but does happen, I feel a sort of Alpha/Beta struggle where I find myself giving that man instant respect due to his height.
It is my postulation that Tolkien uses height in this way.
Oh and not all tall man are blundering brutes who cannot think logical thoughts
I cannot of course, but I know other tall guys who can.