No Living Man

[Firefoot]

Boromir son of Denethor (after whom Boromir of the Nine Walkers was later named) defeated them and regained Ithilien; but Osgiliath was finally ruined, and its great stone-bridge was broken. No people dwelt there afterwards. Boromir was a great captain, and even the Witch-king feared him. He was noble and fair of face, a man strong in body and in will, but he received a Morgul-wound in that war which shortened his days, and he became shrunken with pain and died twelve years after his father.

It sounds to me like the WK feared Boromir not as a threat directly to himself, but as a threat to the forces of Mordor in general. After all, what's a leader without troops to command? The WK probably feared Boromir as a leader and captain of men, not as a threat to bodily harm to himself.