what were The Silent Watchers?

[MLD-Grounds-Keeper-Willie]

I think that it does have something to do with Numenorean. I don't know if tark in orkish slang would be used as a swear word, but its real meaning is this:

In this jargon tark, 'man of Gondor', was a debased form of tarkil, a Quenya word used in Westron for one of Numenorean descent.

-Appendix F, LotR

If we put that with what Snaga said

He's got past the Watchers, and that's tark's work.

-RotK, The Tower of Cirith Ungol

we can see that the Watchers have some kind of weakness to those of Numenorean blood.