I have always found the remark in the Unfinished Tales to be very imagination-stirring in regards to this aspect. Just to quote it, for the purpose of debate for easy referrence:
Quote:
|
Of this Order the number is unknown; but of those that came to the North of Middle-earth, where there was most hope (because of the remnant of the Dunedain and of the Eldar that abode there), the chiefs was five.
|
Which, of course, can be interpreted either way, but it does not rule out the presence of some other "southern Wizards" in, say, Harad and so on. However, given what fate the five in the North met - only one, effectively, remaining fully true to his task - and given that we don't hear anything about any, say, amazing revolution against Sauron in Harad, we can pretty safely assume that if there were any other Istari elsewhere, they were much less successful, probably easily overwhelmed by the servants of the Enemy, if not outright corrupted.
It is true, of course, that while we have an account of the selection of the Five Istari, there is nothing about any others. Of course, they may have been chosen later and so on... And it is quite likely that the uncertainity just reflects the "general knowledge" of an average loremaster in Middle-Earth: the Order was pretty obscure for "mortals", even from retrospect.
But all that said, it still opens interesting possibilities.
Also, in regards to what
Inzil said, a bit of polemic - Saruman's remark about the Five rods of course would imply that there were only five, but then again, Saruman may be speaking only in his own cultural context; after all, if there were for example other five Istari operating somewhere in Far Harad, there was probably no way Saruman would even have any news concerning them in the first place, let alone assume someone to get hold of their rods.