How many Wizards?

[zxcvbn]

In the text of LOTR, Saruman mentions 'the rods of the Five Wizards' and among Tolkien readers it is generally accepted that there were only five of that Order. But after reading this passage from Unfinished Tales I consider it possible that there were more.

Of this Order the number is unknown; but of those that came to the North of Middle-earth, where there was most hope (because of the remnant of the Dunedain and of the Eldar that abode there), the chiefs was five. The first to come was one of noble mien and bearing, with raven hair, and a fair voice, and he was clad in white; great skill he had in works of hand, and he was regarded by well-nigh all, even by the Eldar, as the head of the Order. 1 Others there were also: two clad in sea-blue, and one in earthen brown; and the last came one who seemed the least, less tall than the others, and in looks more aged, grey-haired and grey-clad, and leaning on a staff. But Círdan from their first meeting at the Grey Havens divined in him reverence, and he gave to his keeping the Third Ring, Narya the Red.From the Essay 'Of the Istari'

So the five whom we know were only the chiefs among those who came to the North of Middle-earth. That implies that there were more Wizards in the North, and even more in other parts of Middle Earth, the ones 'off the map' provided in the book. And why not? It makes sense that the Valar would send more than a measly five messengers, considering the sheer size of Middle Earth.

What say you?