The Wizards of Middle-Earth...

[Raynor]

Originally Posted by narfforc

This could be read to imply there were others who were not the chief five.

However, LotR implies that there were only five:

Originally Posted by Tale of Years, RotK

The two highest of this order (of whom it is said there were five) were called by the Eldar Curunir, "the Man of Skill", and Mithrandir, "the Grey Pilgrim", but by Men in the North Saruman and Gandalf.

There are various similar statements in the letters, such as:

Originally Posted by Letter #203

There were five wizards and that is just a unique part of history.