[Scott]

I don't think that it is really a question of them going anywhere...
Now this may seem allegorical, and excuse me Mr. Tolkien if it is, but I've always felt that Sauron and Saruman embodied the evil (Morgoth?) in the world. Whether their physical forms ever transcended to anything higher or lower isn't really the point.
The point is simply that their essence, what they above all embodied, endured and will remain in the world until the "ambarmetta", or end of the world.
The symbolism of Saruman's spirit being wisked from Valannor seems to be only a stress on the point of what he came to represent.

Whether Saruman, or Sauron, ever came back again in a form of material essence before the end of days doesn't matter. Their spirits will endure to corrupt, dominate, ensare, and damn no matter what.