Radagast the Brown??

[Firefoot]

Basically, yes. Yavanna chose Radagast presumably because he was like her - interested in the birds and the beasts. So that potential to become distracted was always there. It didn't have to happen that way; Radagast could have used those gifts to aid their cause. On Saruman, I don't think that he necessarily would have become greedy because he was like Aule, but as in Radagast's case the potential was there. Saruman was clearly always somewhat of a leader, and it follows that "power corrupts"... Each of the Istar were chosen because of their personalities; it just happens that Radagast's and Saruman's were not the 'right' personalities for the task. They didn't live up to the requirements as outlined by Manwë:

"For they must be mighty, peers of Sauron, but must forgo might [Here does Saruman fail - he wants to be too powerful], and clothe themselves in the flesh so as to treat on equality and win the trust of Elves and Men. But this would imperil them, dimming their wisdom and knowledge [This would be where Radagast failed: his wisdom should have told him that he was becoming too involved in his 'hobby'], and confusing them with fears, cares, adn wearinesses coming from the flesh."

Hope this clears things up.