[Orofacion of the Vanyar]

...there was the presence of Melkor that tainted Aman

And in Numenor, there was the presence of Sauron.

I'm going to go back to something I said earlier. The men in Beleriand had no problem dying, there was no revolt against elves. Men would accept their gift accordingly, and with war going on. Why then would they not live harmoniously with the elves and Valar in Aman?