[Boromir88]

Middle-earth is perfect for Men, because Men are mortal. Death comes to them all, and death also quite literally is the nature of Middle-earth.

With Elves, death is not certain, and Tolkien notes their primary weakness is an inability to change. Basically they don't like it. Galadriel tried to turn Lorien into her own personal Undying Lands by stopping Time, stopping her own personal realm from decaying. Try as she might, she could only slow Time, thus only slow decay. So, I think Middle-earth was just an unfit place for Elves, it's nature is in line with the nature of Men, as the Undying Lands is in line with the nature of Elves.