Yes, it would be a harder life, but it's the dignity and sense of pride and ancientry which we just seem to lack nowadays. I'm a nature-lover, and it just makes me sad to see the trees gradually disappearing all over the place so we can make office buildings. I currently live in suburbia and there are parks and trees and things but in time it'll all be industrialized, and I have a sneaking suspicion I'm going to move somewhere more rural at some point in my life. Then again, I love the city, so I guess I can go both ways.
Actually, Middle-earth has got its share of racism... look at the Elves/Dwarves, Rohirrim/Dunlendings, Orcs/Men and Elves.