[Inziladun]

Well, I can think of at least one cultural group of Men that were mostly 'good' and appear to have been little influenced by Elves: the Rohirrim. Even though their homes for long years were in Rhovanion, relatively near large settlements of Silvan Elves in Mirkwood and Lórien, they seem to have had little contact with them and instead regarded them with fear and awe.
Of course, in later years they were much enriched by the Men of Gondor, themselves elevated in knowledge and culture by virtue of their ancestral ties with the Eldar of the First Age, but they were already disposed to aid the West against evil even before then, as the Ride of Eorl shows.
The Rohirrim were not nearly as 'high' as the Gondorians even in the latter's fading years, a warrior culture to be sure. Their behavior couldn't really be considered 'elvish' (the killing of Freca by King Helm "Hammerhand" comes to mind), but they were at heart friendly to ideals that would set them apart from the Easterlings, Men of Harad, and others who don't seem to have any redeeming qualities (at least any visible to the reader).