Edain who remained in Middle-Earth

[Aiwendil]

In "Of Dwarves and Men" (HoMe XII), Tolkien says:

Thus many of the forest-dwellers of the shorelands south of Ered Luin, especially in Minhiriath, were as later historians recognized the kin of the Folk of Haleth; but they became bitter enemies of the Numenoreans, because of their ruthless treatment and their devastation of the forests, and this hatred remained unappeased in their descendants, causing them to join with any enemies of Numenor. In the Third Age their survivors were the people known in Rohan as the Dunlendings.

And of course, the people of Ghan-buri-ghan were descendants of the Druedain who lived in Brethil in the First Age.