When did the last of the High Elves leave Middle Earth ?

[Inziladun]

To further muddy the waters, there is this from Letter #144.

The High Elves met in this book (LOTR) are exiles, returned back over Sea to Middle-earth, after events which are the main matter of the Silmarillion, part of one of the main kindreds of the Eldar: the Noldor (Masters of Lore). Or rather a last remnant of these....those who lingered were those who were enamoured of Middle-earth and yet desired the unchanging beauty of the Land of the Valar. Hence the making of the Rings....when the One goes, the last defenders of High-elven lore and beauty are shorn of their power to hold back time, and depart.

I read that as saying all the Noldor left with the Keepers of the Three. The Prologue notwithstanding, what possible reason would any of them have had for staying in ME when Elrond and Galadriel had gone? I always saw the departure of those two as a vastly symbolic moment: the 'official' acknowledgement of the High Elves that their time was up, and they must make way for the Age of Men.
The Silvan Elves were a different matter. They had spent their entire lives far away from the Sea, and knew nothing of the Sea-longing. I doubt Legolas himself would ever have passed Over Sea had he not chanced to hear the cry of the gulls along the coasts of Gondor. I'm fairly certain many of the Silvan Elves never left, and indeed suffered the 'fading' they were doomed to undergo, holding fast to ME to the end.