The full extent of the Reunited Kingdom

[Zigûr]

the dynasts descended from Aragorn would become just kings and governors – like Denethor or worse. (Letter 256)

The implication I take from this is that the spiritual or moral side of the kingship would wither away, and that perhaps even Eldarion may have been more an administrator than anything.

I don't think Aragorn and Éomer's campaigns in Harad and Rhûn were ones of subjugation and conquest. I think they were to subdue or force into peace those Men who refused to make peace after the fall of Sauron.

For though Sauron had passed, the hatreds and evils that he bred had not died, and the King of the West had many enemies to subdue before the White Tree could grow in peace. And wherever King Elessar went with war King Éomer went with him; and beyond the Sea of Rhûn and on the far fields of the South the thunder of the cavalry of the Mark was heard (Appendix A)

I notice a lot of fan maps seem to draw the Reunited Kingdom's borders around Near Harad and the Sea of Rhûn, but these quotes don't suggest conquest to me.

I doubt the Reunited Kingdom was ever much larger than the traditional heartlands of Gondor and Arnor, and I would think it very unlikely that its territory ever reached anywhere near the extent of Gondor alone during the reign of Hyarmendacil:

The might of Hyarmendacil no enemy dared to contest during the remainder of his long reign. He was king for one hundred and thirty-four years, the longest reign but one of all the Line of Anárion. In his day Gondor reached the summit of its power. The realm then extended north to Celebrant and the southern eaves of Mirkwood; west to the Greyflood; east to the inland Sea of Rhûn; south to the River Harnen, and thence along the coast to the peninsula and haven of Umbar. The Men of the Vales of Anduin acknowledged its authority; and the kings of the Harad did homage to Gondor, and their sons lived as hostages in the court of its King. Mordor was desolate, but was watched over by great fortresses that guarded the passes.(Appendix A)

I don't get the impression Aragorn's kingdom was anything close to this. Of course I might have forgotten something. There's always an extra bit of info about the Fourth Age lurking somewhere.

[William Cloud Hicklin]

I suspect the Reunited Kingdom would have gone ahead and reasserted its ancient nominal claim to everything west of the Mountains, except Rivendell and Lindon, and Rohan by ancient treaty - easy enough given a complete lack of rival claimants or meaningful opposition (the Shire, Bree-land, Orthanc, Fangorn, Druadan Forest and the Glittering Caves were "under the protection of the sceptre.") South of the Gap of Rohan, Ithilien of course; one gets a vague sense that when Aragorn "gave" Nurnen to Mordor's freed slaves it was thereby also something of a Gondorian protectorate. But I don't see Aragorn's ambitions extending farther than the Poros, unless he decided it was time to subdue Umbar once and for all. Rhun? Punitive raids against Sauronian dead-enders, yes, but there's no way Gondor could have meaningfully held all that territory.

In short, I think Elessar was concerned with re-establishing Elendil's kingdom, but no more.