I am sure you will get more learned responses to this but my feeling is that LOTR is masterpiece but one so unusual that it defies categorisation - which may be why the literati get piqued by it's popularity. Why I say it is unusual is that whereas most writers have a story they want to tell and create a setting around it or place it in a setting they know, Tolkien created Middle Earth more or less as back story for his invented languages and the
Lord of the Rings seems to have sprung almost from Middle Earth rather than directly from Tolkiens imagination.
The Hobbit was a catalyst - people and places from the Silmarillion (written long before - although published poshumously) appeared in the Hobbit but the LOTR is so much a product of the whole of Tolkien's creation of middle earth that just about all of the incidents referred to by character in the story were already written (a notable exception being the story of Queen Beruthiel and her cats). So when Faramir speaks of Numenor or Legolas speaks of the elves of Hollin, it is not Tolkien creating an illusion of history; they are speaking of their actual history which happens to be written down elsewhere.
Middle Earth has an "internal logic" and although it is a fantasy world it is a plausible fantasy world - not flawless and if you look around the downs you will find various debates and disputes about the degree to which people "buy in to " the fantasy varies but for me it does fulfil Tolkien's aim of being a created "mythology for England".
As for it's popularity, well I read Tolkien as an adolescent but I did not read any other fantasy at the time - unless you count the Hitchikers series. Although I found escaping from the real world a relief none of the other fantasy books I looked at appealed becasue they seemed poor imitations of Tolkien and lacking the internal coherence of Middle Earth. However in adulthood I have read some very good books which can be classed as fantasy. This in itself may be a reason for the popularity of the genre. JK Rowling spins an enthralling yarn, Philip Pullman's Northern Lights is one of the best books I have ever read of any genre although one of the darkest, and Tery Pratchett is always a joy.
They are all writers who, like Tolkien are genuinely enjoyed by both adults and adolescents ( and bless them all for providing genuine common ground between me and my teenage cousins when they were at their most uncommunicative
).
Their particular popularity with teen-agers may partly to do witht he fact that teenagers are not always well served by other genres. It is very easy for adults writing for adolsecents to seem patronising but books written for adults may have little interest. An adolescent may well have superior intelligence and literacy skills to the average adult but the stuff of the full adult world may have little appeal as subject matter. Shakespeare's words in "The Tempest" are very true : Miranda sees a "Brave new world" but her father counters with the put down "T'is new to thee". A fantasy world is new to everyone. This also levels the playing field for authors too perhaps. I have just read Eragon which Christopher Paolini wrote aged 16 and really enjoyed it despite my prejudice against books that seem to rip off Tolkien.
Escapism also has to be a factor. Adolescence is difficult and I don't think it is getting easier. In Britain, teenagers are under constant pressure from exams and while in some ways are much more sophisticated at an earlier age in others, they have less freedom. Some of the central characters in fantasy works are adolescent : Lyra, Will, Harry Potter, various in Pratchett. In LOTR you could see the hobbits - especially Pippin- as adolescent. They have come from a sheltered world and go out into a wider world where they have little or no experience. They have the raw materials of their intelligence and courage and profit from guidance in some areas and learn the hard way in others - and they return in triumph. Think of Gandalf's words about them growing high in the world and the Scouring of the Shire being the task they were trained for. You could say the story was in part a rite of passage - but then Tolkien claimed to hate allegory!!
NB It took so long for me to write this that this ended up being the third not first response! So sorry for any repetition ...