For me I'd have to be contrary and say both.
But sometimes I think the thrill of the story wears off after many readings, as it doesn't necessarily surprise as much - though with Tolkien I
always notice something new each time I read his work. If the World did not have the story then I don't think I could have enjoyed it in the first place, as its story which draws in the majority of people. Without story then it might just be a gazeteer or a travel book with some wonderful descriptions. But then because the World is so immersive, it keeps me reading even long after I have learned the plot. And sometimes, when I have 'lost the plot'.