He said it himself: that exactly when he needed the time (in the great years) he had none. Clearly implying that he had loads of time before then, strolling around the Shire and whatnot. Would it have been too much to make one quick trip out east?
Say what you want, I think the boy Incanus is too lazy and willing to rely on his unreliable team-mates.
