"I can't be always carrying burglars on my back," said Dori "down tunnels and up trees! What do you think I am? A porter?"
The Hobbit, Chapter 6. It's when Bilbo, Gandalf and the dwarves are trying to take refuge up a tree from the Wargs, and Nori told Dori that he'd left Bilbo behind again.
