Ok, that is a legitimate argument, although you must admit there is a difference between disappearing from an open field and appearing at a rail. In one case it's just a step away from not being seen in the other many.
But yeah, the astonishment of those present does remind of the old man's disappearence, it does all in all point towards Saruman as the culprit that night.
But I still can't believe he would leave so quickly...
