In fact, it's the opposite. Using keyboard commands for movement is more distracting (and measurably slower) than using the mouse. It is precisely that the mouse is less distracting that tricks the mind into thinking the keyboard is faster. See AskTog:
[If you think this is incorrect, providing links to scientific evidence is more productive than downvoting or stating that your own subjective experience is different.]
You're not reading some of the most cogent arguments here: most vim/Emacs/etc power users couldn't even tell you what keyboard command to use precisely because they don't think about it. It's reflex, muscle memory, completely out of mind so that it doesn't get in the way of getting code from your mind to the machine. Maybe it's that way for some mousers as well, but given that code is text, it's kind of doubtful. There's also the fact that if you have to break your concentration to context switch to the mouse, you will lose your flow.
Try coding in an editor, any editor, for 8, 9, 12 hours a day and see if you don't start forgetting the keyboard shortcuts because they become reflexive. Vi and Emacs are just hyperdesigned to enhance this effect.
EDIT: And I don't care that you link to Bruce Tognazzini, a GUI designer, when millennia of musicians have known that they don't think about what combinations of fingers they press to get an F#, they just play a glissando with an F# in the middle.
The thinking is subconscious: you don't know you're doing it and you "forget" how long it takes, which is why you perceive the keyboard as faster. If you bother to read the research I've linked to, it covers all of your objections. I know it's difficult to let go of subjective impressions, but the stopwatch is always right compared to our own internal sense of time.
It's not an appeal to authority: that would be if I were appealing to the authority for the authority's sake; but I am appealing to scientific research presented by the authority. I'm fairly certain you'd agree that appealing to scientific evidence is valid, as you've tried to present it to me as part of your argument. I think the downvotes are because you running some kind of "experiment" on your own is different from a human interface researcher's actual research.
Fair point, however the choice of editor/editing mode is a personal one. Are my personal results of my personal experiment not the only ones that matter to my personal choice?
http://www.asktog.com/TOI/toi06KeyboardVMouse1.html
http://www.asktog.com/TOI/toi22KeyboardVMouse2.html
http://www.asktog.com/SunWorldColumns/S02KeyboardVMouse3.htm...
[If you think this is incorrect, providing links to scientific evidence is more productive than downvoting or stating that your own subjective experience is different.]