"Also, these crazy situations you guys talk about where the lack of a use key will “screw” you, it’s just not true"
Yes, it's true. Perhaps not exactly those mentioned, but other ones are sure to arise. Modes are a bad thing (that was a basic tenet back in user interface classes when I studied computer science back in the eighties).
Making interfaces simple to learn and use is a good thing. But the primary aspect of that is to make sure it does what the user expects at all times.
Anyone remember how you occasionally got stuck on ladders in Half-life because you stumbled into one and the game decided you were going to climb it?