Yeah, everything you said is true. There's a spectrum of comfort / ease in VR that goes something like this...
- room scale VR where you always have 1:1 lock between physical and virtual movement - almost nobody gets sick from this
- teleportation and other gimmicky VR locomotion tricks - pretty easy
- sitting in a cockpit piloting a vehicle - varies depending on the moves you pull, dogfights can be tiring
- smooth locomotion (i.e. moving forward with the W of WASD, or a trackpad/thumbstick) - totally doable if you reprogram your brain to handle it
- smooth head turning and artificial camera control in e.g. cutscenes - produces immediate and severe suffering for most humans, not recommended
- slowly rolling backwards at a stop sign in a truck in Euro Truck Simulator 2 - the ultimate grueling VR challenge!
The bottom line is that you need to feel in control of the link between your physical head movement + orientation changes and the corresponding effect on the virtual camera. Sitting in a cockpit with nearby visual references generally helps with that but there are limits.