Hairsational I think that is not very forward thinking, unless you mean the near future with "ever". Just imagine what will be possible in the future with light fields, AI, increased processing power, etc.
At some point you won't have to fiddle at all. Hell, maybe we won't even use the conventional video format in 5 years but something more interactive. Where you can decide if you have your own body in VR or AR, switch between AR and VR or a mix and match elements, interact with the girl in some capacity, the player understanding your surroundings and doing smart things with shadows and occlusion, etc.
Now, I agree that I don't see AR completely replace VR and definitely not short term (they fill a slightly different need after all). I do think the lines between AR and VR will blur more and more moving in the future. We won't get there right away but it will get more and more fun as we move along. As for this moment, I'm already very excited about what it brings to the table though.