I mean VR deepfakes have been a thing for a very long time... Although the quality does vary, but it's definitely getting there.
https://mrdeepfakes.com/videos/virtual-reality
You can face swap a non-vr videos using deepswap.ai
Although the quality of the video isn't all that great given that it is simply an image that they are referencing.
You can try to generate your own with DeepFaceLab, it's an open sourced tool that is capable of achieving some pretty impressive results. You do need a higher end GPU to achieve decent training speeds however.
I think the biggest bottleneck currently to generate a quality piece of VR deepfake with a custom face as a service is computing power. It's not as simple as stick a face on a VR video, and expect it to work flawlessly. I definitely think this is achievable in the near future though, just need the technology to mature a couple of more years at the slowest.
On the subject of ethics, personally speaking I don't really see an issue of people creating deepfakes for private use.. I don't see that being any different than fantasizing about being with someone, which everyone has done at one point or another. I think the ethics comes in play only when the creator starts to distribute these deepfakes without consent, and that's what really should be monitored and prohibited.