From Tinkering to Presence: My Journey to Drop-In Reality in AR/VR

whoisjgalt

Perfect timing, here is another post in this forum about v81 and Hyperscape. https://forum.sexlikereal.com/d/10315-photorealistic-room-scan-hyperscape. The future of AR/VR is going to be WILD!

dolphin7559

whoisjgalt I mean, you were talking about the OS and AI as if they are sentient beings, and then about having sex with AI. I can't decide whether it's serious or a troll post.

whoisjgalt

dolphin7559 I never mentioned anything about them being sentient beings. Your interpretation of my post is yours alone. Decide how you will.

SchnuppiLilac

Had a quick chat with the VR gods. Their verdict: VR tips = 👍️ , AI claims = 🦄.

whoisjgalt

SchnuppiLilac Thanks, this actually made me laugh! I don't know man. Is it that far of a reach? All AI by design learns from and adapts to its user. Ever spent much time using ChatGPT, Grok, etc.? I'm sure you have. Plus Meta has publicly spent billions hiring the best talent in the AI space recently, and Meta Connect 2025 was largely about its AI vision and its vision to bring the most immersive video experiences to its Quest platform.

I mean, if you conceptualize the perfect AR/VR device and operating system, it would be one that learns and adapts to the user, making incremental improvements that are tailored specifically to the user of the device. If I was a tech billionaire who was looking to ensure my product line appealed to the masses, this is what I would do. I would use AI to ensure the device operated perfectly, and matched the specific users use case and use preferences. And that it did these things really well, and to some extent, automatically.

I just donned the headset again, about 5 minutes ago. And if this is AI psychosis, as someone above concluded, well sign me up. It is just that damn good. I am literally noticing improvements every time I put on the headset these days. Even hours apart.

And, the Snapdragon chip in the Quest 3, well, it was designed with AI in mind. This is not my opinion, it has been well publicized as an extremely AI capable chipset.

XR2 Gen 2 AI Capabilities

On-Device AI & Machine Learning:
A dedicated, 8x more performant neural processing unit (NPU) handles AI and machine learning tasks directly on the device, leading to faster processing and greater efficiency.

Advanced Tracking:
The improved AI enables more accurate and responsive tracking of the user's head, hands, and controllers, which is crucial for precise navigation and interaction within virtual and mixed realities.

Facial Expression & Depth Estimation:
The chip's advanced AI can also process and interpret facial expressions for more immersive avatars and estimate the depth of the surrounding environment, enhancing mixed reality experiences.

How AI Powers Mixed Reality

Seamless Interaction:
By processing data on-device, the AI reduces latency and improves the responsiveness of interactions, making virtual elements feel more integrated with the physical world.

Enhanced Passthrough:
The integrated AI works with the increased camera support to deliver lower-latency, full-color video pass-through, allowing for more realistic and immersive mixed reality experiences.

Hardware Acceleration:
Dedicated hardware acceleration for tasks like positional tracking and passthrough reduces the load on the CPU and GPU, freeing up resources for graphics and other complex operations.

So tell me, why would Zuck harness this capability? I think he is just now starting to unleash it!

SchnuppiLilac

whoisjgalt Ever spent much time using ChatGPT

I have to confess that the VR gods don't even know that I exist. I copied your initial post into ChatGPT, asked for their opinion and posted the result.

I used to be on the PTC too. One day Chewbacca started talking to me through porn stars. Probably everyone here thought I've gone crazy. Until one month later Chewbacca started talking to them as well...

dolphin7559

whoisjgalt I would use AI to ensure the device operated perfectly

The Zuck has got you covered

renstimpy11

You could have saved yourself some effort by dropping a 20mg thc gummy 1hr before session start ... or is that just the next level for you to experience 😅

whoisjgalt

renstimpy11 I’ve been a long time advocate for THC and AR/VR, really adds a new layer of immersion. I’ve found that the passthrough quality is so improved with v81 that I am getting similar results even without THC. That said, I do still consume it often in combination with AR/VR. It’s been interesting to compare with and without.

petermc

Thanks for taking the time to get all this down to help others but that’s far too much work for me to watch a porno.

I’m pretty happy being transported to another space rather than my own where I can move the camera around and the room moves with me. Until passthrough fixes that issue, it’s never going to be anymore than a minor occasional curiosity for me.

whoisjgalt

petermc I agree. Passthrough has always been difficult to become immersed in or feel “real.” Things wouldn’t ever quite line up right, the colors never matched my environment right, the girl always looked like a flat cutout displayed in front of my real space. I got pretty good at adjusting Heresphere settings to make it “work,” but I could never get it perfect, and taking the time to do that would often ruin a session.

Now that I am on v81, it seems to me that the OS is doing all of those adjustments for me. To the point that I deleted 1,000s of hours worth of saved .hsp files and keyframes.

Today as an example, I simply turned on the headset, opened Heresphere, pressed play on the video, recentered my position and orientation, and like magic, everything aligned perfectly within my space and relative to my body position. Not only that but the image no longer looks flat and cut out, but has true 3 dimensions, and feels like the girl is actually taking up space in my room. The colors have been automatically adapting to the lighting in my space as well, even as the light changes over time in a session.

Long story short, my hypothesis is that the v81 OS is beginning to better utilize the very AI capable chipset. I think that the OS is using AI to analyze the insane amounts of data it’s collected through my use patterns to optimize passthrough video for me. It’s learning and adapting to me.

And this past month has been nothing short of amazing. Every time I put in the headset my passthrough experience is better than the last.

I think this is the real future, OS and AI optimizing for the end user. To test all of this, I went back and played old passthrough videos that I deemed completely unwatchable. Scenes I couldn’t fix using Heresphere. Those scenes that I previously couldn’t watch, were now watchable. And in many cases, they were amazing.

This was happening to the videos saved on my external hard drive via xbvr, so my observations are not from studio upscaling. The improvements were happening locally on my quest 3.

I think we are headed towards vast improvements in passthrough quality simply from OS updates and AI machine learning. And I think it’s going to happen fast. But only for those who update their OS. I hear stories of people disabling auto updates, etc.

Just my experiences and my prediction. I could be wrong and I’m open to that. Just trying to document what I am experiencing in some way.

dolphin7559

whoisjgalt Nothing in the update notes about it

Picouser

whoisjgalt Now that I am on v81, it seems to me that the OS is doing all of those adjustments for me. To the point that I deleted 1,000s of hours worth of saved .hsp files and keyframes.

I wish this is true but I think you are just trolling. Like others have said there is zero evidence v81 has this kind of feature.

renstimpy11

whoisjgalt so you think v81 has enabled some type of real-time AI image processing making Heresphere scenes look better? You also think the PT lighting is changing in real-time? Do you think it works for all apps like DeoVR also?
I did just update to v81 (Meta Horizon phone app>Q3 settings>advanced>public test) and I am pretty sure there is a visual improvement for Heresphere but its not night and day, very subtle imo.
I think performance was better also... I do use Quest Game Tuner though.

whoisjgalt

Picouser just trying to engage in conversation about something I’m excited about. I have nothing to gain from trolling. And, I appreciate the skepticism from the group. I’m skeptical myself, and still trying to wrap my mind around what I’m seeing. Could be my brain is finally giving in and becoming more accepting of the medium, or I’m becoming delusional.

But I also know that Meta is far from transparent, and that there are perhaps thousands of different v81 builds out there. That is how the PTC works. Meta tests out new things with PTC participants.

Whatever is happening, I’m enjoying it! And I hope that what I am personally experiencing are signs of where the tech is headed. Time will tell if more people start reporting similar experiences.

whoisjgalt

renstimpy11 yes and no. I'm not sure if it is real time AI image processing, but I think that v81 is allowing AI to make depth-aware passthrough refinements based on the volumes of historical data its likely collected about my environment from the front facing cameras. I also think it is doing this with color balance, and possibly scene anchors. I have historical use patterns of the quest 3 around my house. In those familiar use locations, the passthrough quality on v81 has been far better than if I "stress test" it, and move to a brand new location.

I think with this historical data, and the new data I provide with every use, AI is likely working to make local device updates via the chipset, and also Meta is using the data more globally to push new OS updates.

Interestingly, like you, I barely noticed any difference immediately after upgrading to v81. It was very subtle at first, but the improvements picked up rapidly with use. I've also now had 4 new v81 builds pushed to my headset over the span of about 3 weeks. This feels abnormally fast to me, even on the PTC, and the quality has improved significantly with each new build push.

I have only used heresphere since updating to V81. I only use Deo when traveling (to stream). I would imagine it would work with any video player, and might even work best with whichever video player you use most. But, again, I really don't know, these are all my predictions based on my own experience.

I really believe this is just the beginning, and we will probably see improvements in passthrough latency, smarter auto calibration of 3D video and passthrough video to match the users environment (kind of eluded to this at Connect 25), better dynamic relighting in passthrough, and maybe context-aware blending. My prediction is that we will see all of these before any new hardware is released, and that with AI, there is probably another 15-25% headroom left to gain just from OS updates alone.

dolphin7559

whoisjgalt So how do you know you're not imagining these supposed improvements? I haven't seen you present any evidence that supports what you're claiming. There also isn't any reason for Horizon OS to be tampering with 8K videos watched in a 3rd party app either. These mobile chipsets don't have the performance headroom for that. Camera feed I could believe, videos absolutely not.

SchnuppiLilac

whoisjgalt the OS and AI were "optimizing settings for me." Gamma, saturation, stereo roll, even opacity multiplier — all began shifting automatically

dolphin7559 Heresphere settings being adjusted by the OS would be something that you can see and capture. There would be different settings for every video for the chosen ones, while it's always the same default settings for everyone else. I don't know if it is even possible for the OS to modify settings in 3rd party apps. But for sure Meta didn't develop some secret super AI for moving sliders in an app called Heresphere for a few people watching alpha PT porn. That’s why I don’t buy any part of the story that takes place outside OP's brain.

whoisjgalt

SchnuppiLilac I’m not suggesting the OS is doing anything to heresphere. Rather, I think AI is improving the OS vision model with things like dynamic relighting, improved depth mapping, video reproduction and scaling, and anchoring. All things that can be improved on device given the current tech stack. These improvements make the image appear more “correct” or “real” in my space, so I no longer need to chase manual setting changes in heresphere. All my heresphere settings are at default, outside of a few (opacity multiplier, aspect ratio, resolution). And no, the OS does not change my settings in Heresphere.

As an example of what it is doing, today my room was on the darker side. The passthrough feed was noticeably brighter than my natural vision without the headset on. This made the passthrough feed better match the studio lighting in the video. This is an example of dynamic relighting that made it so I didn’t need to adjust image settings in Heresphere. Another change I noticed today was the borders of the actors appeared exceptionally crisp and perfectly clean. Before with opacity multiplier set to low there was feathering observed. To high and there was a noticeable grey border around the actors appeared exceptionally. Now, it is damn near perfect, even around the hair, no grey room spillover.

It’s cool if no one believes me. I’ll keep having a blast seeing how it improves (pretty much daily at this point). But I do hope someone else on v81 starts noticing similar things soon. The whole AR/VR community will benefit from this.

The past 4 weeks have been like my first day of VR all over again. Every time I put on the headset and watch passthrough, I’m wowed by how crazy good it is. Before, 2 or 3 consecutive days of heavy use would put me in a place where I saw all the flaws and would need to take a break. Not anymore, I can’t get enough…haha😰

dolphin7559

whoisjgalt So based off that you seem to be referring to just passthrough?

alphaRonaldo

whoisjgalt Any screenshots you could share?

whoisjgalt

Sorry, in my last post I meant to say video projection, not video reproduction.

I’m pretty sure the improvements I’m noticing would apply to any video player (Deo, Playa, etc.). I just happen to almost exclusively use heresphere and haven’t had a chance yet to try any others. To busy staring at the insanely hot women filling up my room!

Next Page »