From Tinkering to Presence: My Journey to Drop-In Reality in AR/VR

dolphin7559

whoisjgalt I would use AI to ensure the device operated perfectly

The Zuck has got you covered

renstimpy11

You could have saved yourself some effort by dropping a 20mg thc gummy 1hr before session start ... or is that just the next level for you to experience 😅

whoisjgalt

renstimpy11 I’ve been a long time advocate for THC and AR/VR, really adds a new layer of immersion. I’ve found that the passthrough quality is so improved with v81 that I am getting similar results even without THC. That said, I do still consume it often in combination with AR/VR. It’s been interesting to compare with and without.

petermc

Thanks for taking the time to get all this down to help others but that’s far too much work for me to watch a porno.

I’m pretty happy being transported to another space rather than my own where I can move the camera around and the room moves with me. Until passthrough fixes that issue, it’s never going to be anymore than a minor occasional curiosity for me.

whoisjgalt

petermc I agree. Passthrough has always been difficult to become immersed in or feel “real.” Things wouldn’t ever quite line up right, the colors never matched my environment right, the girl always looked like a flat cutout displayed in front of my real space. I got pretty good at adjusting Heresphere settings to make it “work,” but I could never get it perfect, and taking the time to do that would often ruin a session.

Now that I am on v81, it seems to me that the OS is doing all of those adjustments for me. To the point that I deleted 1,000s of hours worth of saved .hsp files and keyframes.

Today as an example, I simply turned on the headset, opened Heresphere, pressed play on the video, recentered my position and orientation, and like magic, everything aligned perfectly within my space and relative to my body position. Not only that but the image no longer looks flat and cut out, but has true 3 dimensions, and feels like the girl is actually taking up space in my room. The colors have been automatically adapting to the lighting in my space as well, even as the light changes over time in a session.

Long story short, my hypothesis is that the v81 OS is beginning to better utilize the very AI capable chipset. I think that the OS is using AI to analyze the insane amounts of data it’s collected through my use patterns to optimize passthrough video for me. It’s learning and adapting to me.

And this past month has been nothing short of amazing. Every time I put in the headset my passthrough experience is better than the last.

I think this is the real future, OS and AI optimizing for the end user. To test all of this, I went back and played old passthrough videos that I deemed completely unwatchable. Scenes I couldn’t fix using Heresphere. Those scenes that I previously couldn’t watch, were now watchable. And in many cases, they were amazing.

This was happening to the videos saved on my external hard drive via xbvr, so my observations are not from studio upscaling. The improvements were happening locally on my quest 3.

I think we are headed towards vast improvements in passthrough quality simply from OS updates and AI machine learning. And I think it’s going to happen fast. But only for those who update their OS. I hear stories of people disabling auto updates, etc.

Just my experiences and my prediction. I could be wrong and I’m open to that. Just trying to document what I am experiencing in some way.

dolphin7559

whoisjgalt Nothing in the update notes about it

Picouser

whoisjgalt Now that I am on v81, it seems to me that the OS is doing all of those adjustments for me. To the point that I deleted 1,000s of hours worth of saved .hsp files and keyframes.

I wish this is true but I think you are just trolling. Like others have said there is zero evidence v81 has this kind of feature.

renstimpy11

whoisjgalt so you think v81 has enabled some type of real-time AI image processing making Heresphere scenes look better? You also think the PT lighting is changing in real-time? Do you think it works for all apps like DeoVR also?
I did just update to v81 (Meta Horizon phone app>Q3 settings>advanced>public test) and I am pretty sure there is a visual improvement for Heresphere but its not night and day, very subtle imo.
I think performance was better also... I do use Quest Game Tuner though.

whoisjgalt

Picouser just trying to engage in conversation about something I’m excited about. I have nothing to gain from trolling. And, I appreciate the skepticism from the group. I’m skeptical myself, and still trying to wrap my mind around what I’m seeing. Could be my brain is finally giving in and becoming more accepting of the medium, or I’m becoming delusional.

But I also know that Meta is far from transparent, and that there are perhaps thousands of different v81 builds out there. That is how the PTC works. Meta tests out new things with PTC participants.

Whatever is happening, I’m enjoying it! And I hope that what I am personally experiencing are signs of where the tech is headed. Time will tell if more people start reporting similar experiences.

whoisjgalt

renstimpy11 yes and no. I'm not sure if it is real time AI image processing, but I think that v81 is allowing AI to make depth-aware passthrough refinements based on the volumes of historical data its likely collected about my environment from the front facing cameras. I also think it is doing this with color balance, and possibly scene anchors. I have historical use patterns of the quest 3 around my house. In those familiar use locations, the passthrough quality on v81 has been far better than if I "stress test" it, and move to a brand new location.

I think with this historical data, and the new data I provide with every use, AI is likely working to make local device updates via the chipset, and also Meta is using the data more globally to push new OS updates.

Interestingly, like you, I barely noticed any difference immediately after upgrading to v81. It was very subtle at first, but the improvements picked up rapidly with use. I've also now had 4 new v81 builds pushed to my headset over the span of about 3 weeks. This feels abnormally fast to me, even on the PTC, and the quality has improved significantly with each new build push.

I have only used heresphere since updating to V81. I only use Deo when traveling (to stream). I would imagine it would work with any video player, and might even work best with whichever video player you use most. But, again, I really don't know, these are all my predictions based on my own experience.

I really believe this is just the beginning, and we will probably see improvements in passthrough latency, smarter auto calibration of 3D video and passthrough video to match the users environment (kind of eluded to this at Connect 25), better dynamic relighting in passthrough, and maybe context-aware blending. My prediction is that we will see all of these before any new hardware is released, and that with AI, there is probably another 15-25% headroom left to gain just from OS updates alone.

dolphin7559

whoisjgalt So how do you know you're not imagining these supposed improvements? I haven't seen you present any evidence that supports what you're claiming. There also isn't any reason for Horizon OS to be tampering with 8K videos watched in a 3rd party app either. These mobile chipsets don't have the performance headroom for that. Camera feed I could believe, videos absolutely not.

SchnuppiLilac

whoisjgalt the OS and AI were "optimizing settings for me." Gamma, saturation, stereo roll, even opacity multiplier — all began shifting automatically

dolphin7559 Heresphere settings being adjusted by the OS would be something that you can see and capture. There would be different settings for every video for the chosen ones, while it's always the same default settings for everyone else. I don't know if it is even possible for the OS to modify settings in 3rd party apps. But for sure Meta didn't develop some secret super AI for moving sliders in an app called Heresphere for a few people watching alpha PT porn. That’s why I don’t buy any part of the story that takes place outside OP's brain.

whoisjgalt

SchnuppiLilac I’m not suggesting the OS is doing anything to heresphere. Rather, I think AI is improving the OS vision model with things like dynamic relighting, improved depth mapping, video reproduction and scaling, and anchoring. All things that can be improved on device given the current tech stack. These improvements make the image appear more “correct” or “real” in my space, so I no longer need to chase manual setting changes in heresphere. All my heresphere settings are at default, outside of a few (opacity multiplier, aspect ratio, resolution). And no, the OS does not change my settings in Heresphere.

As an example of what it is doing, today my room was on the darker side. The passthrough feed was noticeably brighter than my natural vision without the headset on. This made the passthrough feed better match the studio lighting in the video. This is an example of dynamic relighting that made it so I didn’t need to adjust image settings in Heresphere. Another change I noticed today was the borders of the actors appeared exceptionally crisp and perfectly clean. Before with opacity multiplier set to low there was feathering observed. To high and there was a noticeable grey border around the actors appeared exceptionally. Now, it is damn near perfect, even around the hair, no grey room spillover.

It’s cool if no one believes me. I’ll keep having a blast seeing how it improves (pretty much daily at this point). But I do hope someone else on v81 starts noticing similar things soon. The whole AR/VR community will benefit from this.

The past 4 weeks have been like my first day of VR all over again. Every time I put on the headset and watch passthrough, I’m wowed by how crazy good it is. Before, 2 or 3 consecutive days of heavy use would put me in a place where I saw all the flaws and would need to take a break. Not anymore, I can’t get enough…haha😰

dolphin7559

whoisjgalt So based off that you seem to be referring to just passthrough?

alphaRonaldo

whoisjgalt Any screenshots you could share?

whoisjgalt

Sorry, in my last post I meant to say video projection, not video reproduction.

I’m pretty sure the improvements I’m noticing would apply to any video player (Deo, Playa, etc.). I just happen to almost exclusively use heresphere and haven’t had a chance yet to try any others. To busy staring at the insanely hot women filling up my room!

whoisjgalt

dolphin7559 I've seen improvements in VR as well. I mostly watch passthrough these days, and, from an immersion perspective, there was more room for improvement with passthrough versus VR (alignment issues, color grading, etc.), so that is where I've personally noticed a greater improvement. But in VR, the depth is better and presence/immersion feels better.

whoisjgalt

alphaRonaldo I've been hesitant to post pictures or video, you know, having pictures/video of naked girls in my room with things that could potentially identify me, out in internet world kind of freaks me out. But, I'll think about it.

whoisjgalt

alphaRonaldo I did take some screen shots and video. They turned out scary good. Like, so good that it was kinda uncanny valley looking at them on my desktop and thinking it really looks like there was a girl in my space. With how good some turned out, it would be hard to convince someone they weren’t real. At least someone who is unfamiliar with passthrough AR. I still noticed a very few artifacts at certain points, but that’s because at this point I know what to look for. Someone casually watching the clips or looking at the pictures probably wouldn’t pick up on them.

While I want to share them, because they are so good, I also don’t want them floating around the internet🤣.

It wasn’t a complete waste of time, they are so good it is a new layer of immersion. Almost like capturing a moment in time to return to. It was really fun to go back and watch them outside the headset. Kinda trippy, in a good way. I think I’ll continue to take screen shots and video for my own personal enjoyment. Damn this shit is getting good!

eyelust

whoisjgalt Where in side quest do we adjust these settings: “SideQuest let me unlock Quest 3’s raw horsepower: set to Quest 3 resolution, CPU/GPU level 4, 120 Hz refresh rate, and fixed foveated rendering off”. Is there an app we use or is it in the side quest settings?

dolphin7559

eyelust Quest Game Tuner, Quest Game Optimizer, or adb commands

whoisjgalt

eyelust I use the SideQuest app on my laptop. Plug in a USB-C cable from the quest to the laptop. In Sidequest, on the top right of the screen, you will see a small wrench icon. Click the wrench icon, and you will find these settings.

« Previous Page