I see Meta are releasing an object occlusion for their passthrough. I was wondering if it would be possible to see your hands (etc) in front of the video when using the passthrough mode? The immersion levels will go through the roof if the model doesn't obscure your hands at least. Hope that makes sense.

It should literally be the number 1 focus! Be good to hear from someone with a vision pro where this will look amazing. But hopefully we can at least get metas rough version in some way. Everyone should be asking about this!

    It would be really cool but it will require the video to have depth information to work. Probably will need to be generated with AI if it's to work on all videos and then streamed along with the video as real time depth estimation for the entire video is not feasible in real time at the moment on a quest.

    So, not a small request but I agree, definitely a worthwhile venture. Especially as that depth information can also be used to create proper auto focus.

      4 days later
      doublevr changed the title to Occlusion Passthrough Update .
      a month later

      I saw a review of 'stranger things' on quest. It has a mixed reality section, in the video you could see his actual hands over VR assets, seems like it basically just used the hand tracking for position to cut out the hands. We must be close to this...... Surely! It's next level stuff.

        Quick research conclusion:

        • We might use dynamic occlusions to render video behind user's hands
        • It cannot be used to move your hand in the space of the video (let's say you want to move your hand behind one girl, in front of another) - that requires 3D objects, not stereo image. Requires deep research to figure out such usage. We might come back to this idea later.

        Integration:

        • Requires heavy modification of the image rendering
        • Most likely will affect performance (FPS drop, can't predict how much)
        • Makes more sense for 3D objects and volumetric scenes with "real" depth

        We will try to make simple integration with hands in front of the rendering mesh first and see where it can lead us 🙂
        Stay tuned!

        So how will that work while fapping, the occlusion should be an option, turn on turn off. Toggle switch.

          7 days later

          Check this out. This is what can be done (potentially).
          Let's see if we have enough interest to implement it.

          Keep in mind:

          • Rendering performance drops
          • It's not really easy to improve quality (flickering on the edges, etc) without losing more performance
          • This only works on Q3 at the moment

          How cool is that?

          I want it97%
          Nah3%

            Nice, already looks cool! Curious to try it out. It might not be perfect yet but tech has to start somewhere, meta will keep improving the algorithm, hardware will keep getting better, etc. So, it's a yes for me. Make it toggable though and also assignable to a hot key for quick switching.

            iddqd When can we expect this update to drop? It looks very promising

              Brilliant, I knew it would be a little janky due to the way it has to be done on Quest but it's there! Obviously it would look perfect on Vision Pro but who has one of those 😂 but it's worth implementing now as the next Quest (not Q3s) will have this nailed like on vision pro. Can't wait for it to go live! Great job!
              P.s. You gave it a worst case scenario there, fingers wide open is what it struggles with so it will look better if a person is not trying to break it by doing things you wouldn't naturally do.

                Repaelleinad Brilliant, I knew it would be a little janky due to the way it has to be done on Quest

                Of course it would be, Everything in VR is essentially in a beta state "janky". I'd rather have something now and see it improve over time than wait another decade for perfection, just because a few people say, 'Why release this now if it’s not ready?' It's VR, updates and improvements are part of the process.Even AVP doesn't look super refined in terms of occlusion yet, but that's still in beta too.I'm hoping there's an option to toggle these features on or off. It's always good to have options."

                  6DOF is natively PT. It’s like perfect

                  SexisAmazing You do realize that that green tint around the model is most likely going to stay there? It's interesting because on the Apple Vision Pro, developers dont have access to the cameras, so there's no 'native passthrough', but it works with a 'work around' in heresphere via ALVR. However there's the exact same green tint around the model as a result, which you see here above too. So it seems they're doing something similar to achieve this.

                  So, it's going to be cool, but also not exactly 'perfect' at least not right away. But maybe it's fixable.

                    Thanks guys for your feedback. Didn't expect so much interest!
                    Unfortunately this feature has lots of conflicts with our playback and rendering logic.
                    Can't promise releasing any time soon, but we might get back to that a bit later.

                    That'll give PT scenes a huge boost in immersion!

                    Now that Meta announced the Passthrough API, can we get a "passthrough matching AI" feature? I always thought it would be cool if passthrough videos' image settings automatically changed based on my room's lighting. However, Meta didn't offer devs access to the passthrough feed, so it would've been impossible until now. It would be especially useful at night, since my warm room lights don't match the bright white studio lights of the recording sets. Manually changing the DeoVR image settings can help make the contrast less noticeable, but it's not enough, and it's time consuming

                      Yes, now we can (probably) finally color correct the passthrough feed.

                      lolitovtb we added passthrough layer opacity slider, so you could set it manually. Not yet automatically, but usefull anyway

                        a month later

                        Bro if I had a 12inch I would hope this occlusion works for my 12 incher as well so I can whirl it around in front of these many hotties

                        Bro I was watching this scene here
                        https://www.sexlikereal.com/scenes/cuddle-with-stepdadagain-49132?t=144

                        Great for pt BTW, but she sitting in front of me on the edge of the bed and I just thinking damn with an occlusion cock! Yeah bro we gonna need more then hands occluded in virtual porn bro.

                        Damn bro women could use vr too, they can take out the men's body and strap up, will the dildo be occluded too?

                        Bro will the whole body be occluded? That be crazy to see in regular vp, the whole body blocking the male model body. How will that work? I'm sure there will be flaws but thats everything in vr.

                        Hey bro, we need more pov women fucking each other with strap on, with top hotties starts bro!

                        And fucking needs to be hardcore, I seen this female with strap on bro, omg literally smashing like a guy. I thought it was guy, cus she had no hair, but bro the way she was fucking the female...I thought damn why porn hotties in vr don't fuck like this with strap on? Most time it's kinda less hardcore-ish.

                        Idk 🤷‍♂️...but...just sum thoughts I had about the whole occlusion thing.