• Content

  • Passthrough production feedback and improvements thread

On another note and something I think would be especially interesting in PT, would it be possible to get an extra channel with the video stream that has depth data? It could be encoded in the alpha mask red parts, or however it works with the separate mask data used with the AI PT.

Having depth data opens up a whole range of possibilities from depth clipping your virtual hands (which unfortunately are not enabled for pico4 in deovr) with the model to more realistic perspective re-projection on head movements where you have actual perspective changes instead of the 2d warping what we now have with the 6DOF. And more.

The hands clipping would add a big level of immersion, I have implemented this in a VR viewer for Gaussian splatting where I have some scenes with human models and I was amazed by the immersion it provides. Of course there is no haptic feedback, but the visual feedback alone tricks me into a sensation of touching.

As for how to get the depth (estimation) data, I don't think this can be done in realtime on the device itself. But if preprocessing like the AI image segmentation for PT masks infra can be leveraged, then there are options like https://depth-anything.github.io/ which I believe is the current state of the art for monocular depth estimation. Of course here we have stereo and we can use that extra information and then I believe state of the art is https://haofeixu.github.io/unimatch/ although I haven't really been following the progress that closely.

    YKBnqkvq
    Yes, we have experimented with depthmaps a lot. As for depth the more precise estimators arise from stereoscopic estimators, rather then monocular. And definitely that may be useful. But during testing we found that even on precomputed depth the 6dof reconstruction looks bad and have too much artifacts for comfortable watching (you can see that kind of artifacts on lifecast demos for example, which uses same idea). At closeups the dephmap-based 6dof is becoming nearly unwatchable... But still we are keeping watch over video inpainting, splatinngs and other volumetric videos progress.
    Another problem is streaming - 8 bit is not enough for accurate depth representation and most of codecs loose some info during compression which is ok for video, but not for depthmap.
    As for on-device calculations, we found it possible to calculate low-quality (128x128) depthmap on device, which is potentially enough for occlusions both in room objects for AR (Quest only yet) and hands in VR

        AlexSLRLabs Another problem is streaming - 8 bit is not enough for accurate depth representation and most of codecs loose some info during compression which is ok for video, but not for depthmap.

        You don't have to encode that information into the video. There are better ways.

            AlexSLRLabs Awesome, great to hear that this is an area of active experimentation! Thanks for the detailed response, much appreciated.

            Makes sense that you need high precision depth to make it useful, especially at close distances. There is a reason depth maps typically represented by the reciprocal of the distance from the camera. I hadn't realized that you'd need a 16 bit channel and the video codecs are optimized for color perception. Not having personally experimented with depth estimations on "in the wild" video before, I also wonder how much effort would be needed to for calibration to eventually end up with reasonable accurate world coordinates. I can imagine its all a lot more involved that it might seem! Although those SfM techniques to a remarkably good job at it. Sounds like a fun challenge I'll pick up when current side projects allow 🙂

            I have not looked into lifecast enough yet, this is a good reminder to check it out, thanks.

            While thinking about this, normal information would be cool too, combined with the PT color information to apply lighting from the real surroundings to the subjects. I'm excited to see where all this goes in a few years with new developments and more capable hardware.

              Manny_S I checked out the sample scene and as I expected the 45 degree variant is more comfortable to watch than the 90 degree one.

              I like the animation to improve UX explaining the viewer has to turn their head. On this topic, one thing I noticed in this sbvr video (https://www.sexlikereal.com/scenes/a-special-visitor-40250?t=2241) is that they chose to rotate the scene while keeping the "virtual" camera horizontal. I like this idea because it prevents the initial double vision and puts the onus on me to rotate the scene, which I am used to now anyway because of the experience with aligning a scene in PT.

                  I don't really understand the current obsession with PT. I pay extra for scripts, not PT. Is this much effort going into the script AI too?

                  7 days later

                  I don't know if it is just me but it appeared to me that the latest SLRO AR PT scenes had some scaling/size ratio issues along the depth axis. I have complained a few times about the AR PT videos of VRB/VRPorn in which objects/performers get increasingly larger the farther they are away from the viewer. In the SLRO AR PT videos, it's just the other way round, i.e. objects seem to get increasingly smaller with distance (smaller than objects in reality is what I am trying to say). While I can perfectly align the performer's dick with my own in a reclined BJ position, I am having a hard time ignoring the different size ratios at that distance (positions closer to the viewer are fine).

                  Wondering whether this is something that happens already during filming or later during stitching/post-production, and whether it would be possible to change this.

                    SchnuppiLilac I noticed this too. If the girl stands up and moves to what should be the end of the bed, she is tiny

                      doublevr Nooooo, now what?
                      Who is going to shoot the regular scenes?
                      Or are you just giving up on then?

                        doublevr do you know when the scene will be released? can not wait to see it

                          There is what, 30 VR vids of Melody already, and people are complaining about a first AR PT vid? GTFO.

                            Metamucil I couldn't care if Melody gets a dozen PT scenes. She has a mountain of fantastic VR scenes already. It's about losing Alex Nash to PT where there is no room for creativity and all scenes are carbon copies of each other. I would have no interest in SLR Originals if he stopped producing his VR scenes.