• Content

  • AI Upscale proof based discussion

doublevr

None of that is seen in the headset ¯(ツ)/¯

You can when you know where to look. As the benefit is mostly seen in the background it is still very subjective if the upscale is really better, but it is there and in some parts undenieable.
Also the pink lights show less jaggies in the upscaled picture, and they are a huge distraction in an otherwise great video. Unfortunately they are still there, so it is not a huge improvement, but a slight one nonetheless.

I am wondering if and how much it would be better if the upscaling would have been done on the "RAW" material and not an already sharpened final output. This already has some slight sharpening artifacts which usually get worse with upscaling, and also limit how aggressive you can use sharpening during upscaling before it becomes too much.

      ibins we can provide you one to try. just let me know the video and where to send

        spacepirate That's (imho) part of the problem- there are definitely studios who are just doing raw upscaling so instead of 5k they can release "7k/8k". Technically they're not wrong, but their image quality obviously doesn't improve with the increased resolution.

        Topaz VEAI is a very cool application but agree with Rakly3 that it is challenging to actually get worthwhile results. I've been messing around with it for 6+ months in my personal life, and things can look hugely improved w/ a 2d comparative split screen of before/after...but then after spending a week to render 1 scene and putting on the headset to watch what you've made, it often looks either entirely too "artificial" or alternatively it only looks marginally improved from the original (meaning it was not worth the time & effort).

        And the bigger argument to be made imho is that with camera improvements in the past year or so, along with the upcoming 8k cameras- there will be no need for AI upscaling. Already there are studios whose scenes I have not been able to "improve" at all using Topaz, even just with the 2d comparison.

            doublevr Wankz upscale looks absolute garbage to me. Seems like overupscaled. Totally unwatchable.

            LOL, dude. Are you on one of your pointless crusades again? Such exaggeration is ridiculous. At worst, the AI upscales is hardly any different to the original file (despite being substantially bigger), at best, the picture is a bit more clear and less grainy.

            Nobody said that you had to remaster any of your own scenes though. If you don't want to do it, simply don't do it. But maybe you shouldn't base your whole strategy on only your individual vision "aka I can't see it so nobody can see it". That's the best way to ruin VR since visual perception can be wildly different between different persons...

            Rakly3 That is not simply upscaling though. It's the AI interpreting the image according to parameters you set.

            That's also what Wankz is doing...

                  LordCrash wankz remasters are badly stretched for me. It feels lacking a lot and 0 immersion. Every body and object is very unnatural. Totally can't watch.

                  For some reason original videos were fine. No idea how did they manage to break it completely. At least for me.

                  Never understood you guys screaming for such remasters. Not the first time users are totally wrong.

                      spacepirate

                      can't really make sense of many things people are screaming for. glad it works for some of you guys seeing magic where there is none while we are sticking to proof based stuff.

                      once again, the "gains" you see with upscales posted here are not necessarily improving perception. many times the image feel very unnatural with that "AI" crap

                        doublevr

                        wankz remasters are badly stretched for me. It feels lacking a lot and 0 immersion. Every body and object is very unnatural. Totally can't watch.

                        They are bad compared to modern 6k/7k videos, but they are an improvement to their original videos.
                        That said there also sometimes quite huge differences how much they improve. I was pretty blown away how much better their first remaster was, I still have the original and the remastered file, and the difference is huge. (Gina Valentina and Bailey Brooke in Easter Bunnies).

                        The improvement of some newer scenes was quite a bit less, I am wondering if they cut back quality, to push out more remasters, as they said I think that it needs multiple days to upscale just one video.

                        I think the problem is more the expectation. Is an AI Upscaled video better then a video that is natively shot in the higher resolution? Of course not. Is it better then the original, well usually yes. If it is worth all the additional processing, well at least questionable, certainly for new things shooting with better cameras is always the superior solution. But for older things, where you either have the choice of a crappy video or a somewhat less crappy video most will prefer the latter.

                            ibins I was pretty blown away how much better their first remaster was

                            In the gearvr the remastered are not as noticeable, but since I got a quest 2 recently 😏...
                            the remastered is very noticeable 👍

                              10 months later

                              Just met Topaz creators. They confirmed its useless to upscale 4K to 8K. Also it's intended for flat content and significant work is required on their end to make it work with vr because of projection.

                              We arranged they will take one video and do their best to upscale the very center of the video to see if it is even worth doing any further research.

                                2 months later

                                No visual improvements? Are the improvements worth it should be a better question. Without a doubt there are visual improvements. Again, the source material must have good exposure and noise/block levels. Nothing will be able to fix that. If those old really blocky scenes were being upscaled, I can see why they would not work. I will work on getting some samples up for people to compare on their own.

                                Here are a couple of screenshots of an upscale from 4k to 8k that I am working. Would love to hear back from other users to see if they notice any difference. First is the original 4k version, second is the upscaled 8k version, and the third is the upscaled 8k with a little color tweaking for my preference.

                                Original 4k

                                Upscaled to 8k (No color grade)

                                Upscaled to 8k (color grade)

                                • [deleted]

                                • Post #38
                                • Edited

                                I recall a discussion about this topic from some months ago.

                                AI upscaling for VR has potential. However, for there to be visible improvement one would have to improve the textual models generated specifically for VR -- from what I've been told, and what seems logical.

                                One would have to begin with the studio file or the original recording in highest resolution, so the film doesn't suffer from iterative downgrading (through compression from the various encoding formats, most of which are lossy, not lossless).

                                A company, like Topaz, would then have to re-train their AI working from images captured with a fisheye lens.

                                  [deleted] we suggested Topaz a deal to upscale our library, but they admitted they were not able to provide any visual gains.

                                  There were some points for research, but nothing major would be expected.

                                      • [deleted]

                                      • Post #40
                                      • Edited

                                      doublevr

                                      You could try this and use an in-house model (developed for VR). It's free.

                                      https://towardsdatascience.com/deep-learning-based-super-resolution-with-opencv-4fd736678066

                                      The difficulty would be training it for your needs. But you have a fantastic library to draw textures from!

                                      You know what you could also do... Ask Meta. Of course, they are really creepy to deal with, and they say they HATE porn (because secretly they just LOVE it)... But if you claim to be a researcher you could probably get help and ideas...

                                        a year later

                                        With better algorithms and better screens there should be more of the difference. The only thing it is only relevant for old videos while we are looking bright into the future