AI Upscale proof based discussion
)
midorijin that is NOT correct.
The higher resolution of video production - the better. No matter what is headset resolution highest video resolution is always better.
Important is to render videos at original camera resolution. 4K camera should be rendered in 4K videos, 6K camera in 6K, 8K in 8K and so on. Yet some producers use to shoot in 5.4K and render it in 6K, 7K or 8K. They use video upscaling technique, which brings no actual improvement. They use it solely for marketing purposes. They are selling you 8K which is actually no more than 5.4K.
doublevr That the video is "normally" delivered at the maximum resolution of the camera, I fully agree, especially that beyond 6k, do we really see a difference with current headsets?
Nevertheless, in my opinion (which only concerns me), I do not completely agree that upscaling does not bring anything.
The example I showed above shows it quite well.
In 1920p upscale, the video is clearly sharper and more pleasant to watch (original video in 1440p, it does not officially exist in higher resolution).
Well, personally, beyond 4k videos (1920p, max 2048p), I don't care, not that I wouldn't like to have a better quality (5k or 6k max), but having all this on your hard drives, it requires a lot of space, so 4k is a good compromise between weight and quality. 
midorijin I must do really poor communication as you are trying to address your concerns so softly. Sorry about that.
Also trying to make sure we are discussing the same thing. Finally I've got you.
Can you share 1440p and 1920p files so I can take a look? If you can make 3860 or similar that would be really appreciated. Have to check the whole video, not just screenshots. Otherwise 1440p might be low enough to see a positive effect. Not that anyone is producing any such videos now
that title
Upscale brings no quality improvements
is just wrong
upscaling can bring some improvements but they are not dramatic improvements like filming in higher resolution. are the relatively minor improvements from upscaling enough to justify the time and cost of doing it? maybe they are maybe they arent but to say that it brings no improvement totally isnt true.
doublevr No, don't worry, I just went ahead to avoid any misunderstanding, since I imagine that with u/thejinx2Na, since it annoyed you for more than a year, it must have annoyed you.
So, it was just a 15-second test I did in December.
My PC has been used for several months for re-encoding, so unfortunately it doesn't have the time to manage this upscalling for the moment.
I agree that making a comparison on a simple image is not the most representative, but with a video, you need the time to go back to the user interface, restart the new video, come back to exactly the same passage ... personally, I take for granted that just by going back to the user interface, the eye has lost the information of comparison of the video (unless there is a "really" flagrant difference as if we went for example from a video with a lot of block of visible pixels to a super clean video.
I'd have to take a moment to make the video, but not easy.
Translated with www.DeepL.com/Translator (free version)
spacepirate we can disagree on that. the best thing would be to provide a sample showing that like I did
Can upscaling bring higher perceptual quality?
Well this is highly subjective. Upscaling can not bring any more real details, but in can render the available details in a way that is subjectively more pleasing to the eye.
And in fact, your comparison that should prove it does nothing actually shows that upscaling can do a lot.
The edges are a lot better defined, the railings on the ceiling look much better, and the mesh on the ceiling actually seems to have even more details. Whatever upscaling algorithm has been used seems to do a great job in detecting and filling up patterns. Even the pink lights, which look terrible in the original picture look while far from perfect quite a bit better.
It does that at the cost of quite some sharpening halos, so it might be subjective which one is better, especially as most of the improvement seems to happen in the background, while the sharpening halos are most disturbing around the model and the lingerie.
But you certainly can not say it does nothing.
Also WankzVR did some AI upscaling of their old scenes some still shot with 2k and they improved a ton. Especially the actresses look so much better in their upscaled versions, unlike the background which still looks like paintings, which made me wonder if they actually used the photoshootings of the girls they do with their videos to train the NN.
Are the comparable to scenes shot in real with the same pixelcount? Of course not and by a large mile.
But do they look better then the originals? Hell yes.
So you can not say, upscaling never ever can bring anything. The amount and if there even is an improvement varies highly on the algorithms used and scenes, but there are definitely some that show visible improvements, although of course they are no match to scenes already shot with much higher pixelcounts.
You could argue, that it is not upscaling that does the improvement, and you could get the same with the original file and post processing during playback, which of course is true, but with offline upscaling you have basically infinitely more computing power available to do so, so the end result can and in most cases will be better.
- Edited
ibins None of that is seen in the headset ¯\(ツ)/¯
Did my best to see a difference, but couldn't see any.
Wankz upscale looks absolute garbage to me. Seems like overupscaled. Totally unwatchable. I would prefer original files much more.
With current state of tech there's absolutely no reason to upscale. Nor upscale fans on reddit should assault me for stating that LOL
I don't do any AI upscaling myself, but to me, I can definitely see the difference between an older NA 3k video file with a AI-enhanced 8k video file (done by a fan, not by NA themselves).
Here is a screenshot of my own for comparison. Both are from the same frame and both from the "left" lens.
(Left 8k, AI-enhanced) || (Right 3k, original)
https://i.imgur.com/5DRclv7.png
The left side (remaster) is definitely clearer and has more detail (look at the guy's hair between his belly and dick), and it is smooth (it doesn't have the jaggies on the edges like common "upscale" videos do). The right side (original) also has some compression artifacts (look at Karla's cheekbones) and lots of jagged edges that the remaster removed.
But it's the AI-enhanced that makes all the difference. I do agree with what @doublevr is saying regarding how many studios are simply exporting lower res files to higher res (like 7k and 8k) and all they are doing is using some basic "sharpening" filters. I don't like this practice, as it is false advertising and just balloons the file size. But from my experience, seeing the few really well-done AI enhanced remasters are pretty cool for those legacy scenes from years back.
g2kbuffetboy I can definitely see the difference between an older NA 3k video file with a AI-enhanced 8k video file (done by a fan, not by NA themselves).
do you have a link for that?
doublevr Sure thing. I'll DM you on reddit because it is a "free" link and I don't want to support those on these forums =)
g2kbuffetboy otherwise can't really see any real difference. Neither it is going to be seen in VR. Can't justify any effort in that direction.
- Edited
doublevr I bought a copy of Topaz Labs video AI enhancement. It CAN do quite some impressive things. But there are many caveats and does not work in every instance.
PH default quality 400% zoom
AI Enhance
Been experimenting but the longer the video, the harder it is to get consistent results. So I'm looking into using different enhancements of the same video and then use masking/blending to try and get the best results. But it's a lot of work + am still learning.
None of that is seen in the headset ¯(ツ)/¯
You can when you know where to look. As the benefit is mostly seen in the background it is still very subjective if the upscale is really better, but it is there and in some parts undenieable.
Also the pink lights show less jaggies in the upscaled picture, and they are a huge distraction in an otherwise great video. Unfortunately they are still there, so it is not a huge improvement, but a slight one nonetheless.
I am wondering if and how much it would be better if the upscaling would have been done on the "RAW" material and not an already sharpened final output. This already has some slight sharpening artifacts which usually get worse with upscaling, and also limit how aggressive you can use sharpening during upscaling before it becomes too much.
Rakly3 if thats not proof i dont know what is
spacepirate That is not simply upscaling though. It's the AI interpreting the image according to parameters you set.
Rakly3 yeah but no one else is doing regular upscaling... theres no benefit to it
- Edited
spacepirate That's (imho) part of the problem- there are definitely studios who are just doing raw upscaling so instead of 5k they can release "7k/8k". Technically they're not wrong, but their image quality obviously doesn't improve with the increased resolution.
Topaz VEAI is a very cool application but agree with Rakly3 that it is challenging to actually get worthwhile results. I've been messing around with it for 6+ months in my personal life, and things can look hugely improved w/ a 2d comparative split screen of before/after...but then after spending a week to render 1 scene and putting on the headset to watch what you've made, it often looks either entirely too "artificial" or alternatively it only looks marginally improved from the original (meaning it was not worth the time & effort).
And the bigger argument to be made imho is that with camera improvements in the past year or so, along with the upcoming 8k cameras- there will be no need for AI upscaling. Already there are studios whose scenes I have not been able to "improve" at all using Topaz, even just with the 2d comparison.