shinobi1987 It depends entirely on the studio tbh. As you said, some studios do just upscale, which will not look any better than their smaller resolution counterparts.
However- many studios do not upscale, or alternatively do upscale but only from (for example) 6k to 7k, and their 6k/7k/8k versions will look profoundly better than their 4k versions. Just take a look at any of the VRIntimacy scenes here on SLR, which are shot by CzechVR. The difference between their 8k version and 4k version will be almost night and day.
You should be able to tell the difference with SLR Originals fairly easily as well, even just comparing their 6k to 4k versions. I was going to suggest that streaming quality might be affecting how it looks, but sounds like you're watching the downloaded versions, so guess that's not the issue.