I agree with adjusting IPD settings and definitely experiment with facial interface. I use quest 3 without it using BoboVR S3 strap. That gives better FOV and helps adjusting lenses.
If I have to guess your eyes are too "sharp" and brains are just wired the way you notice these errors better. Same goes for me 🙁 That blessing is curse in VR world, because that is a "feature" of VR porn. Some studios and camera/lens combinations do it better than others.
I have always wondered why it is so hard to make proper projections? Why we use 180 FOV or even 220 videos if edges are distorted? Why we use 8K resolution if projection is only 80% perfect? Why to even bother filming group sex etc with multiple models left and right if you can properly focus only to the center?
If I would know anything about programming and had proper skills my first job would be to make the best projection possible.