Post by Rune on Oct 18, 2022 10:21:34 GMT
VoodooDE was one to focus on this issue in 2019:
VoodooDE directly mentions the Index having a big advantage due to running native Steam drivers (start from 7:30 in the vid). One dude mentions this in a comment:
"Thanks Voodoo very interesting comparison. Your comments about additional overheads when running steamvr on oculus link is very relevant. I have noticed that my oculus pcvr native games run a lot better than the games through steamvr."
VoodooDE found on the same rig:
Quest + Link (identified as Rift-S in 72 Hz) 1808x2000 = 45.2 fps
Rift-S 1648x1776 = 51.2 fps
Pimax 8K (identified as Vive MV) 3684x3116 = 16.1 fps
Index 2016x2240 = 48.5
Let's just focus on Rift-S and Index (leaving streaming out of the equation for now). To the naked eye, all may look fine here, Rift-S is close or faster than the Index - but here it's extremely important to note the res. Let's do it differently - by calculating the res in pixels:
Rift-S 2.9 mill pixels = 51.2 fps
Index 4.5 mill pixels = 48.5 fps
So Index is pushing 55% more pixels, and has nearly same speed as Rift-S (Rift-S is 6% faster). While everything may look good, this was a really massive (well-hidden) difference.
Personally I did notice the great difference between using native Oculus drivers for my CV1 or Steam drivers too - but I thought Oculus had much better ASW support.
Some time later, I got a message on Reddit from one user who discovered a strange problem - he had both Index and Quest 2, and noticed when using similar res, the Quest 2 was much slower - back then I though this was the price for streaming, but it gets more complicated - again these results, like VoodooDE, are measured using the exact same rig:
These were interesting results - for long I suspected that streaming had a great impact.
For the OpenVR Benchmark, cpu means very litte (I've seen no pattern to rigs with faster cpus scoring better at all), but if someone cools a gpu in liquid nitrogen and performs great overclocking this will impact the score. This is a serious issue comparing different rigs - or at least it's very important not to compare results with some mad gpu-overclocking monster, lol.
I don't have a Quest 2 - but Nalex just got his 3080 Ti, which in gaming benchmarks has same speed as the 3090, or within a (few) percent(s):
- Nalex had not oc'ed his card, and I trust his numbers. Nalex got these numbers with his Quest 2 when running same res as Index res 100% (4.5 mill pixels per eye) - and even if he does not have the fastest cpu, it's more than enough to bottleneck the gpu:
- testing my own rig with same res and Index I got:
My 3090 is not oc'ed much by Asus, and I run the silent bios with reduced clock speeds - but let's just reduce my results down to the 76 fps many get with plain vanilla 3090s. Even if we calculate Index + 3090 as 76 or 80 fps, the difference to Nalex' 57 fps is massive.
I know some may think it's due to the cpu, but it really is not. Nalex was using Airlink + Streaming, and first we thought streaming had a great impact. Nalex then tested against his CV1 and found to some great surprise that when using same res, CV1 was only 2-5% faster than a tethered Quest 2. Streaming had very little impact.
Then Nalex tried to use Virtual Desktop, which I think supports native Oculus drivers - and increased his 57 fps to:
Now we're getting closer to the theoretical 76 fps he should be getting - and of course he is still streaming. But it seems other drivers had a great impact here.
Kojack then tested his HP G2 vs Quest 2 and found (same rig) - to check that streaming is close to free (he used Airlink, not Virtual Desktop):
So not much impact of the streaming. Kojack concluded - when comparing to the Index etc:
"The thing to remember with an Index vs Quest 2 comparison in an OpenVR benchmark is that the Index is running natively in OpenVR, while the Quest 2 is being forced through an OpenVR to Oculus wrapper written by Valve."
So it's complicated - and real difference are easily hidden.
And no one seems to be doing the real experiement - like BabelTechReviews would be able to. I might just ask him.
Also when comparing different hmds and rigs, of course it's very important that the rigs have similar results using same hmd and same gpu - like this dude testing his Vive Pro 2 with a RTX 3090 - he wrote:
"I'm hitting around 1850mhz stable on the GPU (3090) and 5ghz on the CPU (i7 8700K). My score is around 38/39 FPS. Comparatively, I would get in the 80fps range on index."
www.reddit.com/r/Vive/comments/p9zipg/anyone_else_benchmarked_the_vive_pro_2/
So he got same 80 fps using Index as I do (assuming he's not lying) - and got 38/39 fps with the Vive Pro 2 native res - and I was just surprised to see this, when I ran the benchmark comparing to my own rig:
- and I found another Vive Pro 2 benchmark, where the difference was similar. Again, not the perfect comparison due to different rigs, but Nalex getting 14% more performance by just using Virtual Desktop for the Quest 2 - VoodooDE measuring much lower performance with the Rift S - and so on, it all adds up indicating that different drivers impact performance - and that Index profits from having native SteamVR driver support.
Now with the Odyssey+ it's more difficult - I have not found any results, and initially there were issues with WMR, like VoodooDE mentions, but these have been resolved, at least G2 works fine in the test now. Read than some said on Reddit they were getting surprisingly great results with the Odyssey+, and I've seen some trends of Pimax only scoring about 20% lower than Index - so maybe some companies are better at fintuning drivers than other - and without proper benchmarks we may just never know.
For now it is what it is - but does correspond well with my own experiences playing games with my CV1 using StreamVR or native Oculus drivers.
I could deactivate the Index and test the CV1, but I'm having issues with Index showing up like a third monitor and not working. And no one cares much about the CV1 these days anyway, so comparing Quest 2 vs. Index would be the most important - although the above results are doing that to some degree already.
VoodooDE directly mentions the Index having a big advantage due to running native Steam drivers (start from 7:30 in the vid). One dude mentions this in a comment:
"Thanks Voodoo very interesting comparison. Your comments about additional overheads when running steamvr on oculus link is very relevant. I have noticed that my oculus pcvr native games run a lot better than the games through steamvr."
VoodooDE found on the same rig:
Quest + Link (identified as Rift-S in 72 Hz) 1808x2000 = 45.2 fps
Rift-S 1648x1776 = 51.2 fps
Pimax 8K (identified as Vive MV) 3684x3116 = 16.1 fps
Index 2016x2240 = 48.5
Let's just focus on Rift-S and Index (leaving streaming out of the equation for now). To the naked eye, all may look fine here, Rift-S is close or faster than the Index - but here it's extremely important to note the res. Let's do it differently - by calculating the res in pixels:
Rift-S 2.9 mill pixels = 51.2 fps
Index 4.5 mill pixels = 48.5 fps
So Index is pushing 55% more pixels, and has nearly same speed as Rift-S (Rift-S is 6% faster). While everything may look good, this was a really massive (well-hidden) difference.
Personally I did notice the great difference between using native Oculus drivers for my CV1 or Steam drivers too - but I thought Oculus had much better ASW support.
Some time later, I got a message on Reddit from one user who discovered a strange problem - he had both Index and Quest 2, and noticed when using similar res, the Quest 2 was much slower - back then I though this was the price for streaming, but it gets more complicated - again these results, like VoodooDE, are measured using the exact same rig:
These were interesting results - for long I suspected that streaming had a great impact.
For the OpenVR Benchmark, cpu means very litte (I've seen no pattern to rigs with faster cpus scoring better at all), but if someone cools a gpu in liquid nitrogen and performs great overclocking this will impact the score. This is a serious issue comparing different rigs - or at least it's very important not to compare results with some mad gpu-overclocking monster, lol.
I don't have a Quest 2 - but Nalex just got his 3080 Ti, which in gaming benchmarks has same speed as the 3090, or within a (few) percent(s):
- Nalex had not oc'ed his card, and I trust his numbers. Nalex got these numbers with his Quest 2 when running same res as Index res 100% (4.5 mill pixels per eye) - and even if he does not have the fastest cpu, it's more than enough to bottleneck the gpu:
- testing my own rig with same res and Index I got:
My 3090 is not oc'ed much by Asus, and I run the silent bios with reduced clock speeds - but let's just reduce my results down to the 76 fps many get with plain vanilla 3090s. Even if we calculate Index + 3090 as 76 or 80 fps, the difference to Nalex' 57 fps is massive.
I know some may think it's due to the cpu, but it really is not. Nalex was using Airlink + Streaming, and first we thought streaming had a great impact. Nalex then tested against his CV1 and found to some great surprise that when using same res, CV1 was only 2-5% faster than a tethered Quest 2. Streaming had very little impact.
Then Nalex tried to use Virtual Desktop, which I think supports native Oculus drivers - and increased his 57 fps to:
Now we're getting closer to the theoretical 76 fps he should be getting - and of course he is still streaming. But it seems other drivers had a great impact here.
Kojack then tested his HP G2 vs Quest 2 and found (same rig) - to check that streaming is close to free (he used Airlink, not Virtual Desktop):
So not much impact of the streaming. Kojack concluded - when comparing to the Index etc:
"The thing to remember with an Index vs Quest 2 comparison in an OpenVR benchmark is that the Index is running natively in OpenVR, while the Quest 2 is being forced through an OpenVR to Oculus wrapper written by Valve."
So it's complicated - and real difference are easily hidden.
And no one seems to be doing the real experiement - like BabelTechReviews would be able to. I might just ask him.
Also when comparing different hmds and rigs, of course it's very important that the rigs have similar results using same hmd and same gpu - like this dude testing his Vive Pro 2 with a RTX 3090 - he wrote:
"I'm hitting around 1850mhz stable on the GPU (3090) and 5ghz on the CPU (i7 8700K). My score is around 38/39 FPS. Comparatively, I would get in the 80fps range on index."
www.reddit.com/r/Vive/comments/p9zipg/anyone_else_benchmarked_the_vive_pro_2/
So he got same 80 fps using Index as I do (assuming he's not lying) - and got 38/39 fps with the Vive Pro 2 native res - and I was just surprised to see this, when I ran the benchmark comparing to my own rig:
- and I found another Vive Pro 2 benchmark, where the difference was similar. Again, not the perfect comparison due to different rigs, but Nalex getting 14% more performance by just using Virtual Desktop for the Quest 2 - VoodooDE measuring much lower performance with the Rift S - and so on, it all adds up indicating that different drivers impact performance - and that Index profits from having native SteamVR driver support.
Now with the Odyssey+ it's more difficult - I have not found any results, and initially there were issues with WMR, like VoodooDE mentions, but these have been resolved, at least G2 works fine in the test now. Read than some said on Reddit they were getting surprisingly great results with the Odyssey+, and I've seen some trends of Pimax only scoring about 20% lower than Index - so maybe some companies are better at fintuning drivers than other - and without proper benchmarks we may just never know.
For now it is what it is - but does correspond well with my own experiences playing games with my CV1 using StreamVR or native Oculus drivers.
I could deactivate the Index and test the CV1, but I'm having issues with Index showing up like a third monitor and not working. And no one cares much about the CV1 these days anyway, so comparing Quest 2 vs. Index would be the most important - although the above results are doing that to some degree already.
