TimeSpy no longer as reliable for CPU testing vs. other benchmarks?

[Dev]

I can link some test results but TL;DR

Note 1: Yes TimeSpy is more a combined/GPU weighted test but noting it’s still a mainstream popular benchmark for CPU/GPU.

Note 2: I have elaborated but the test conditions are also checked across each benchmark including clock speeds/temps/utilisation per core etc. These parameters were checked by me to ensure essentially the lower CPU score on TimeSpy is as reasonable as possible isolated to what TimeSpy is producing vs. other benchmarks.

Basic test setup:

I was testing a Lenovo Legion Pro 7i 13900HX/4090 laptop. CPU UV & slight GPU OC applied via throttle stop & MSI AB. Ran TimeSpy, TimeSpy Extreme & Cinebench tests.
Ran all tests under stock conditions vs. tinkered.

Issue:

13900HX CPU should score ~ 18000 range in TimeSpy especially looking at other people with matching hardware configurations (with test results from months ago). Issue is my CPU consistently scores low to mid 16K in TimeSpy… User config/hardware error I hear?

On CineBench R23 with same test conditions my CPU scores ~32K multi core & 2,050 single core running multiple 10 mins benchmarks. My Cinebench results are pretty much exact match of other user scores with similar tweaks except TimeSpy.

From that I’m gathering there’s either been some Windows OS updates since beginning of year or issue with 3D Mark TimeSpy especially with Windows 11 vs. 10 but apparently also newer windows updates.

UV/OC forums are not recommending TimeSpy as a reliable indicator anymore. Wonder what devs here have to say.

[UL_Jarnis]

Time Spy is old and it's CPU test was never designed to scale past 8 cores. It was released during the time when 4 cores was the norm and very few games used more than that for anything. It is effectively obsolete today.

This is why you see "unreliable" Time Spy CPU scores on 13900HX. It has too high core count so some of its cores are not used and due to Intel P/E core scheduling being what it is, score varies depending on where the threads land. This is exactly why we no longer recommend Time Spy for systems with more than 8 CPU cores unless you want to check system performance for old games that have very naive threading code and poor thread scaling.

(Note that many old games perform exactly like Time Spy on CPUs like this, but usually the framerates are so high that you don't really care)

If you look at the benchmark it no longer recommends Time Spy. We know it is no longer a good benchmark for modern hardware. So in a way you are stating the obvious. We know.

Time Spy Extreme CPU test and CPU Profile CPU test are different and do scale at least up to 64 cores and are both still quite valid. They do not have this issue and you can use them perfectly fine.

For modern graphics performance testing, we recommend using Speed Way or Steel Nomad, depending on if you want the test to use raytracing or not. For modern CPU performance testing, we recommend CPU profile test.

[Scardigne]

Originally posted by UL_Jarnis:

Time Spy is old and it's CPU test was never designed to scale past 8 cores. It was released during the time when 4 cores was the norm and very few games used more than that for anything. It is effectively obsolete today.

This is why you see "unreliable" Time Spy CPU scores on 13900HX. It has too high core count so some of its cores are not used and due to Intel P/E core scheduling being what it is, score varies depending on where the threads land. This is exactly why we no longer recommend Time Spy for systems with more than 8 CPU cores unless you want to check system performance for old games that have very naive threading code and poor thread scaling.

(Note that many old games perform exactly like Time Spy on CPUs like this, but usually the framerates are so high that you don't really care)

If you look at the benchmark it no longer recommends Time Spy. We know it is no longer a good benchmark for modern hardware. So in a way you are stating the obvious. We know.

Time Spy Extreme CPU test and CPU Profile CPU test are different and do scale at least up to 64 cores and are both still quite valid. They do not have this issue and you can use them perfectly fine.

For modern graphics performance testing, we recommend using Speed Way or Steel Nomad, depending on if you want the test to use raytracing or not. For modern CPU performance testing, we recommend CPU profile test.

cpu profile test does not even load my 5950x 100% anymore it only loads all cores to 80% when on max threads, therefore the max threads and 16 threads are VERY similar, why is that?? some kind of update to cpu profile broke it

[UL_Jarnis]

5950x has 16 cores. "Max threads" will try to push 32 threads but SMT isn't same as real cores and as the load is quite heavy, there is very little to be gained for going to 32 threads on such configuration over 16 threads. This is completely normal.

[Scardigne]

Originally posted by UL_Jarnis:

5950x has 16 cores. "Max threads" will try to push 32 threads but SMT isn't same as real cores and as the load is quite heavy, there is very little to be gained for going to 32 threads on such configuration over 16 threads. This is completely normal.

Ok but is I'm sure the benchmark in the past maxed out cores to 100% load instead of 80%

[Dev]

Ok appreciate the explanation. That would explain why on TimeSpy I’m seeing abnormally low CPU scoring but on TimeSpy Extreme my overall score (including GPU & CPU separate scores) are closer to top percentile. I’ll checkout the other recommended benchmarks as well