Understanding Benchmark results?

[Lee Chun Bey]

hey guys. i was testing some of the nvidia panel settings and the effects on benchmark results and i stumbled across something that i didn't know what it means: the Variance results
I say that because, when testing the "optimize for compute performance" (for mining and other stuff) the average FPS dropped significantly but at a cost of no Variance at all and reduced Cpu usage. Here are the results:

"optimize for compute performance" ON:
Points; 13451
FPS/ AVG=91 MIN=45 MAX=130 VARIANCE=0 CPU=57%

"optimize for compute performance" OFF:
Points; 18755
FPS/ AVG=128 MIN=49 MAX=153 VARIANCE=4 CPU=80%

So here are the question: what does this variance really means and, about the cpu usage, it dropped because of the lower framerate being pulled by the gpu?

[Mr Deap]

Higher variance, the more it's placebo when the variance is at 0, it mean it's the representation of the performance of the hardware running Assetto Corsa videogame.

[Lee Chun Bey]

So, something like "system stability"?

[Mr Crisp]

The Nvidia option `Optimise For Compute Performance` is for non-game applications that will use CUDA cores and large amounts of VRAM- you don't use it for games, useful for rendering though.

The reason your CPU use is lower with a lower framerate is because there is less draw calls being sent to the CPU each frame, therefore you can expect lower CPU use when you have lower unbound frames- perfectly normal.

I have no idea what Mr Deap is on about, what he says has nothing to do with what you asked.

[Mr Deap]

Originally posted by Mr Crisp:

I have no idea what Mr Deap is on about, what he says has nothing to do with what you asked.

It's a simracer thing.

Here an example with variance at 9
https://steamcommunity.com/app/244210/discussions/0/490123938439609931/

[Mr Crisp]

Originally posted by Mr Deap:

Originally posted by Mr Crisp:

I have no idea what Mr Deap is on about, what he says has nothing to do with what you asked.

It's a simracer thing.

Here an example with variance at 9
https://steamcommunity.com/app/244210/discussions/0/490123938439609931/

Its still not clear what you were trying to say, that a variance in the benchmark being a higher number is a placebo? And at zero means what exactly, better performance, smoother frames and gameplay? Your post is confusing.

What does it mean when its zero? What does it mean when its not zero?

[Lee Chun Bey]

as strange that it may sound, i'm opting for letting in the compute performance because, in the games that i play, i stays above 60 fps, my cpu goes colder and in some cases like csgo im seeing more fps - don't even understand why.
but about the variance i could not get it yet :s

[Marty]

Originally posted by Mr Crisp:

Originally posted by Mr Deap:

It's a simracer thing.

Here an example with variance at 9
https://steamcommunity.com/app/244210/discussions/0/490123938439609931/

Its still not clear what you were trying to say, that a variance in the benchmark being a higher number is a placebo? And at zero means what exactly, better performance, smoother frames and gameplay? Your post is confusing.

What does it mean when its zero? What does it mean when its not zero?

Best to just ignore his posts, nobody ever knows what he is saying including him. At best his posts arepure BS but often enough its so mixed up you cant even determine what type of crap it is.

[Mr Deap]

One need to use google or being less lazy with a real dictionary from planet earth

Originally posted by Marty:

Originally posted by Mr Crisp:

Its still not clear what you were trying to say, that a variance in the benchmark being a higher number is a placebo? And at zero means what exactly, better performance, smoother frames and gameplay? Your post is confusing.

What does it mean when its zero? What does it mean when its not zero?

Best to just ignore his posts, nobody ever knows what he is saying including him. At best his posts arepure BS but often enough its so mixed up you cant even determine what type of crap it is.

var·i·ance
/ˈverēəns/
noun
noun: variance; plural noun: variances
the fact or quality of being different, divergent, or inconsistent.

It really mean what it mean, thus the higher the variance value, the higher the placebo. When the variance is 0 that's pretty much what you will experience while playing online, offline, full grid, with mods, etc...

The dev actually put the effort to add this value in the game which is a nice feature.

[Mr Crisp]

Originally posted by Mr Deap:

One need to use google or being less lazy with a real dictionary from planet earth

Originally posted by Marty:

Best to just ignore his posts, nobody ever knows what he is saying including him. At best his posts arepure BS but often enough its so mixed up you cant even determine what type of crap it is.

var·i·ance
/ˈverēəns/
noun
noun: variance; plural noun: variances
the fact or quality of being different, divergent, or inconsistent.

It really mean what it mean, thus the higher the variance value, the higher the placebo. When the variance is 0 that's pretty much what you will experience while playing online, offline, full grid, with mods, etc...
The dev actually put the effort to add this value in the game which is a nice feature.

I think you can assume that we know what the word variance means- there is no need to be pretentious about it.

What you still failed to actually explain was exactly what variance means in the benchmark- you say its a placebo then you say its a nice feature. Well which is it?

What do you mean when the variance is at zero, it pretty much will be the experience playing AC- this doesn't explain anything.

You're really not making any sense and haven't explained jack about what it actually means.

i'm starting to think your knowledge is nothing more than placebo.

[Mr Deap]

You seem challenged with explaination of the simulated value of the benchmark. It's always entertaining for some reason to post in the forum.

[SpaceGeek77]

Originally posted by O Grande Ednarg, Amigo do Ogima:

hey guys. i was testing some of the nvidia panel settings and the effects on benchmark results and i stumbled across something that i didn't know what it means: the Variance results
I say that because, when testing the "optimize for compute performance" (for mining and other stuff) the average FPS dropped significantly but at a cost of no Variance at all and reduced Cpu usage. Here are the results:

"optimize for compute performance" ON:
Points; 13451
FPS/ AVG=91 MIN=45 MAX=130 VARIANCE=0 CPU=57%

"optimize for compute performance" OFF:
Points; 18755
FPS/ AVG=128 MIN=49 MAX=153 VARIANCE=4 CPU=80%

So here are the question: what does this variance really means and, about the cpu usage, it dropped because of the lower framerate being pulled by the gpu?

The way I understand is, a variance of 0 means that the performance numbers in the benchmark is what should be expected while playing the game. And a higher variance value means that the performance numbers in the benchmark can variate while playing the game.

Tldr; VARIANCE=0 Performance similar to benchmark during game play

VARIANCE>0 Performance can vary from benchmark values

[ling.speed]

Originally posted by Lubeless Lu:

The way I understand is, a variance of 0 means that the performance numbers in the benchmark is what should be expected while playing the game. And a higher variance value means that the performance numbers in the benchmark can variate while playing the game.

Tldr; VARIANCE=0 Performance similar to benchmark during game play

VARIANCE>0 Performance can vary from benchmark values

Definitely it does not work like this. The benchmark is much less sophisticated. For once its a replay, in normal driving you get more load, especially on the cpu.

Afaik (and it was maaany years ago) variance was about frametimes, with V=0 you got smooth frame generation, with V=high you got tearing and framerate jitter.
Cant remember how exactly it was calculated but it was pretty simple math.

also thats a 4 year necro mate


edit: damn i cant remember what the Variance was, or even if we knew properly back in EA... but brain tells me it was something stupid, like the more gpu heavy the settings were the lower the variance, and most times id did not matter at all.

[SpaceGeek77]

Originally posted by ling.speed:

Originally posted by Lubeless Lu:

The way I understand is, a variance of 0 means that the performance numbers in the benchmark is what should be expected while playing the game. And a higher variance value means that the performance numbers in the benchmark can variate while playing the game.

Tldr; VARIANCE=0 Performance similar to benchmark during game play

VARIANCE>0 Performance can vary from benchmark values

Definitely it does not work like this. The benchmark is much less sophisticated. For once its a replay, in normal driving you get more load, especially on the cpu.

Afaik (and it was maaany years ago) variance was about frametimes, with V=0 you got smooth frame generation, with V=high you got tearing and framerate jitter.
Cant remember how exactly it was calculated but it was pretty simple math.

also thats a 4 year necro mate :yukari4:


edit: damn i cant remember what the Variance was, or even if we knew properly back in EA... but brain tells me it was something stupid, like the more gpu heavy the settings were the lower the variance, and most times id did not matter at all.

So what your saying is lower Var equals smoother and better performance. And high VAR equals worse performance.

Ik its an old thread but it was the only one talking about the VAR value, so i thought id take an educated guess as to what it meant.

ive been able to bring it down from 4 to 0 but i cant say i notice much of a difference.

[ling.speed]

Originally posted by Lubeless Lu:

So what your saying is lower Var equals smoother and better performance. And high VAR equals worse performance.

Ik its an old thread but it was the only one talking about the VAR value, so i thought id take an educated guess as to what it meant.

ive been able to bring it down from 4 to 0 but i cant say i notice much of a difference.

I was one of the closed beta testers, with quite a bit of inner workings knowleage, and it was never explained what variance does (afair). It was one of the code and froget type of features. I bet Stefano would not remember now what he wrote that for too haha.


My experiance with Variance is that the higher the load on GPU the lower it gets. And it can get high when suffering from stutters or tearing. But its not a direct indicator of performance. You can have smooth as butter frames with like 8-16 variance. Or you can have 0 variance "no matter what you do". It varies from setup to setup.

If the variance in game is anything like in math, it could be indicator of some frames having high frametime (like a microfreeze). In AC these often come from apps and some extra stuff in CSP, both of which (again afair) is still rendered even if not on screen in benchmark.

Now putting more load on the GPU and reducing overal framerate increases overall frametime, which could hide these "microfreezes" (which are CPU dependant) and bring variance even to zero. So making the graphic look better is likely to drop variance.

So idk, if you have variance above 20 i'd start looking into it, but below its anyones game.


edit: if you are curious, "render stats" app (the kunos one) shows a frametime graph which is great for checking if game works ok. It should be a flat line.
And there is also the "render stats CSP" version that shows exactly what operation uses which resource, but its hard to use without indeapth knowleage.

[Mr Deap]

Originally posted by ling.speed:

Definitely it does not work like this.

Fun time incoming.

When variance is high, it mean it may not result this performance result in practice. Like if the benchy show 128fps. It mean it will less likely have this result while gaming. It's like what you get with COVID studies last year.

When the variance is low. It show again 128fps. The person will get 128fps while playing online. It's like a stamp of approbation it will run exactly like the benchy when playing.