Steam Game Reviews

[BillyBozarth]

Is Steam manipulating user reviews to make games appear more highly rated than they actually are?

Over the years Steam has changed up how it displays user reviews on games. Here are some changes I have noticed:

1. They added an "off topic" review filter that was set to on by default. This was supposedly to help with review bombing however I never noticed Steam remove positive reviews in that metric. Essentially it appears as if Steam only removes "off topic" reviews when they are negative. Recently I have noticed this asterisk completely disappear altogether. Has this "feature" been removed?

2. Recently I have noticed the numbers not adding up in the review scores. For instance Battlefield 2042 currently shows the following numbers for reviews under all time:

Mixed 44% are positive
All 241,424
Positive 101,507
Negative 139,917

Simple math (101,507 / 241,424) * 100 = 42%
Where is the extra 2% coming from?

All these numbers are using standard filters and are related to each other (another words I am not applying different filters for the denominator and then numerator ect..)

3. They recently changed the color on the mixed tag to be less yellow. This isnt manipulating numbers but is considered a "nudge" in marketing terms to try and make a product appear more appealing than it is.

It would make sense that it would be in Steam's best interest to manipulate all game reviews towards a positive skew as they make a percentage profit off of their sales.

TLDR
Am I missing something here or are Steam reviews not to be trusted.

[potato]

引用自 BillyBozarth：

Is Steam manipulating user reviews to make games appear more highly rated than they actually are?

no

[BillyBozarth]

引用自 potato：

引用自 BillyBozarth：

Is Steam manipulating user reviews to make games appear more highly rated than they actually are?

no

Can you explain the 2% difference then? Or is no just what you want to believe? I would love to see where I am wrong here by explanation rather than "no"

[Low Standards]

引用自 BillyBozarth：

引用自 potato：

no

Can you explain the 2% difference then? Or is no just what you want to believe? I would love to see where I am wrong here by explanation rather than "no"

It is what he chooses to believe. Nothing will ever change that.

They are going to need to do that to survive though? I don't know the numbers, but I bet Early Access from indies is one of the biggest products they sell here. This leads to a lot of very, very bad games, mediocre games, and just plain scams.

That will never change. I do notice that Switch, PS4 and PS3 games are of overall higher quality. I don't mean they are good, just not to the level here.

[Crazy Tiger]

The average rating top right only counts reviews by purchases done on Steam itself and excludes key activations and off-topic filtered reviews. The numbers down below in the review section also include key activations and off-topic filtered reviews.

That's where the difference comes from.

[Spawn of Totoro]

引用自 BillyBozarth：

引用自 potato：

no

Can you explain the 2% difference then? Or is no just what you want to believe? I would love to see where I am wrong here by explanation rather than "no"

Purchased from 3rd party sites (Game keys) may be not included in the reviews. Private reviews may also not be included.

There are more then just off topic reviews to consider.

Honestly, 2% isn't really much and is considered in the realm of an acceptable margin of error for statistics and measurements. Also, if it isn't repeatable (on multiple games), it may just be a random miscalculation.

And with out knowing the name of the game, we can't really go check on the numbers our selves.

[BillyBozarth]

引用自 Crazy Tiger：

The average rating top right only counts reviews by purchases done on Steam itself and excludes key activations and off-topic filtered reviews. The numbers down below in the review section also include key activations and off-topic filtered reviews.

That's where the difference comes from.

So the mixed number I used came from left over the graph with any standard filters applied. Not the top right under the game summary. It is located right above the graph and under it is the breakdown of numbers that go into it. These are the numbers I used. Having a number displayed as the heading to a graph with numerator and denominator when that number does not represent the graph or the numbers under it is a bit misleading if this is actually the case.

[BillyBozarth]

引用自 Spawn of Totoro：

引用自 BillyBozarth：

Can you explain the 2% difference then? Or is no just what you want to believe? I would love to see where I am wrong here by explanation rather than "no"

Purchased from 3rd party sites (Game keys) may be not included in the reviews. Private reviews may also not be included.

There are more then just off topic reviews to consider.

Honestly, 2% isn't really much and is considered in the realm of an acceptable margin of error for statistics and measurements. Also, if it isn't repeatable, it may just be a random miscalculation.

And with out knowing the name of the game, we can't really go check on the numbers our selves.

All calculations and the name of the game is included in the post. Battlefield 2042

[Spawn of Totoro]

引用自 BillyBozarth：

Battlefield 2042

Miss that, sorry, thank you.

[Crazy Tiger]

引用自 BillyBozarth：

引用自 Crazy Tiger：

The average rating top right only counts reviews by purchases done on Steam itself and excludes key activations and off-topic filtered reviews. The numbers down below in the review section also include key activations and off-topic filtered reviews.

That's where the difference comes from.

So the mixed number I used came from left over the graph with any standard filters applied. Not the top right under the game summary. It is located right above the graph and under it is the breakdown of numbers that go into it. These are the numbers I used. Having a number displayed as the heading to a graph with numerator and denominator when that number does not represent the graph or the numbers under it is a bit misleading if this is actually the case.

I know. The number above the graph is the same as top right. That number only represents the applicable amount of reviews for the average rating, not the actual full amount of reviews made.

The difference comes from one number excluding key activations and off-topic filters (average rating) and the other number including key activations and off-topic filters.

[BillyBozarth]

引用自 Spawn of Totoro：

引用自 BillyBozarth：

Battlefield 2042

Miss that, sorry, thank you.

I wish I could include a screenshot. The numbers I am using are from the graph section of the reviews. I am interested if you show different numbers than me and if so what that variance if any is in the percentage rating. We could choose many other games as well but this one had a could data set to test with.

[BillyBozarth]

引用自 Crazy Tiger：

引用自 BillyBozarth：

So the mixed number I used came from left over the graph with any standard filters applied. Not the top right under the game summary. It is located right above the graph and under it is the breakdown of numbers that go into it. These are the numbers I used. Having a number displayed as the heading to a graph with numerator and denominator when that number does not represent the graph or the numbers under it is a bit misleading if this is actually the case.

I know. The number above the graph is the same as top right. That number only represents the applicable amount of reviews for the average rating, not the actual full amount of reviews made.

The difference comes from one number excluding key activations and off-topic filters (average rating) and the other number including key activations and off-topic filters.

Ok this makes sense. So essentially they are including additional criteria not specified in that number. It used to also give the option to exclude off topic or not. I usually always left this on as it never excluded off topic positive reviews so I felt it just artificially manipulated the data.Do you know if they removed this option to include off topic reviews?

[Spawn of Totoro]

引用自 BillyBozarth：

引用自 Spawn of Totoro：

Miss that, sorry, thank you.

I wish I could include a screenshot. The numbers I am using are from the graph section of the reviews. I am interested if you show different numbers than me and if so what that variance if any is in the percentage rating. We could choose many other games as well but this one had a could data set to test with.

"This summary uses only reviews written by customers that purchased the game directly from Steam."

So customers who purchased the game out side of Steam are not included.

183752 in the graph.

57685 reviews from other purchases.

引用自 BillyBozarth：

Ok this makes sense. So essentially they are including additional criteria not specified in that number. It used to also give the option to exclude off topic or not. I usually always left this on as it never excluded off topic positive reviews so I felt it just artificially manipulated the data.Do you know if they removed this option to include off topic reviews?

That option is there, only if reviews were removed for review bombing. The game you stated has not had games removed as such.

Also, review bombing reviews that get removed during that time frame are not just negative reviews. Any positive reviews during that same time frame are also not included in the scores.

[BillyBozarth]

引用自 Spawn of Totoro：

引用自 BillyBozarth：

I wish I could include a screenshot. The numbers I am using are from the graph section of the reviews. I am interested if you show different numbers than me and if so what that variance if any is in the percentage rating. We could choose many other games as well but this one had a could data set to test with.

"This summary uses only reviews written by customers that purchased the game directly from Steam."

So customers who purchased the game out side of Steam are not included.

183752 in the graph.

57685 reviews from other purchases.

Ok so I input the numbers in the graph into a spreadsheet. It appears you and crazy tiger are correct.

Here is what I learned:
The overall numbers have a separate nominator denominator than any of the applied filters and even the graph on most recent.

The most recent are likewise separated from "overall" with separate nominator denominator.

This is slightly confusing and to me a bit misleading. Running the numbers with this insight they do match up. (they appear to not be rounded)

Here are the numbers:

Overall
21612
7119
2209
824
572
445
380
1548
1409
1065
1731
1072
2151
3545
2176
1698
3455
1528
2358
1721
2468
1807
1020
9799
6926
1724
Most Recent
91
106
84
56
238
1048
677
521
594
500
389
372
195
159
147
138
157
121
191
128
94
105
121
117
61
79
67
74
62
62

[Spawn of Totoro]

引用自 BillyBozarth：

引用自 Spawn of Totoro：

"This summary uses only reviews written by customers that purchased the game directly from Steam."

So customers who purchased the game out side of Steam are not included.

183752 in the graph.

57685 reviews from other purchases.

Ok so I input the numbers in the graph into a spreadsheet. It appears you and crazy tiger are correct.

Here is what I learned:
The overall numbers have a separate nominator denominator than any of the applied filters and even the graph on most recent.

The most recent are likewise separated from "overall" with separate nominator denominator.

This is slightly confusing and to me a bit misleading. Running the numbers with this insight they do match up. (they appear to not be rounded)

Not transparent, I can agree with (and that they should make it more so), but if they add up, then they are not misleading.

[BillyBozarth]

引用自 Spawn of Totoro：

引用自 BillyBozarth：

Ok so I input the numbers in the graph into a spreadsheet. It appears you and crazy tiger are correct.

Here is what I learned:
The overall numbers have a separate nominator denominator than any of the applied filters and even the graph on most recent.

The most recent are likewise separated from "overall" with separate nominator denominator.

This is slightly confusing and to me a bit misleading. Running the numbers with this insight they do match up. (they appear to not be rounded)

Not transparent, I can agree with (and that they should make it more so), but if they add up, then they are not misleading.

I can agree with that. I think you answered all the questions. Another question I have is how the heck did you know all this lol?

NVM just saw that you were moderator