Steam reviews need a five star rating scale.

[FOXDUDE69]

The simplistic thumbs up or thumbs down thing can be very misleading. Games aren't just good or bad, there's a gradient there and I know you can explain your rating in the text but when you go to a store page and you see an "overwhelmingly positive" rating of 98%, you might think it's the hottest thing since Jessica Alba, only to buy the game and realize it's just a 4 out of 5 game.

But that's not the biggest problem, the biggest problem is when you have a game that is overall good but has some serious issues. Or a game that's just "good enough" for the price. The reviewer is forced to pick from positive or negative rating and since the experience was overall positive, they pick positive which usually results in games having a "very positive" rating, sitting around 85% when in reality it's a 2.5 or 3 out of 5 game.

The absolute worst scenario is one like Fallout 3's which is left in an abysmal state because of steam's lack of standards and Bethesda's let-the-fans-fix-it attitude resulting in a game that doesn't even launch without being modded by the paying costumer and requires further work to run well. Every new player needs to go through this process but the game is real good after you you spend a good amount of time fixing it, so they leave positive reviews... on a game that doesn't even launch. And it's sitting right now with a "mostly positive" rank on steam.

A simple five star rating scale would help a lot and provide a much more honest and accurate average rating.

Thank you for reading.

[Crazy Tiger]

Messaggio originale di Foxdude:

Messaggio originale di Gallifrey - CSSC Gaming Founder:

There's a definite demand for reviewers to have a more nuanced scoring system. It appears frequently in the body of the reviews "if I had the option I would have...". And the biggest players all use the five point scale where you're being expected to part with money for the product.

Fundamentally because of it's extreme simplicity while allowing a bit of nuance.
A smaller scale doesn't allow for nuance, and a bigger one needlessly complicates things.
Like the 10 point scales that use decimals. Effectively making them 100 point scales!
Ludicrous! And a binary scale doesn't reflect reality. Things are not good or bad.
Our games, the majority of them, have a varying mix of good and bad in them.
Ultimately our review ratings should reflect that. Creating a more accurate overall score.
That's my 2 cents anyway.

3 cents actually since I've invested so much time talking about this topic already!

Somehow I missed this post.
Point taken, which is why I think a 3 star rating is probably best.
Even if it would mean that a lot of the reviews might turn to neutral.
Change can be a good thing in this, though.
I think it's good to think about how this affects the overall rating.
About that, will games with 500 pos, 1000 neutrals and 200 neg be rated as pos or mixed?
Leaving more neutrals than pos/neg could be a possibility that should be tackled.

[Gallifrey - CSSC Gaming Founder]

Score Average 4+ - Highly Positive
Score Average 3.5 - 4 - Mostly positive
Score Average 2.5 - 3.5 - Mixed
Score Average 2 - 2.5 - Mainly Negative
Score Average less than 2 - Highly negative.

S.x.

[Quint the Alligator Snapper]

If you need to calculate one single one-dimensional summary statistic for the userscore, I think that case would be handled by simply counting the 1000 neutrals as half of each. So, essentially, you have 1000 positive and 700 negative. And then just reuse the existing system.

Alternatively, Gallifrey's idea above.

[Nx Machina]

Messaggio originale di Gallifrey - CSSC Gaming Founder:

Score Average 4+ - Highly Positive
Score Average 3.5 - 4 - Mostly positive
Score Average 2.5 - 3.5 - Mixed
Score Average 2 - 2.5 - Mainly Negative
Score Average less than 2 - Highly negative.

S.x.

Reviews are from a single perspective which is subjective and is not a clear indicator of a game overall. As they say one man's trash is another man's treasure.

Example:

Fifa - All bells and whistles but on the pitch, dull.

PES - No bells and whistles but on the pitch, exhilarating.

Neither are reviews but they are my experience of both games and I only play one - PES.


Two games you do not recommend and I highly rate in fact I replay them.

Star Wars Knights of the Old Repulic - a timeless classic, brilliant.

Divinity Original Sin EE - Divinity series in all it's glory, wonderful.

Again neither are reviews just my experiences with the games.


Finally you said Skyrim would be a 5 - To reach a 5 Skyrim has to be modded.

[Nightlight]

Messaggio originale di Crazy Tiger:

I think it's good to think about how this affects the overall rating.
About that, will games with 500 pos, 1000 neutrals and 200 neg be rated as pos or mixed?
Leaving more neutrals than pos/neg could be a possibility that should be tackled.

I don't have a guess on how it'd work if Steam switched to a star rating system, but if Valve added a "neutral" option to the existing system I see them making it so that "neutral" reviews don't count towards the overall score (like how reviews that are not from direct Steam purchases are treated.) One of the reasons I think they'd do that is to discourage people from hiding their review bombs under the "neutral" label.

[cinedine]

Messaggio originale di Gallifrey - CSSC Gaming Founder:

Score Average 4+ - Highly Positive
Score Average 3.5 - 4 - Mostly positive
Score Average 2.5 - 3.5 - Mixed
Score Average 2 - 2.5 - Mainly Negative
Score Average less than 2 - Highly negative.

Except that's not how these scales are applied on most services.

5.0 - most likely only sponsored reviews
4.5 - 4.9 - good
4.0 - 4.4 - somewhat good
1.1 - 3.9 - trash
1.0 - most likely will kill your machine

obligatory xkcd[xkcd.com]

There are some sites that actually do have more distinguished scale with actual meaning. Sotes with closed communities or which are dominated by regulars. Kongregate, Rotten Tomatoes, and IMDB are some that come to mind.
I looked up some tripadvisor ratings ... I am unable to find one entry with a thousand reviews and barely any past hundred. So the sample size is pretty low. Take a look at google ratings for these places - lots more, most of them 1s or 5s.

Google on the rating system changing to binary:
https://blog.youtube/news-and-events/five-stars-dominate-ratings

"Gamers" themselves have actually put games in a similar vein. Everything below an 80 % is considered trash tier by many.
I mean just look at the OP:

Messaggio originale di Foxdude:

The simplistic thumbs up or thumbs down thing can be very misleading. Games aren't just good or bad, there's a gradient there and I know you can explain your rating in the text but when you go to a store page and you see an "overwhelmingly positive" rating of 98%, you might think it's the hottest thing since Jessica Alba, only to buy the game and realize it's just a 4 out of 5 game.
.

JUST a 4 out of 5. The game isn't the greatest thing ever but just "good".

[FOXDUDE69]

Messaggio originale di cinedine:

"Gamers" themselves have actually put games in a similar vein. Everything below an 80 % is considered trash tier by many.
I mean just look at the OP:

Messaggio originale di Foxdude:

The simplistic thumbs up or thumbs down thing can be very misleading. Games aren't just good or bad, there's a gradient there and I know you can explain your rating in the text but when you go to a store page and you see an "overwhelmingly positive" rating of 98%, you might think it's the hottest thing since Jessica Alba, only to buy the game and realize it's just a 4 out of 5 game.
.

JUST a 4 out of 5. The game isn't the greatest thing ever but just "good".

Yeah, because there is a difference between a very good game and a masterpiece and pretending there isn't is just dishonest debate. I never said a 4 was bad in fact I already stated in this thread that I consider a 4 a good game. Something that only a contrarian would disagree with.

The point, which you completely, perhaps intentionally, is that the 98% approval rating does suggest a masterpiece. Which is why a more nuanced rating system would prove useful in generating more accurate overall ratings.

Now you could argue that people would gravitate to the extremes but if that would turn out to be true, then you'd end up with a system no different than the one you have now, losing nothing.

[cinedine]

Messaggio originale di Foxdude:

The point, which you completely, perhaps intentionally, is that the 98% approval rating does suggest a masterpiece. Which is why a more nuanced rating system would prove useful in generating more accurate overall ratings.

And the point that you completely missed multiple times is that an approval rating or a recommendation is NOT a score.
A recommendation rating of 98 % does not suggest anything else but people thinking something is worth it.

Messaggio originale di Foxdude:

Now you could argue that people would gravitate to the extremes but if that would turn out to be true, then you'd end up with a system no different than the one you have now, losing nothing.

So why change it if there is a reasonable chance that it will end up just the same only more convoluted?

Also what people don't get is that you cannot simply change the system now. A scale can be easily be downsized to less options. But scaling up is not that easy. What do you do with all these thumbs up and downs? Convert them into 1s and 5s? Which will immediately turn the scale into that J-distribution which makes it pointless? Or invalidate all reviews?

[Washell]

All this arguing about how a 5 star rating should be interpreted is rather sinking the chance of a non-binary system ever seeing the light of day here.

[Start_Running]

Messaggio originale di Washell:

All this arguing about how a 5 star rating should be interpreted is rather sinking the chance of a non-binary system ever seeing the light of day here.

You mean the idea that diferent people have different mental conceptions as to what any option means acttually shows the inherent problems with the system thus making implementation more unlikely? Imagine that.

[Quint the Alligator Snapper]

Messaggio originale di cinedine:

Also what people don't get is that you cannot simply change the system now. A scale can be easily be downsized to less options. But scaling up is not that easy. What do you do with all these thumbs up and downs? Convert them into 1s and 5s? Which will immediately turn the scale into that J-distribution which makes it pointless? Or invalidate all reviews?

Why not turn them into 2s and 4s?

Anyhow, others have already propose a simpler three-point system, which eliminates the problem you've mentioned.

Messaggio originale di Washell:

All this arguing about how a 5 star rating should be interpreted is rather sinking the chance of a non-binary system ever seeing the light of day here.

As if all the arguing that's ever happened about any other proposal has had much effect on its prospects. I remember all the "explaining" (read: arguing using armchair business analysis of questionable reliability) that people here did saying that stuff like ignoring forum posts didn't matter and wasn't important to Valve's business, and then years later, suddenly the feature appeared anyway.

[cinedine]

Messaggio originale di Quint the Alligator Snapper:

Messaggio originale di cinedine:

Also what people don't get is that you cannot simply change the system now. A scale can be easily be downsized to less options. But scaling up is not that easy. What do you do with all these thumbs up and downs? Convert them into 1s and 5s? Which will immediately turn the scale into that J-distribution which makes it pointless? Or invalidate all reviews?

Why not turn them into 2s and 4s?

Anyhow, others have already propose a simpler three-point system, which eliminates the problem you've mentioned.

Yeah why not?
Although that be rather pointless as it would move all games toward 3 which in best case is neutral and more realistically is considered trash by most people.
A three point system won't change anything on this. A 0 is not helping anyone with their decission. If you take it as a rating, it too will inevitably end up as another negative. Anything that is not good is bad. Our brains don't work in a D&D alignment grid but a black and white view. Do or do not.

[Quint the Alligator Snapper]

Messaggio originale di cinedine:

Messaggio originale di Quint the Alligator Snapper:

Why not turn them into 2s and 4s?

Anyhow, others have already propose a simpler three-point system, which eliminates the problem you've mentioned.

Yeah why not?
Although that be rather pointless as it would move all games toward 3 which in best case is neutral and more realistically is considered trash by most people.
A three point system won't change anything on this. A 0 is not helping anyone with their decission. If you take it as a rating, it too will inevitably end up as another negative. Anything that is not good is bad. Our brains don't work in a D&D alignment grid but a black and white view. Do or do not.

Except "our brains" clearly don't, since people are also perfectly able to sort out the differences between things that are "somewhat" vs. "very" satisfactory.

You may say people tend to rate toward the extremes of the scale, but the "problem" (assuming it is one) is that the people with milder opinions are underreporting their opinions, while those with stronger opinions are more enthusiastic about reporting them. Even so, there are clearly people who rate using the middle rating(s).

And if all we use is a 1-bit statistic, we basically end up forcing everyone to display polarized opinions, even if they aren't actually polarized.

[Nx Machina]

Recommended and Not Recommended offers a middle ground - pros and cons within text of the review if the review is well written and not just a one liner.

With a neutral option you literally cannot see the wood for the trees simply because by definition a neutral review is not a answer.

My response to a neutral review - "why bother posting it when you make no commitment, why not simply post your shopping list which would be more informative of your eating habits."

[cinedine]

Messaggio originale di Quint the Alligator Snapper:

Messaggio originale di cinedine:

Yeah why not?
Although that be rather pointless as it would move all games toward 3 which in best case is neutral and more realistically is considered trash by most people.
A three point system won't change anything on this. A 0 is not helping anyone with their decission. If you take it as a rating, it too will inevitably end up as another negative. Anything that is not good is bad. Our brains don't work in a D&D alignment grid but a black and white view. Do or do not.

Except "our brains" clearly don't, since people are also perfectly able to sort out the differences between things that are "somewhat" vs. "very" satisfactory.

Can you? You can rank experiences in order satisfaction. But if you are forced to decide whether an experience was a "somewhat" or "very" satisfactory one you will end up overthinking stuff as there always will be something in between your options.
And it's not like it matters. It was satisfactory. You'd do it again.