Numeric Rating System (1-10 / 0 -5)

[Snake4eva]

Hey community, i'm wondering what are your thoughts on Steam's binary rating system (Recommend or Not Recommend) and do you think a numeric rating system would be more useful? Also which rating would you prefer? 0 - 5 or 1 - 10 and why?

[cinedine]

引用自 Snake4eva：

Recommend or not recommend would be useless to choose between 5 wines if they waiter would recommend all 5 based on different metrics.

And guess what. That's exactly what they do.
The recommend the whine based on your dish, your taste and your wallet. If you have a rich dish you don't want an overpowering whine and if you don't like dry, they won't recommend one to you. And if you don't look like a million dollars they will not recommend you the 1890s reserve.

[Snake4eva]

It is ranking that is lost in the recommend or not recommend. You can recommend something with a low ranking or not recommend something with a high ranking. However to suggest that something is high quality and meets all your subjective metrics and give it a 0 rating is illogical. Saying i recommend game X because i recommend game X is not an illogical reason in a non numeric system. Saying game X is a 5/5 because I say it is, is not an adequate review in a numeric system. The criteria for assigning a score may differ from person to person but simply asserting a score is illogical because numeric values intuitively invites meaningful graduation along a scale. Take a thermometer, the liquid in the thermometer is subjective and can be chosen at will, as each reviewer can choose their metric. But once chosen the boiling and melting point are fixed and the spectrum of ratings in between follow logically. To presume that people recommend products without doing some introspection or indirect rating is what i find surprising. I may recommend something to a friend that i hate but i look on its features and then rank it by comparing it to other things they've played and make the recommendation based on that rating. It may not be a 0 - 5 but it is numeric in the sense of how closely related it is to something else they like. I can do that by comparing the features with other games they've played and counting the number of similar features to assign a rating. Now that rating may be obfuscated by saying it looks similar to something you may like but that rating has occurred. Maybe its me and how i think about things. When i judge things i don't usually think strictly in binary, yes or no, recommend or not recommend, its at least ternary, maybe or average which establish some sort of degree to the endpoints.

[crunchyfrog]

Nope, nope nope.

As others have pointed out this is one of the many repeated requests and it's without doubt the dumbest one.

Why? Because it DOES NOT WORK. Adding a simple score conveys NO information of any use.

For instance, a decent game let down by bugs and lack of polish might cause a 7 out of 10.
Another game which is doen on a budget by a small team, but they punched above their weight might also get a 7 out of 10.

They are strakly different, and yet commadn the same score, proving how useless and misleading a score is.

And Valve definitely want to avoid that because at the bottom line, what you get is the PUREST form - a recommendation or not. End of.


And that's exactly how it should be.

[Nx Machina]

引用自 Snake4eva：

To answer all your questions i do both. If i recommend a dish it is because i've rated it. If the waiter recommends a whine to distinguish between them the waiter must've weighed each wine and ordered them by some preference. Recommend or not recommend would be useless to choose between 5 wines if they waiter would recommend all 5 based on different metrics. To rank them then numeric metrics are intuitive even if not explicitly stated. If a friend asked why i recommended the dish i'd say its tasty, presentation is good etc. If they asked my why this dish as compared to another dish or a similar dish but from a different restaurant then i'd have to rank them to distinguish. Quality is relative and recommending or not masks the quality of a product. A highly rated product may not be recommended but it gives relative measure of the user's perception of quality.

Good for you but in my circle of friends we do not rate products, authors, games, restaurants, movies, local takeaways, websites etc. We recommend or not based on our experiences but always say, give it a try, you may like something I missed or hate something I recommended. None of us are sheep, we decide for ourselves.

As for waiters they recommend a wine based on the meal you are having not on how pissed you want to get.

[TKOdarkshadow]

引用自 crunchyfrog：

Nope, nope nope.

As others have pointed out this is one of the many repeated requests and it's without doubt the dumbest one.

Why? Because it DOES NOT WORK. Adding a simple score conveys NO information of any use.

For instance, a decent game let down by bugs and lack of polish might cause a 7 out of 10.
Another game which is doen on a budget by a small team, but they punched above their weight might also get a 7 out of 10.

They are strakly different, and yet commadn the same score, proving how useless and misleading a score is.

And Valve definitely want to avoid that because at the bottom line, what you get is the PUREST form - a recommendation or not. End of.


And that's exactly how it should be.

But the current recommendations suffer from literally the same thing. If it’s subjective and no optional written info is included, what’s the use?

[Snake4eva]

引用自 Darren：

引用自 Snake4eva：

If numeric rating systems are such useless tools then why are exams, online retail reviews and so many other systems built upon such systems? Pass or fail, recommend or not recommend is not an adequate representation of the vast majority of subjective review of products or metrics that are generally encountered in life (@cinedine)

You are doing that thing where you are trying to compare two very different things again.

Numbers work when they are objective and you need to take a single person's opinion into consideration only. They are used for ranking something.

They are useless is a 9 isn't better than an 8. So they work really well for exams (this is the order of how well these people did) which is objective you did or did not meet the criteria to get this point and this is your total points.

But it is useless when you apply it to more subjective things. Multiple review sites gave Cyberpunk a 10/10 given the issues it has it is most definitely not a 10/10 game and is worse than games those same places gave a 9/10 even when that 9/10 game had similar issues. This means that number is quite literally useless for ranking or assessment (you can't sort by the top N reviews and know you will like those games). You don't even know whether they recommend you purchase the game or not.

The point is 10/10 and not recommend does two things? (1) It shows that the game meets the highest quality set out by the standard of the reviewer (2) That even though it meets a high standard there are still subtle nuances that the reviewer thinks maybe important to some and not recommend it on that basis. Many people who don't recommend Cyberpunk still praises the game and objectively admits that it is of high quality. They still don't recommend it because they identify that it isn't what it promised, many features were sacked and there are other games in its category that has better mechanics. Even though they give it a binary rating they still do comparisons to justify their recommend / not recommend. Also product reviews based on numeric systems are anything but objective. So numeric systems are ideal in objective cases but are popularly used in subjective areas also and with great effect.

[crunchyfrog]

引用自 TKOdarkshadow：

引用自 crunchyfrog：

Nope, nope nope.

As others have pointed out this is one of the many repeated requests and it's without doubt the dumbest one.

Why? Because it DOES NOT WORK. Adding a simple score conveys NO information of any use.

For instance, a decent game let down by bugs and lack of polish might cause a 7 out of 10.
Another game which is doen on a budget by a small team, but they punched above their weight might also get a 7 out of 10.

They are strakly different, and yet commadn the same score, proving how useless and misleading a score is.

And Valve definitely want to avoid that because at the bottom line, what you get is the PUREST form - a recommendation or not. End of.


And that's exactly how it should be.

But the current recommendations suffer from literally the same thing. If it’s subjective and no optional written info is included, what’s the use? :lunar2019deadpanpig:

No they're not. That's incredibly dishonest.

They're not the same thing because there's this little thing called TEXT. You know the ACTUAL review to back it up.

[Darren]

引用自 TKOdarkshadow：

引用自 crunchyfrog：

Nope, nope nope.

As others have pointed out this is one of the many repeated requests and it's without doubt the dumbest one.

Why? Because it DOES NOT WORK. Adding a simple score conveys NO information of any use.

For instance, a decent game let down by bugs and lack of polish might cause a 7 out of 10.
Another game which is doen on a budget by a small team, but they punched above their weight might also get a 7 out of 10.

They are strakly different, and yet commadn the same score, proving how useless and misleading a score is.

And Valve definitely want to avoid that because at the bottom line, what you get is the PUREST form - a recommendation or not. End of.


And that's exactly how it should be.

But the current recommendations suffer from literally the same thing. If it’s subjective and no optional written info is included, what’s the use?

Because those two games would also have gotten different recommendations. The 7/10 decent game let down by bugs and lack of polish would have gotten a not recommend (from me) while the 7/10 game from the smaller team with the low budget that punched above their weight would be more likely to have gotten a recommend.

Basically the score is not equal to the recommendation, and the recommendation is far more valuable (especially to me). I can't actually review Cyberpunk 2077 on GOG because GOG does use the star system, and my star rating (of 4) would be misleading as people would construe that as positive and consider purchasing the game but I actually don't recommend that people buy it at the moment (the people who I feel are the ones that would enjoy it are the ones that already would have made the purchase of the game).

However give the nature of the game, and the issues it wouldn't be fair to give it a rating of less than 4 stars out of 5 even though I don't recommend that people purchase it.

[Snake4eva]

引用自 cinedine：

引用自 Snake4eva：

Recommend or not recommend would be useless to choose between 5 wines if they waiter would recommend all 5 based on different metrics.

And guess what. That's exactly what they do.
The recommend the whine based on your dish, your taste and your wallet. If you have a rich dish you don't want an overpowering whine and if you don't like dry, they won't recommend one to you. And if you don't look like a million dollars they will not recommend you the 1890s reserve.

Yes but they've ranked the wines based on various criteria to determine the order from highest to lowest and make a recommendation. They didn't just recommend it because they recommend it or simply because they liked it. Some rating occurred based on a number of factors using assumptions and sometimes input from the user.

[TKOdarkshadow]

引用自 crunchyfrog：

引用自 TKOdarkshadow：

But the current recommendations suffer from literally the same thing. If it’s subjective and no optional written info is included, what’s the use?

No they're not. That's incredibly dishonest.

They're not the same thing because there's this little thing called TEXT. You know the ACTUAL review to back it up.

Uh, you're the person telling me there's no text in a review with a number attached to it though. I'm not sure why you seem to believe once you add a number system to it, all reviews will just magically lose the optional text-based part.

[DiceDsx]

引用自 TKOdarkshadow：

引用自 Shogun Blade：

And those reviews are a singular personal opinion of the reviewer which is subjective.

You can add a numeric value, percentage value, star value to a review on Steam in the text BUT that does not give validity to the review whether positive, negative or even neutral.

Yes, but at that point you seem to just be thinking that no reviews really matter. Recommending or not recommending doesn't give validity to the review since it's subjective, right? Or what's different about a number system?

Imho, it's both invalid and valid at the same time: invalid because everyone uses a different metric to evaluate a game, and valid because you can extrapolate information to help you choose if you should purchase or not.

Also, we could always just have a number system slapped ontop of the recommendation
system, so like "I rated it 9/10 as a game, but it's still not my type of game, so it's not recommended by me." :gk1_skull:

You could. You could also write "It's a fantastic game, but it's still not my type of game, so it's not recommended by me."

The score doesn't add anything meaningful, it's all in the text.

[TKOdarkshadow]

引用自 76561198007284667：

引用自 TKOdarkshadow：

But the current recommendations suffer from literally the same thing. If it’s subjective and no optional written info is included, what’s the use?

Because those two games would also have gotten different recommendations. The 7/10 decent game let down by bugs and lack of polish would have gotten a not recommend (from me) while the 7/10 game from the smaller team with the low budget that punched above their weight would be more likely to have gotten a recommend.

Basically the score is not equal to the recommendation, and the recommendation is far more valuable (especially to me). I can't actually review Cyberpunk 2077 on GOG because GOG does use the star system, and my star rating (of 4) would be misleading as people would construe that as positive and consider purchasing the game but I actually don't recommend that people buy it at the moment (the people who I feel are the ones that would enjoy it are the ones that already would have made the purchase of the game).

However give the nature of the game, and the issues it wouldn't be fair to give it a rating of less than 4 stars out of 5 even though I don't recommend that people purchase it.

I still don't see what the issue is with allowing for extra info from the reviewer. If someone considers a 3/10 to be God-tier and recommend it, that's up to them. If you wanna ask Steam to slap on some adjectives to the numbers, that's alright as well.
Example: 7-Good, 8-Very Good, 9-Great, 10-Perfect

[TKOdarkshadow]

引用自 DiceDsx：

引用自 TKOdarkshadow：

Yes, but at that point you seem to just be thinking that no reviews really matter. Recommending or not recommending doesn't give validity to the review since it's subjective, right? Or what's different about a number system?

Imho, it's both invalid and valid at the same time: invalid because everyone uses a different metric to evaluate a game, and valid because you can extrapolate information to help you choose if you should purchase or not.

Also, we could always just have a number system slapped ontop of the recommendation
system, so like "I rated it 9/10 as a game, but it's still not my type of game, so it's not recommended by me."

You could. You could also write "It's a fantastic game, but it's still not my type of game, so it's not recommended by me."

The score doesn't add anything meaningful, it's all in the text.

So if you get a ton of "It's a fantastic game, but it's still not my type of game, so it's not recommended by me" and the ratings are only based on recommendations, the rating of the game is much lower, despite it being a fine game. We shouldn't want that.

[Snake4eva]

引用自 DiceDsx：

引用自 TKOdarkshadow：

Yes, but at that point you seem to just be thinking that no reviews really matter. Recommending or not recommending doesn't give validity to the review since it's subjective, right? Or what's different about a number system?

Imho, it's both invalid and valid at the same time: invalid because everyone uses a different metric to evaluate a game, and valid because you can extrapolate information to help you choose if you should purchase or not.

Also, we could always just have a number system slapped ontop of the recommendation
system, so like "I rated it 9/10 as a game, but it's still not my type of game, so it's not recommended by me." :gk1_skull:

You could. You could also write "It's a fantastic game, but it's still not my type of game, so it's not recommended by me."

The score doesn't add anything meaningful, it's all in the text.

The score highlights the quality of the game on the chosen numeric sale. The same comment you gave above could not be associated with a less than 5 out of 10, fantastic does not correspond to the appropriate numeric value. A 5/10 is average, below 5 is below average, 10 is perfect or almost perfect. You can write whatever you want in your review the point is on a numeric scale the illogical act of a mostly positive review with a low score highlights the inherent contradiction between the two. In a binary system any review logical or illogical be affixed to a recommend or not recommend. A number invites meaningful ratings along the scale.

[Snake4eva]

引用自 TKOdarkshadow：

引用自 DiceDsx：

Imho, it's both invalid and valid at the same time: invalid because everyone uses a different metric to evaluate a game, and valid because you can extrapolate information to help you choose if you should purchase or not.


You could. You could also write "It's a fantastic game, but it's still not my type of game, so it's not recommended by me."

The score doesn't add anything meaningful, it's all in the text.

So if you get a ton of "It's a fantastic game, but it's still not my type of game, so it's not recommended by me" and the ratings are only based on recommendations, the rating of the game is much lower, despite it being a fine game. We shouldn't want that.

Precisely the point my friend. Cyberpunk on Steam for example i've zoomed in specifically on the not recommended reviews and most people say its a good game. I understand their reason for not recommending it but if you did not read their review you would not know that they thought the game was actually still good.