Weighted review score?

[Sytch]

As I understand, for now the percentage shows just positive divided by all or negative/all ratio depending on which reviews prevail. It's not right, there may be 10 negative reviews with high upvotes and 20 positive with just a few upvotes, nevertheless positive will win at 66%. Instead, they should be weighted. Not only by the approval percent but also by the absolute numbers so that 150 people out of 200 will be more significant than 15 out of 20. The actual playtime probably should be also taken for consideration as people with experience should be treated more seriously than those who just launched the game and wrote an extatic review based on the fanciness of the main menu. I'll drop the details as the implementation may vary a lot and is a subject for discussion but the idea should be clear.

It's confusing to see the "mostly positive" review rating of 77% and lots of negative reviews with high upvotes just below it.

[Gus the Crocodile]

Weighting by playtime will skew results to the positive; people who have a positive impression of the game are clearly much more likely to spend more time playing.

Can't say, either, that I particularly agree with the idea of unpopular opinions being worth less. That sounds pretty anti-democratic.

[Sytch]

Democratic decisions are always decided by the majority, it's how the voting system works. But it's not binary and I don't propose something binary either. Weighting the score gives the opportunity for every review to participate in the final score, and the more people support it, the higher the weight of it. But it doesn't preempt or replace other reviews, just adds to them.

Most popular reviews are already on the top while new and less popular are somewhere in the bottom and barely someone thinks it's bad or unfair. People aren't interested in unpopular opinions, they want some "average Joe opinion" be it positive or negative. Something that most people share in common. Valve probably should show the recent reviews in the top for 1-2 days, especially for the early access games where everything changes pretty fast so old, negative and heavily upvoted reviews may stay there for ages though they aren't relevant anymore. But that's another topic. Kinda.

I saw negative reviews from players with literally hundreds of hours of playtime. And I believe it's a really worthy review since they experienced most of the things the game has to offer and they know what they're talking about. There are plenty of variables that could be considered in the equation. The review weight may fall off gradually (i.e. multiplied by some coefficient divided by the number of days passed from the review publishing date, maybe with sqrt or ln), addressing the issue described, so older reviews are becoming less important over time. I don't suggest anything specific, no ready to use formulae, just wanted to point out that the current rating system is flawed. And I want to have some local metacritic replacement because it's much easier to review the game right on Steam than login to MC in the browser, and also everyone in Steam may review their games while quite a fraction of the Steam players are registered on MC. There are also curators exist now so it all becomes even more similar.

[IronBack]

personally i noticed some of these reviews refrence stuff that doesn't even work properly in the actual game is there a way to identify people who are in cahoots with the devlopers

[ScarletFameTTV]

Yes please.

[Gus the Crocodile]

Originally posted by rkfg:

Democratic decisions are always decided by the majority, it's how the voting system works.

It wasn't the "majority rules" aspect of democratic voting that I was worried about preserving - clearly, that is the case either way, in some sense. My concern is the "every vote is worth the same amount" aspect, which disappears under your proposal.

Originally posted by rkfg:

Most popular reviews are already on the top while new and less popular are somewhere in the bottom

And that's a significant problem with using the "helpful/not helpful" ratings as global rankings. There is an inherent snowball effect skewing the data away from being an actual representation of what the community thinks of each review relative to each other review - the reviews with the most votes are the most visible, and thus the most likely to attract more votes.

If you want to use the helpful/not helpful data in a fair manner, you would have to have listed the reviews in a random order so as not to bias the sampling, or alternately force everyone to rate every single review (ideally both, because in the latter case you're still going to lose people's attention and thoughtfulness after a while and the snowball effect will remain).

Ideally you would also let people either review or vote on reviews, not both, to minimise variance in the amount of representation each person gets. But I don't think any of this is going to happen, and until it does I think the best thing to do is aggregate on the raw review numbers as is currently done. If people feel a certain game doesn't have enough positive or negative reviews, they can take a few moments to add their own.

Originally posted by rkfg:

People aren't interested in unpopular opinions, they want some "average Joe opinion", be it positive or negative.

Well surely, if the community, on average, doesn't share the "unpopular opinions", then they won't make up a majority of the reviews, by definition. Declaring the largest observable group "unpopular" would be a pretty weird use of language.

Originally posted by rkfg:

I saw negative reviews from players with literally hundreds of hours of playtime. And I believe it's a really worthy review since they experienced most of the things the game has to offer and they know what they're talking about.

I didn't say such reviews don't exist. I said people liking the game more will be more likely to spend more time in it. The existence of outliers doesn't mean that relationship isn't evident.

As it happens I also disagree that high-playtime negative reviews are "really worthy" on the whole. If someone chooses to spend hundreds of hours of their leisure time doing something they don't like, their value system is clearly very different to mine already, so I can't trust their ideas of what's worthwhile and what isn't. Probably the only times I would come closer to putting faith in the values of someone who voluntarily spent that much time not enjoying themselves, is if they were paid to do it so I know they're not just insane. But if I want to read paid reviews it won't be on Steam anyway.

[<™[-=> Cookiez <=-]™>]

Weighted review scores make sense.

However, you can't just weight by number of reviews.

Look at this guy with over 2000+ reviews.

They are all fake.

https://steamcommunity.com/id/kanehero/recommended/1105530/

[𒐪⎝ Epylector ⎠𒐪]

>play time amount should affect how much attention a review gets

Bots would immediately abuse that to play a game 24/7 so they could then post a review taking advantage of all the attention that would gain them.

[cSg|mc-Hotsauce]

Originally posted by <™-=> Cookiez <=-™>:

Look at this guy with over 2000+ reviews.

They are all fake.

https://steamcommunity.com/id/kanehero/recommended/1105530/

Report the spam and move on.

[Start_Running]

Originally posted by Sytch:

As I understand, for now the percentage shows just positive divided by all or negative/all ratio depending on which reviews prevail. It's not right, there may be 10 negative reviews with high upvotes and 20 positive with just a few upvotes, nevertheless positive will win at 66%.

Yeah since what's being counted is not how many people liked the review but rather the sentiment of the reviews.. That's perfectly reasonable.

Instead, they should be weighted. Not only by the approval percent but also by the absolute numbers so that 150 people out of 200 will be more significant than 15 out of 20.

Or we leave the system as it is and not bother with the funky math that basicallyamounts to giving people who have not bought the game a way of influencing the game's review score.

The actual playtime probably should be also taken for consideration as people with experience should be treated more seriously than those who just launched the game and wrote an extatic review based on the fanciness of the main menu. I'll drop the details as the implementation may vary a lot and is a subject for discussion but the idea should be clear.

And this wouuld only sevrve to skew the reviews towards positive since the negative reviewers will generally tend to have less than 2 hours . Becase you know, refunds.

It's confusing to see the "mostly positive" review rating of 77% and lots of negative reviews with high upvotes just below it.

What's confusing. The upvotes are just a measure of people who found the review helpful. What action they took after reading the review is an unknown though. I mark multiple reviews as helpful, negative and positive in many cases based on the information the present.

Sounds like you're trying to make the review system match your opinion of what a game should eb rated. Its the only explanation I can think of for the baffling leaps of logic you're making.

[Peter]

AImost 7 year old thread..

[Niele]

Originally posted by Petey:

AImost 7 year old thread..

And in 7 years, almost nothing changed, except now some people will use the review feature to farm steam points
https://steamcommunity.com/profiles/76561198179415841/recommended/

Will the rate of fake useless review increase over the next 7 years? stay tuned

[𒐪⎝ Epylector ⎠𒐪]

We already have weighted reviews via people marking reviews as helpful/funny or granting awards. And people already can sort by positive/negative to see what each side has to say. Maybe things were a little different 7 years ago, but it's definitely a non-issue as things stand.

[Sytch]

Why would you guys unearth this lmao