New Steam Review Format

[w@ke_]

I noticed this morning that Steam is separating user reviews into "Overall" and "Recent" categories, with a review breakdown for each. Seems like a very smart move to me. It's a service to developers who do a good job supporting and improving their games post-launch (and an incentive to do so, too!), and it's a service to players who can see how the game's progressed. I know if I check back on a game I was on the fence about before, and I see a big improvement in recent vs. overall reviews, I'll be a lot more motivated to buy it.

What do other people think?

[Crunchy]

I did notice that it change but i did not notice this. Yes i think its required. Should not they be able to link the review to the patch level of the game at the time of review?
That would seem to make more sense to me.

I generally find this a drag as i have a very difficult time finding where the current state of the game fits in with the reviews.

Some games, like maybe ck2, appear to have gotten worse with patches.
Many of the neg reviews there seem to indicate that some patch ruined the game but i dont really know where that is.

This issue is not limited to steam of course but steam doesnt have much of an excuse here other than maybe lack of resources to make a robust review system.

Reviews could be automatically linked with the patch level of the game at review. The ratings could break down by patch level and use a current patch level rating and an overall.

I dont think they want to.
Like maybe stepping on the toes of other review systems? A game platform monopoly?
maybe if their API was open enough to allow aggregation of their data?
read the dates of all the patches and link all the reviews by dates to the patch periods.
then use time played and if possible show people who have completed the game.

maybe a sort by number of achievements.

Sorry going off rails. But basically the review system is just a big bunch of yelling.

I almost always weight the negative reviews higher unless they are filled with stuff unrelated to actual gameplay which happens often enough. I generally ignore those.

[Gregorovitch]

When I read about thisd I thought yeah, this is a stonking good idea: games hyped to the statosphere that turn out meh will get trimmed, games with problems on release the devs got sorted out will get upgraded, games with more substance that immeidately apparent will get up-voted and so on. All good stuff.

So a looked a small totally unscientific sample of games I play or know a bit about and I don't really know what to make of it now:

PoE...............88%[7k] / 88%[236]
W3.................95%[35k] / 97%[2.5k]
FO4...............80%[61k] / 78%[5k]
Civ5...............96%[76k] / 95%[2.1k]

So...these games are thought of pretty much trhe same now as they were on release. Fine.

CK2...............92%[15k] / 86%[400]
EU4...............92%[17k] / 84%[1k]

So what is going on here? I know that both games are much better now than they were on release, so why the down votes? People coming in late and spending a lot on DLC thinking they ain't really all that for the money? People who don't like Paradox DLC policy flaming the review sections? Late comers disproportionatly not really into this complexity of game? Who knows, I'm not going to read thirty thousand reviews to find out.

Problem is this is the wrong answer, but it isn't possible to say if this is a problem with the new Steam review system or perhaps with Paradox's DLC policy. More importantly I now know I can't trust these "recent review" figures to do anything other than mislead me.

SD2..............65%[1.3k] / 45%[77]

This one's a shocker. Informed opinion seems to be SD2 has been improved considerably with the recent DLC. Perhaps the DLC was released too late to have any effect on the recent review figures, I don't know. Whatever, I imagine SD2's connections are spitting blood over this and I wouldn't blame them. You just release a new DLC that improves your game considerably and this is what you get?

So, yeah, my initial enthusiasm for this initiative has taken a significant dive - too many anomolies that are too misleading and potentially unfairly detrimental to small devs whose games don't get tens of thousands of reviews to smooth the numbers quickly enough to get a meaningful "recent reviews" score.

[w@ke_]

I wondered about that too, but I think it may be a self-correcting phenomenon. There are definitely times when a game gets a badly-needed update but it pisses off a vocal minority, or otherwise introduces some growing pains. That can tank a score, but only for a period (unless there's a really dedicated hate-brigade). It sucks for the people who form their impression in the middle of the mess, and it sucks for the studio. But I'd be surprised if it had a lasting effect on the recent score. Hatedoms often – but not always – seem to exhaust themselves after a period.

I could be wrong, though.

[Mezmorki]

SD2 gets worse .... look at the scores for Sector Zero.

Overall: 84% (very positive)
Recent: 66% (mixed)

The problem I have with this new system is that while it's providing another data point it's also making the scores more volatile and fickle. If a bunch of gamers design to hate-vote on a game (like they often do on metacritic) now you just need a bunch of the "recent" votes to be negative to show a huge overwhelmingly negative result. Seems prone to abuse, potentially in both directions.

[Gregorovitch]

Originally posted by Mezmorki:

If a bunch of gamers design to hate-vote on a game (like they often do on metacritic) now you just need a bunch of the "recent" votes to be negative to show a huge overwhelmingly negative result. Seems prone to abuse, potentially in both directions.

Good point. I'm beginning to wonder just how much thought went tinto this.

[Crunchy]

Need the number of reviewers at the time.
For sd2 its really hard to judge as there are plenty of people that vote neg for non game issues.
I think at sd2 release plenty of people were happy about the game as it had appeal that did not get overshadowed by drawbacks until mid to late game. hours and hours of play.

and go read the negative reviews for EU and ck2. at least ck2 and i think the same issue exists between the two.

They introduced some mechanic in one the late patches that people appear to hate. Apparently it patches the base game so the issue is unavoidable.

[Crunchy]

It also seems that ck2 gets a whole lot of recent hate due to the DLC path.
That also appears to be the neg reviews for EU4

[Troy]

Originally posted by rogue_LOVE:

I noticed this morning that Steam is separating user reviews into "Overall" and "Recent" categories, with a review breakdown for each. Seems like a very smart move to me. It's a service to developers who do a good job supporting and improving their games post-launch (and an incentive to do so, too!), and it's a service to players who can see how the game's progressed. I know if I check back on a game I was on the fence about before, and I see a big improvement in recent vs. overall reviews, I'll be a lot more motivated to buy it.

What do other people think?

This is the sort of thing the AppStore and other online retailers use too. I think it's a good improvement. It will be interesting to see what happens to new games once they get a half a year on down the road.

[Crunchy]

here is a question i have...
Do these review counts have a real effect on a studio?
Such as the relationship between publisher and studio.

[Troy]

Originally posted by Crunchy Gremlin:

here is a question i have...
Do these review counts have a real effect on a studio?
Such as the relationship between publisher and studio.

The reviews affect sales. Sales affect everything, so I would guess it would impact those kinds of relationships.

[Gregorovitch]

I understand a rule of thumb prevelent in the industry regarding Metacritic is that every point over 80% your game gets equates to ~100,000 additional sales. I see no reason Steam scores would be any different.

[Nasarog]

I can see where this could be abused, but something like this is needed.

[Hans Lemurson]

Are there any games where the Recent review score is noticeably better than the total? Or is it a general trend that review scores get more negative over time?

[BlueTemplar]

Sector Zero was released more than 30 days ago.

[BlueTemplar]

Originally posted by Gregorovitch:

I understand a rule of thumb prevelent in the industry regarding Metacritic is that every point over 80% your game gets equates to ~100,000 additional sales. I see no reason Steam scores would be any different.

Kind of, though it's not a linear relationship :
http://arstechnica.com/gaming/2014/04/steam-gauge-do-strong-reviews-lead-to-stronger-sales-on-steam/

The median game that receives a score of 70 on Metacritic sells about 89,000 estimated copies on Steam. At a Metacritic score of 80, the median sales go up to about 182,000. At a Metacritic score of 90, the median game skyrockets to just over 803,000 copies on Steam.