new post on the Steam Blog
has further thoughts from Valve on the process of
refining the user review process on Steam to make them more useful to other
users. They offer an extensive discussion of what makes for helpful reviews, and
how they evaluate reviews for this quality. They note that one of the problems
they must deal with is that the system for reviewing reviews itself is as
subject to abuse as the review system itself. Here's word on what they are doing
What we are changing
Up to this point, we've tried to maintain just showing the simple math behind
how we calculate whether a review is helpful or not--the percentage and number
of people who indicated a review as helpful. We like systems that are
transparent and easy to understand, as they are also easier to believe and
trust. Unfortunately, this has resulted in a system that allows a small group to
manipulate reviews to a degree that is clearly decreasing the value of Steam for
many other players. So we're making two main changes.
- Firstly, our system will use a new method of
calculating the helpfulness of each review, taking into account the users
that are trying to manipulate the system. One way we're doing that is by
counting the helpful ratings on reviews differently for users that are far
outside the norm. Ratings from users that follow normal patterns of rating
will continue to be counted the same way that they have, whereas accounts
that rate an excessive number of reviews on an individual game will see the
weight of each individual rating count for less and less.
- Secondly, store pages will now show the default
helpful positive and negative reviews in a similar proportion to that of the
overall review score for the game. For example, if the game is reviewed
positively by 80% of reviewers, then the ten reviews shown by default on the
store page will be 80% positive, showing eight positive and two negative.
This should keep the reviews shown on a game's page from being so easily
manipulated by a few determined players and should more accurately represent
the overall sentiment of the people playing the game.
We're rolling out both of these changes as a beta today. You can turn the new
method on and off to see how it impacts the default display of reviews on any
given store page. Note that these changes only impact the default listing of
reviews (Called "Summary") and the "Most Helpful" display option.