In an effort to prevent developers from artificially inflating review scores on Steam, Valve has changed the system so that reviews written by users who received a game through a developer or CD-distributed license key no longer count towards the review score. The change also affects reviews from Kickstarter backers who receive a Steam key as part of their backer rewards.
The Steam rating system was updated in May to display an overall rating (positive, negative, or mixed) at the top of each game page. The idea was to better showcase a game’s current state, especially as more of them get early access and evolve over time. Unfortunately, Valve claims that many developers have abused the system by handing out keys to friends or paid review services to bolster the overall review score.
“The review score has also become a point of fixation for many developers, to the point where some developers are willing to use deceptive tactics to generate a more positive review score,” Valve wrote in a blog post.
“The majority of review score manipulation that we see by developers is through the process of handing out Steam keys to their game, which are then used to generate positive reviews. Some developers organize their own system using Steam- keys on alternative accounts. Some organizations even offer paid services to write positive reviews.”
Valve estimates that “at least 160 titles have a significantly higher percentage of positive reviews from users who activated the product with a CD key, compared to customers who purchased the game directly on Steam.” While it admits there could be legitimate reasons for the discrepancy – including a large audience outside of Steam and popular Kickstarter campaigns – for the most part it says “the abuse is clear and obvious”.
Valve will not be removing the ability for developers to issue Steam keys. But in cases where developers are found to have duplicated or generated reviews in bulk, or found review accounts associated with the developer in question, “it will end business relationships.” Gamers who purchase or receive a game outside of Steam can still write a review, but it will no longer contribute to the overall review score.
“This does mean that the review score category displayed for about 14 percent of games will change; some up and some down,” continues Valve’s blog post. “Most changes in the review score category are due to games being on the fringe of the review score, such as 69 percent positive or 70 percent positive. A 1 percent change in these cases can mean the difference between a review score category of ‘ Mixed’ and ‘Positive’. Approximately 200 titles that have had only one or two reviews will no longer receive a score at all until a review is written by a customer who purchased that item through Steam.”
While the change is largely positive for consumers, developers who have large fan bases outside of Steam or rely on crowdfunding could be negatively impacted. Valve is also working on other issues, including a discrepancy between the most helpful reviews and the review score, which could be caused by users deliberately trying to skew the system, or where reviews are useful “simply because they’re funny”.