Metacritic Analysis

The XeNTaX website has the results of a research project they undertook to analyze the game review data hidden away on the Metacritic website. The results can be viewed in this Adobe Acrobat-format document, which sums things up with this conclusion:
The data at Metacritic leaves much to be desired and seems to be heavily biased. When critic scores do not comply with user scores in the majority of cases, which has been shown in this paper, and the selection of critics is seemingly under no formal control, the validity and accuracy of the data is low. Caution is necessary when using Metacritic information to guide your own decision to buy or not buy a game. Until there is more transparency on how this process takes place at Metacritic, more transparency on the flow of funding from which sources and the observed biases are removed, the database is of limited use for the end-users.
View : : :
18 Replies. 1 pages. Viewing page 1.
Newer [  1  ] Older
18.
 
Re: Metacritic Analysis
Jan 22, 2014, 08:30
18.
Re: Metacritic Analysis Jan 22, 2014, 08:30
Jan 22, 2014, 08:30
 

Metacritic is dreadful but we all knew that anyway. It IS revealing when the professional reviews dont at all jibe with the consumer reviews.

There's no other website that shows that dissonance so for that reason only I find metacritic marginally "useful"
17.
 
Re: Metacritic Analysis
Jan 21, 2014, 03:23
17.
Re: Metacritic Analysis Jan 21, 2014, 03:23
Jan 21, 2014, 03:23
 
Do you even understand how averages are clacualted and the law of large numbers?

So with regards to my OP, QED.
I have a nifty blue line!
Avatar 46994
16.
 
Re: Metacritic Analysis
Jan 20, 2014, 19:02
16.
Re: Metacritic Analysis Jan 20, 2014, 19:02
Jan 20, 2014, 19:02
 
robdot wrote on Jan 20, 2014, 11:06:
I dont know how anyone could put any weight on the user scores at metacritic.
Ditto. The user reviews are primarily fanboys who will love a game no matter what (9/10), or disgruntled players who feel that any imperfection (or change they don't like) deserves a 1.

Most people are merely content or discontent with their games. Which doesn't offer a lot of motivation to write a review. It's a very strict case of the silent majority.
15.
 
Re: Metacritic Analysis
Jan 20, 2014, 17:17
15.
Re: Metacritic Analysis Jan 20, 2014, 17:17
Jan 20, 2014, 17:17
 
Watch gameplay videos and read negative comments (if the forums don't remove them).
14.
 
Re: Metacritic Analysis
Jan 20, 2014, 16:14
14.
Re: Metacritic Analysis Jan 20, 2014, 16:14
Jan 20, 2014, 16:14
 
Spend less than $10 per game and reviews don't matter. Or $20, or whatever your "pocket change" threshold is. Take a shot, throw the dice, judge for yourself.
"We don't stop playing because we grow old; we grow old because we stop playing."
-George Bernard Shaw
Avatar 2718
13.
 
Re: Metacritic Analysis
Jan 20, 2014, 16:10
13.
Re: Metacritic Analysis Jan 20, 2014, 16:10
Jan 20, 2014, 16:10
 
The first ten user reviews are by someone affiliated with the project, the rest are teenage fanboys voting one, or ten.

I wonder if the pro scores would be different if you removed the top 5 ,and the bottom five.

When zombi u came out though, it got screwed by many early reviewers who obviously had not completed the game, and did not understand the game at all.
12.
 
Re: Metacritic Analysis
Jan 20, 2014, 14:27
12.
Re: Metacritic Analysis Jan 20, 2014, 14:27
Jan 20, 2014, 14:27
 
As a consumer Metacritic has been very useful and will remain useful. I go there for the user reviews and as a aggregater for the normal site reviews. I only trust game reviews from trusted reliable sources but the site is good if a title has any red flags.
11.
 
Re: Metacritic Analysis
Jan 20, 2014, 13:35
nin
 
11.
Re: Metacritic Analysis Jan 20, 2014, 13:35
Jan 20, 2014, 13:35
 nin
 
ASeven wrote on Jan 20, 2014, 13:32:
This study is more a condemnation of horrible "gaming journalism" more than it is of metacritic. Creston is 100% on the spot.

Pretty much that, exactly.

10.
 
Re: Metacritic Analysis
Jan 20, 2014, 13:32
10.
Re: Metacritic Analysis Jan 20, 2014, 13:32
Jan 20, 2014, 13:32
 
This study is more a condemnation of horrible "gaming journalism" more than it is of metacritic. Creston is 100% on the spot.
9.
 
Re: Metacritic Analysis
Jan 20, 2014, 12:56
9.
Re: Metacritic Analysis Jan 20, 2014, 12:56
Jan 20, 2014, 12:56
 
Verno wrote on Jan 20, 2014, 11:27:
robdot wrote on Jan 20, 2014, 11:06:
I dont know how anyone could put any weight on the user scores at metacritic.

Metacritic in general sucks. I use it as an index of reviews and for the coming soon features more than anything else.

Everyone's favorite Bethesda actually used it to screw Obsidian out of a bonus payment for New Vegas by one point.
Avatar 15920
8.
 
Re: Metacritic Analysis
Jan 20, 2014, 12:37
8.
Re: Metacritic Analysis Jan 20, 2014, 12:37
Jan 20, 2014, 12:37
 
lol I guess he doesn't know why metacritic became a goto place to read the quick barometer of game quality.

The cock gobbling gaming press.

I'd say most Metacritic reviews are almost accurate outside major AAA games with hordes of morons as fans. They are only off by plus or minus 5 to 10 % and usually in the downward direction.
7.
 
Re: Metacritic Analysis
Jan 20, 2014, 12:14
7.
Re: Metacritic Analysis Jan 20, 2014, 12:14
Jan 20, 2014, 12:14
 
Game play videos and general Forum vibe (knowing what to ignore to)... pretty much all one needs.
Avatar 17232
6.
 
Re: Metacritic Analysis
Jan 20, 2014, 11:27
Verno
 
6.
Re: Metacritic Analysis Jan 20, 2014, 11:27
Jan 20, 2014, 11:27
 Verno
 
robdot wrote on Jan 20, 2014, 11:06:
I dont know how anyone could put any weight on the user scores at metacritic.

Metacritic in general sucks. I use it as an index of reviews and for the coming soon features more than anything else.
Playing: Trials of Mana Remake, FF7R
Watching: Into the Light
Avatar 51617
5.
 
Re: Metacritic Analysis
Jan 20, 2014, 11:14
5.
Re: Metacritic Analysis Jan 20, 2014, 11:14
Jan 20, 2014, 11:14
 
lol
Avatar 57188
4.
 
Re: Metacritic Analysis
Jan 20, 2014, 11:06
4.
Re: Metacritic Analysis Jan 20, 2014, 11:06
Jan 20, 2014, 11:06
 
I dont know how anyone could put any weight on the user scores at metacritic.
3.
 
Re: Metacritic Analysis
Jan 20, 2014, 10:43
3.
Re: Metacritic Analysis Jan 20, 2014, 10:43
Jan 20, 2014, 10:43
 
The data at Metacritic gaming media in general leaves much to be desired and seems to be heavily biased. When critic scores do not comply with user scores in the majority of cases, which has been shown in this paper, and the selection of critics segregation of advertising and reviews is seemingly under no formal control, the validity and accuracy of the data is low. Caution is necessary when using Metacritic gaming media information to guide your own decision to buy or not buy a game. Until there is more transparency on how this process takes place at Metacritic gaming media, more transparency on the flow of funding from which sources and the observed biases are removed, the database is of limited basically no use for the end-users.

There.
Avatar 15604
2.
 
Re: Metacritic Analysis
Jan 20, 2014, 10:25
2.
Re: Metacritic Analysis Jan 20, 2014, 10:25
Jan 20, 2014, 10:25
 
I agree. But there are always people who will scream 'conspiracy theory' and refuse to believe/trust/understand the very simple mathematical process that governs the whole process.
I have a nifty blue line!
Avatar 46994
1.
 
Re: Metacritic Analysis
Jan 20, 2014, 10:00
1.
Re: Metacritic Analysis Jan 20, 2014, 10:00
Jan 20, 2014, 10:00
 
Official game reviews are more suspect than ever, that's why user scores are relevant. Not to say there aren't plenty of gaming companies posting user reviews themselves to up their scores...
18 Replies. 1 pages. Viewing page 1.
Newer [  1  ] Older