I take no pleasure in writing this. David Braben developed one of the essential games from my childhood. Braben was the man behind the classic Elite, the way ahead of its time space trading sim which was released in 1984. I was 12. All of these open world games where you have freedom to do what you want, when you want, go back to Elite. I spent more time playing that game than I care to admit.
But Braben said something really silly earlier in the year and tried to “clarify” his thoughts at the BAFTAs in London.
Yes, this is another Metacritic story. I don’t post this to give David a virtual noogie, although he clearly deserves it, but I really want the readers of our little ol’ blog to understand where I’m coming from when it comes to Metacritic and how people look at it and some of the insane ideas that people inside this industry come up with when it comes to the aggregate site and how it relates to people like us who are trying to do a job.
So off we go…
Before Braben backtracked, his original statement was this:
“Most reviewers are excellent at what they do, and it is a very hard job with, frankly, little glory. As an industry, there is something we could do to recognize this – effectively a Metacritic for reviewers.”
So far, nothing too crazy. In truth, I have always found it unnerving when reviewers don’t take their job seriously, and I know plenty who don’t, or when a game site assigns a review to someone who isn’t qualified to do it. When you read a review that starts, “I normally don’t play/like this genre, but here’s my 800 words on the subject anyway. You can trust me. Really. (Hey how many points for a made basket again?)
When those reviews get lumped in on Metacritic with those from qualified writers, it’s rather unfair to the reader and to the game. So go on David, you have my attention.
“The best reviewers give spot-on reviews pretty soon after a game is released. They do not wait to see what others say, but nevertheless consistently come very close to the final average score. There could be a prize for the best each year. Don’t forget – this is not intended to influence reviews – just to encourage and reward consistency – as it is not a high reviewer that gets the reward, it is the one that gets the best result.”
Whew, a lot to digest there.
First off, let me explain how this works.
The quality of the writer has nothing, at all, to do with whether a review is posted first. We have all read reviews from major websites that are published on day one that make us turn our heads like a dog that hears a funny noise. The size of the website has more to do with the speed of the review than the awesomeness of the writer or freelancer. So that’s just wrong.
In addition, there is never, ever, a “right” score for a game. Ever. We’re talking about a lot of variables here: the scale of the website, the influence of the writer, so on and so forth.
I gave NCAA Football 10 a “B-” grade, which in truth was being generous. This was turned into a 67 on Metacritic. That review was posted soon after release and I stand by that article 100%. I feel I was fair, honest, and wrote my ass off, thank you very much.
The overall score for NCAA 10 was an 83.
See here: http://www.metacritic.com/game/xbox-360/ncaa-football-10/critic-reviews
See that 67 at the bottom?
If you want to use the Braben System of Reviewer Judgment I’m just some kook who should get a real job. I’m also apparently an idiot because I missed the average score AND an opportunity to earn a prize!
Braben basically wants reviewers to conform to one big, easy to digest opinion and if you review a game a week or two late or you don’t agree with the masses you’re not good at this whole “being a critic” thing. Bad idea, David. On a whole lotta levels.
Apparently, what sparked this was, according to Eurogamer:
Braben’s suggestion followed musings over the problems that face developers of children’s or casual games. He argued it was often difficult for such a title to get a fair, balanced hearing from ‘core’ orientated reviewers.
I think that’s fair — to worry about that sort of thing I mean. I know I wouldn’t want Barnes reviewing my brand new fluffy animal Candyland game either. I shudder at the thought. But this isn’t the answer, either.
You want to get your kid’s game out to the right reviewers? I have a suggestion:
Ask the editors.
We listen. Honest.
If you have a niche game, and developers of wargames have dealt with this for *years*, you ASK if they have anyone on staff who has familiarity or a real interest in the product. If you just blind send a game to a website there’s a good chance, if it’s a kid’s game or a movie tie in game it will get assigned to the “B Team” — that’s for the new guys, or the guys editors rarely give AAA reviews. (Yes, pecking orders exist; it’s how Brandon gets all the bad DS games.)
Do you see reviews of niche games like text based sports games, or hardcore wargames on every site? Of course not because you have to have faith that the website will review it with some level of competency. If you don’t feel that way — don’t send them a copy. Done and done.
But “grading” critics based on how close they follow the average score on Metacritic and on how quickly they post reviews is a terrible, terrible idea.
Now onto the backtrack.
Braben tried to qualify his statement by saying:
“I think a lot of people assumed I was criticising journalists, which wasn’t the intention.”
No, that’s not what I thought.
“What I was really saying was that actually, it’s a really hard job. Because if you want to try and make your reviews stand out, you end up producing something that, with time, will make you look silly. So, in actuality, hitting Metacritic early on is a challenge, and I am impressed by the writers who manage to do it. Essentially, yes it’s a very hard job and I apologise to those who either misunderstood what I said or disagreed. I’m happy to debate with them if they want at some time!”
Anytime, Mr. Braben.
(Elite was awesome btw! xoxoxo)
Lastly, Marc Doyle, co-creator of Metacritic and a man with whom I have exchanged several emails over the past year, and a guy I happen to like quite a bit, said this which made me smile:
“Penalizing a brilliant critic who happens to utilize the lower end of its publication’s scale more often than a middling critic who never gives lower than a 6/10 doesn’t make sense to me.”
Good on you, Marc.