You may have read the blistering article at Destructoid about how developer Topware may have used, well let’s just call them ‘interesting’ tactics, to inflate review scores for the RPG Two Worlds II.
I’m not going to rehash the article as you can head over to Destructoid to read it and then onto a few more sites who are covering the story with published emails and all sorts of craziness.
I want to talk about something else, which you can read after the jump.
People like a juicy story like this. Whether or not you care is another matter entirely but any sort of behind the scenes drama…it gets eyeballs. This is particularly the case I imagine because a lot of people already think that a lot of game writers and websites are on the take or are at least heavily influenced by outside sources so this sort of thing just fuels the fire.
I will tell you my own little Two Worlds II story.
As some of you know I run the Mad Catz site GameShark.com. Now, I know GameShark isn’t the name you think of when you think ‘review site’ but we get a good amount of traffic, have assembled a great team of writers, and I deal with various PR people and firms on a daily basis to procure review builds, set up interviews, and so on. I’ve been dealing with PR in some capacity since the mid 90s so I have a pretty fair grasp on how this little dance works.
We get a good number of early review builds from various companies. We have been working on our Fight Night Champion review for a few weeks, Barnes has had Killzone 3 for a week or two, and Brandon has been De Blobbing like mad for the past several days.
But we don’t always get early code like your IGNs and GameSpots and other large, mainstream outlets because
A: We aren’t as high on the pecking list as those sites
B: PR sometimes gets a limited number of review copies to hand out and that have to pick and choose who gets what.
After all I am working out of my house in rural central Ohio. I miss a lot of press tours that aren’t called E3 and PAX. This is why some of our reviews go up on launch day and others…don’t. It’s all part of the process and I have learned to deal with it.
Anyway I tell you all of this because we rarely get Southpeak (the US publisher of Tw2) games early without me sending out an inquiry “Hey we’d like to cover Game X” email.
Southpeak responded with a “We are only giving out review copies to sites who posted screenshots, positive previews, and trailers.”
We didn’t qualify. Oh well. We’ll survive without an early review of Two Worlds: the Sequel.
Still, the Southpeak rep was basically saying in no uncertain terms, “You didn’t scratch my back so I’m going to take my toys and go home.” Again, that sucks, but they can hand out copies to whomever they want.
What is concerning to me — and what should concern you if you read websites to obtain information about a game you are considering purchasing is that we work in a terribly unique field. The gaming press simply cannot function without help from the companies we cover. I can think of very few industries where the relationship is so intertwined. We can’t operate like Consumer Reports even though that sure would make things a lot clearer to the people who should be priority number one in all of this: you guys.
It’s an inevitably toxic relationship that the press has with game companies. It’s unavoidable. The companies want good coverage for its games, and the press wants, above all else, that magic word that every editor has to have in order to succeed:
Access.
Without access you’re cooked.
This is why, in my opinion, the Destructoid article had to rely on so many anonymous sources. No one wants to get their access cut off. It’s our lifeblood.
This also leads me to Metacritic, a site that many gamers rely on to obtain a quick snapshot of how a game is doing in according to various media outlets, of which GameShark is one.
The gaming press uses Metacritic as the Gold Standard upon which to judge a game’s critical success. Nevermind the fact that Metacritic alters review scores to its own grading system and that various PR outlets either don’t realize this or simply don’t care.
Example: Websites that use the letter grading system (1Up, GameShark, etc) will see its score of a “B” metamorphosis into a Metacritic score of a 75. A “B-“magically turns into a 67. A “C” is a 50.
Metacritic isn’t really at fault here even though I would love to see that site at least show what the original score was and not just its self-determined number. The issue is that people see “50” and they don’t think “average” they see 50 and think “whoa, that’s a failing grade. This game sucks” when that’s not what the Metacritic scale or the website scale wanted to convey – worse still, you better believe PR sees a 50 as a failure. It may as well be a 0.
Anything below the “green” score and you are likely to hear about it from PR, depending on the game and the PR person in question. (Not all of them operate like this, so lumping them in one ball isn’t fair.) A yellow or red score on Metacritic and unless you are a really, really big site you run the risk of getting on a PR’s bad side which could then limit your preview access to future games, early review code, and even game convention appointments.
Sound petty? It’s nothing new. Hell the political news scene is the same way and those are supposed to be grown ups covering important stuff, not the framerate of the hottest first person shooter or the dialogue in a shitty RPG.
So, what does this mean to you?
Laughably, very little. In the press and gaming industry in general we tend to wrap ourselves inside a plastic bubble and forget about the people who matter, which is the user. All of this press and insider trading mumbo jumbo doesn’t affect you at all – or at least it shouldn’t.
My advice has always been from day one, when it comes to game coverage, if you honestly read such things to base a buying decision upon then you should follow writers – do not follow websites. Websites are entities. Writers have their own opinions. Leave all of this other nonsense to us and hopefully we won’t forget about the reader in the process.
I read Jim’s article last night, and though how many times I had heard similar stories during my 25 years in the bicycle industry. Any bit of journalism that relies on reviews to pull readers eyes will have this problem. In the bicycle world you’ll start noticing that some places only give glowing reviews. Generally this is the beginning of the end of the cycle.
The cycle starts with a new publication trying to gain readership by having forthright and honest reviews. The positive reviews start to garner readers and advertising dollars. At some point they start to softball the reviews, or drop them completely, to appease the advertisers. You’ll also start noticing full-page ads right next to positive reviews of said advertised product. This is when the readers start departing and bad-mouthing the publication. Finally a new editor, publisher, or PR manager is brought on board and the cycle starts all over again.
I think this same cycle has repeated itself time and time again in many industries. What makes this particular incidence interesting is the publicness even with all the anonymity. Hopefully articles like this keep the debate at the forefront of the minds of the consumer/reader. Speaking about it only increases the confidence readers have with the reviews. Hopefully this translates, if nothing else, into a less sleazy and more open use of PR within the video game industry.
…great advice. I enjoy Game Trailers and find their video reviews superior to IGN and Gamespot’s, but the problem with GT is you never know who’s behind the opinions. They have the same voice announcer for most of their videos. Even if the reviews are informative and well written, I wonder, who wrote this? How familiar are they with the genre? Do we have the same sort of favorite games, or are they coming from a totally different place? I suppose it doesn’t matter if you read game reviews for entertainment, but if you’re trying to decide where to drop your hard earned $60, they’re important questions to ask.
Great insight themanmonkey (first time I have ever typed those words, ever). But you’re right. I even get skittish when sites do the whole “site skinning” ad thing. Gameshark does this too — an outside dept handles that and every time we do it I feel just a little dirty.
I have been thinking about how thoroughly the “cult of the new” has infected both video and board games. (And I’m perhaps far more guilty than most.)
This press manipulation only likely works dependently on first week sales. After that, no one cares.
If you could get people to just refuse to buy a game until one month after release, some of this would go away. But you just can’t, short of defacing every website, gaming magazine, and Gamestop with 10-foot-high “It’s All a Lie!” signs.
…or writing a lot more articles like this. Nice work.
I think that some of what Bill wrote about here are things that people might suspect go on in games journalism, but they’re not really quite clear on how it works. Exposing it with frankness and honesty is one way to squash it.
Reviewing games a month late would never fly in this business. We didn’t get an advance copy of Killzone 3 _three freaking weeks ago_ like all the bigger sites did. I kind of got upset about it, because it was looking like our review was going to be nearly a month behind everybody else’s. Bill said to me “It’s OK, I don’t like reviews that early anyway”. And it sort of hit me that I’d rather wait and play the game without the onus of having this “priveleged” early copy and some sense of favoritism given to these other sites. That being said, we couldn’t do something like a month-late review, at that point you’re really behind the times.
We _did_ get a copy last week so at least we can have launch day coverage (isnt’ that right, Bill?). It came with a bag of KZ3 army men. It did not buy the game an 90+ Metacritic score.
As for the Metacritic thing, one of the problems is that is now a marketing tool. It is a big deal to get the green scores, because that means most likely your game will sell well. Of course, that didn’t work out for Enslaved, but more on that when I do my inevitable “Why Enslaved sucked” article here. Nonetheless, when Killzone 3 hits with an 85 score _three weeks_ ahead of release, that pretty much ensures good sell-through at least for the first week or two. Consumer confidence in the product is built up, and the “wait and see” crowd isn’t waiting for the launch-day reviews that are more common. It’s a marketing scheme, and I’m really glad that we’re not participating in it and instead running what I feel is an honest and on-the-level review of the game. Three weeks after everybody else.
Great article. I’m pretty tired of the sensationalism out there these days; it’s nice to have a level-headed perspective out there.
Your comments about “access” and the toxicity of the press/industry relationship remind me of another beloved past time: motorcycling. It’s arguably even worse than the situation with games journalism. Even if a games journalist isn’t provided a review copy of a game, it won’t demolish a bank account to go buy (or rent) it. Granted, a late review doesn’t get you the same hits and that may, depending on your perspective, bankrupt you. However, with motorcycles, going out and buying or renting a new motorcycle isn’t such a simple proposition.
The advice at the end of your article, Bill, is the same advice I follow there. Find the good writers and listen up. Thankfully, I found a great alternative for motorcycling and it just might be the case that I have found one for gaming here.
I can’t imagine how one could even write about something like motorcycles without having promotional items and press handouts.
As readers here may or may not know, I’ve been reviewing tabletop games for nearly ten years now and I used to come down really hard on the “review copy racket”. Because tabletop games are a much smaller industry, what you had happening was TONS of softball, “nice” reviews and very little honesty. Because folks were trying to get free stuff from publishers. So that they could build their collections, not report on them with integrity.
Eventually, I came to realize that I actually could not do this professionally without getting review copies from publishers. I couldn’t afford it, and the peanuts and used dryer sheets that Gameshark pays me per column barely covers the cost of most games. And since I am a pro writer, I would like to make at least _some_ money. 😉
Anyway, I eventually had to bite the bullet and go around and ask for review copies of games. Now, it’s not a big deal because I believe that I’ve proven that you can take the press and review copies and still be honeset, forthright, and uncompromised.
However, there was an issue last year with Fantasy Flight Games, one of the larger publishers of tabletop games. I’ve become very critical of them over the past two years because of a number of reasons I won’t go into here, but their CEO took it strangely personally and barred me from their press list. Now, FFG is well within rights to do so and I’m not specifically entitled to any of their overpriced games for free. But the fact of the matter is that after covering them for years, giving their games numerous GotY awards, and generally appreciating what they were doing it was not only insulting, but it also sent the message that maintaining a good standing with the company and its press department depends on giving them good press.
So I was actually happy to be rid of them, because I felt like my integrity was compromised by having a relationship with a company that was effectively “buying” good reviews and notices by passing out free games. But now, my coverage of their games is very limited because I simply can’t afford to go out and pay $60-$100 for everything they release.
It is a Faustian bargain to some degree, and I do think that younger or less headstrong writers working in games writing could be easily swayed by all the press events, the handouts, and the perks. But your point about review copies and the potential expense of this kind of journalism really hits home for me- because I actually need them in order to do my job.
So I’m curious, is there a ‘barometer’ of how confident the company is that the game will be a critical success with how they approach a reviewer?
Thanks Bill, hopefully that makes up for my forgetting who watched “Red Dawn” so much I got called out on the following podcast. I know what you’re saying about the obvious plugs for games, it’s stuff like theme skins that made me stop going to 1UP, Gamespot, etc well over a year ago. One of the real measures of being a true professional is learning to be honest about your alliances and conflicts.
You know what’s really funny? On the tabletop side, it’s usually the publishers of worst games with little to no commercial prospects that approach reviewers. Usually, I’ve got to go out and email everybody and beg.
I don’t know if there’s an actual barometer, but like with Killzone 3 that’s a game that’s going to get a 80+ Metacritic score. Period. The budget and talent are there, the fanbase is there, and the marketing is there. So it’s “safe” for them to send it out to reviewers early. A game like Two Worlds II is WAY riskier given the reception of the first game, the developer/publisher reputation, and the overall low volume of hype surrounding it. Which really means that they should beat the stump about how different the game is (apparently, it’s really funny for example) and how it offers an alternative to other games in its class. Instead, they do this shady shit and it sabotages their whole brand. Which means Two Worlds III is immediately tarred with this kind of reputation. As is anything else they put out.
I doubt Kane & Lynch, for example, will ever recover from that fiasco. And there was that business with Arkham Asylum that mostly got shunted off because the game was so damn good, but it was still impropriety.
Has Metacritic stopped adjusting scores depending on the average score the reviewing site awards? If that is not the case, and they do still adjust via mean score, regularly giving 80+ reviews in order to maintain “access” would seem to be a self defeating proposition.
Eventually you’ll have to give out a 90 something just to keep it green. Alternately I guess you could pick unknown / non influential publishers / developers and regularly give them crazy low scores, but somebody would pick up on that… right?
Concerning previews, and even day 1 reviews, I’ve always tried to extract the relevant data (sub genre info, gameplay mechanics, etc.) from the opinions presented, as I realize that the writers of such articles are highly motivated to present the game in the best possible light.
I may be in the minority, but I rarely buy a game that hasn’t been out for at least a month, and if you averaged the ages of the games I’ve bought over the course of the last 12 months it’d certainly total up to at least a year, probably several thanks to a few outliers from GOG.com.
Also I regularly call bull on Game Informer’s previews to my friends who are exposed to games journalism almost entirely from that one source, due to its being free, (actually they pay you to sub it don’t they.) Maybe not as good as an “Its a LIE!” sign, but I try.
One of the reasons that I _do_ think of Game Shark when I think “review site” is because you allow writers to have their own voice. The idea of hitting a magazine or large site is sort of laughable to my workflow, but I _will_ keep an eye out for certain names. It would bore you for me to name everyone on my mental list, but I can say that everyone who writes for No High Score is on it.
Even that, though isn’t enough for me to get past the introductory paragraph. Certain sites (I won’t name names…) pound their copy into powder and the writer’s own voice is dimmed. Who wants day-old jalapeño? Game Shark attracts high-quality writers and lets them do their thing. Thank you for that.
I appreciate these types of articles and would like to think that while it may not change the state of the industry, it may help those relative few that can see fairly objectively to take notice and change their habits in some ways. Perhaps that’s too optimistic, but it’s what brings me to places like this in the first place for coverage. Thank you!