The Critic and the Cloud

With GPS records, computer stats and timestamps in hand, they will eventually strike. They’ll comb through the complaints and the cries until they find a mismatch, and then they’ll issue a statement.

The critic never drove to that point, they’ll say. He did not traverse the highway mentioned in the third paragraph.

And he most certainly did not toggle the hardest difficulty setting.

What a terrifying back-and-forth between a New York Times car critic and the spurned CEO of Tesla Motors this week. As each side argues over a negative car review’s truths, the Internet watches with popcorn in hand, gobbling up the “gotcha” attempts while picking sides. But I’m not enjoying it.

All I can think is, God, what if that ever happened to me? And it very well could. Really, Tesla’s show of defiance against the enthusiast press has shown a lot of manufacturers’ hands. Product makers already have the data – mountains of it – for exactly this kind of pushback…perhaps none more than video game producers.

———

I am, among other things, a guy who gets paid to review games. Sweet gig, right? I swim in new titles, typically before they reach store shelves, and as far as other people can tell, all I do is describe how awesome my playtime was.

Far be it from me to complain about my line of work. It is indeed “awesome” (sub “bitchin’” or “righteous” as needed), and I have busted my ass for over a decade to call it a bona fide job. But sometimes, a game to review lights a bundle of dynamite. It might arrive barely a week before launch, especially if my outlet hasn’t been deemed a priority by a publisher, and yet I’m expected by editors to pound out a proper, top-to-bottom review within a brief “launch window.”

For perspective, consider film critics, who only get one viewing and a few days to toss their thoughts in the dryer; even that can feel like too little time to make sense of 94 minutes of Bradley Cooper. Comparatively, an average game requires 8 hours, at the very least. That number typically climbs, whether thanks to optional content, multiplayer testing or non-content disruptions like frustrating challenges or required grinds.

None of that should shock any reader of gaming site Unwinnable – nor should the fact that critics deal with those situations in wildly different ways. Those ways often speak to their critical voice.

In some cases, my style of review would be decimated by an angry, data-wielding CEO. Here’s an example: in February 2012, I reviewed quest game Kingdoms of Amalur: Reckoning without beating it.

EA could very well have dug up my Xbox Live data and responded to my review with the following truths: I only unlocked 24% of the achievements, and I had a six-day gap between some of those achievements. Over a year since the game launched, I can’t even check to see how much I leveled my character or my total progress in the game, as I gave the game away weeks after my review ran. But by my recollection, I got through roughly half of the story content – and nearly all of the side quest content up to that point – at about 30 hours of play.

———

No great expose came and went; no blog post lifted my career and tossed it upon a decimated trailer park. Possibly because EA didn’t bother checking critics’ playthrough stats, or because my review was largely positive.

But what if someone attacked me with a data-driven gotcha? Never mind that I believe quest games should be consumed slowly, the way busy adults (my target reader and expected demographic at that outlet) play them, rather than gorged upon; never mind that I lost a lot of time to side quests, which, as I said in my review, Amalur practically forces its warriors to complete, lest they fall behind in the main questline; and most importantly, never mind that 30 hours made abundantly clear both the game’s action mechanics and its storytelling techniques.

Instead, my review would be bludgeoned with a “did not finish!” asterisk.

Truth be told, my personal bias tilted a little weaker than my review’s score, 8/10, may have illustrated. Which is why I hate numbers in reviews. Indeed, Amalur was an 8/10 experience: its huge, beautiful world exploded with a smooth combat system, satisfying upgrades and a more-than-serviceable plot. The game fulfilled the cost-to-content equation, and with polish. But badly-paced side missions interrupted the story’s flow, and the game’s strides had already been topped by better quest games only months earlier. Those are truths reflected in my descriptions and final judgments – and the reasons I didn’t really go back to the game after reviewing.

None of that nuance exists in the cloud, and I’m terrified that I will one day ever have to defend a review; I’ve always believed that my byline is defense enough, which is why I very rarely reply to comments. (Just to cover all of my bases: I fully stand behind my Amalur review, and in no way have I ever produced an insufficient, misleading review.) But rest assured, before long, a game critic’s feet will be put to the fire either by a publisher or a sleuthing fan. An achievement or a leaderboard score somehow won’t match the text, and the pitchforks will come out alongside accusations of an agenda.

Unless such controversy proves that a critic flat-out lied – for example, criticizing a specific moment without proof that he/she got that far – I’ll rush to his/her side with a giant shield. Critics deserve space to play games at a reasonable pace. To not reach the very end of a massive game (or unlock multiple endings), so long as the criticisms don’t hinge on its total narrative arc. To not always test the easiest or hardest difficulties whenever they prove irrelevant. Quite frankly, to quit playing a game early if its mechanics have been laid bare and its frustrations proven wholly debilitating. So long as those choices match the typical audience of an outlet, and are addressed by the text (which is tough to do with an itty bitty word count), then they’re valid.

This will only get worse as the world of games and software shifts to an always-online, cloud-dependent model. If the code lives on a server, so will the subjective experience of a pre-release critic. Game companies have already trumpeted what they can do with anonymous usage data, from pushing in-game ads to “perfecting” motion-control add-ons. Their mountains of data will only grow, and while the average fan may merely feel creeped out by the trend, the critic-publisher relationship will only grow further strained by it – unless game makers learn early on to let go.

It’s not just producers who should cool their heels on wielding data, Tesla-style. Magazine and website editors may soon also be tempted to peruse their critics’ achievement lists before printing a review. It’s not a far-fetched idea. Modern, online games, more than any media, leave a breadcrumb trail of their players’ progress, and fact-checkers never want to be caught with their pants down.

To them, I insist, stick to the critics’ words. A lousy critic will run full-speed through a game to reach what seems like the ending, ticking every achievement along the way. A real critic will drive a game the way his/her readers would – poking their heads in every corner, spending more energy on the moments that prove the most exciting and revealing — then leave the place behind once it’s been set on fire, 100% completion be damned.

———

You can find Sam Machkovech on Twitter, giant shield in hand, @SamRed.