Evidence-Based Design

This is creepy.

This is funny.

While it’s old news by now, this article on the lack of social play in Warhammer is very good. Conan had similar issues, as I wrote about earlier this year. While encouraging social play isn’t as crucial in a PvP game with prebuilt sides, it’s still sad to see any MMO deny its fundamental nature.

Speaking of PvP: in the midst of a discussion on PvP itemization in World of Warcraft, some talk on tracking player performance in battlegrounds. I don’t know what Blizzard’s planning, but I hope they haven’t forgotten that a bunch of people have been trying to solve this problem in other contexts.

Getting back to Warhammer, gamerDNA’s been printing some interesting articles based on Xfire data and surveys. If you’re not familiar with them, they’re a data mining company pretending to be a social networking company, so we can expect lots of intesting stuff from them in the future. Subscribe to the RSS feed if you haven’t already.

In an oddly similar direction, this NYT article suggests that people are eating better food these days because the NPD says so! Somebody call the Daily Plate and Sparkpeople and Fitday and hell, Weight Watchers Online, and let them know about their exciting new business model.

In addition to your friends’ scores, Rock Band song leaderboards provide some interesting business data.

Every time you play a song in Rock Band while connected to Xbox Live, your score is applied to the song’s leaderboard. What’s interesting about that is that the number of entries on the leaderboard can tell us how many users have played the song. And what’s interesting about that is that it tells us how many users have purchased the song, if it was downloadable content. (Not counting people that purchased the song and never played it, and not counting if the users-played count is over 100k because the leaderboards don’t go that far, but hey.)

Rock Band DLC Statosphere adds it all up.

skedastic of the Qt3 boards massaged the numbers a bit to remove time factors and to group by artist. There’s some debate in the thread over his statistical methods. I would like to debate because Rush and Nine Inch Nails are relatively low, and goddammit, I want more Rush and Nine Inch Nails.

Am I not getting more Rush and Nine Inch Nails because the team can see that they’re poor investments? I don’t know, but I haven’t seen any patterns in the DLC releases that would imply they’re trying to provide more songs by proven bands — yet. I wonder what their lead time and content production pipelines look like. Is it harder to get assets out of studio musicians than it is artists?

In the future, you’ll be able to store saves for your Steam games on Valve’s servers.

I don’t know how often people play the sort of games they’d want to save on multiple machines — in other words, narrative games as opposed to multiplayer. I can understand burning some time at a friend’s house with a fast game of Some Multiplayer Shooter, but I don’t think I want to sit down and pick up a story where I left off. Narrative games would seem to lend themselves to longer and less social sessions.

But since Valve knows what you’re doing all the time, they must know how often people play the same narrative game on multiple machines, so maybe they see some demand.

Or … maybe there’s another reason. What can they data mine out of save games that they don’t already know? They already know how far you’ve gotten — if that’s what “Highest Map Played” means. There must be something interesting in there …

Our latest dev blog talks about our iterative design process here at Spacetime, including an application of a simple data mining system.

Gamasutra has a brief article on production-phase metrics for single-player games at Bioware. I don’t know how true this is of the rest of the industry, but it sounds like theirs is driven by QA, rather than design.

Ideally, QA should feel comfortable giving subjective design feedback to the team — walking across the office to tell the designer that the new minigame sucks. Unfortunately, designers aren’t always willing to listen. Walking across the office with a printout of this chart could be more successful.

That kind of stupid designer pride leads them to be unwilling to back metrics initiatives or to listen to community feedback. Grow up and take advantage of the tools available to you — they make your job easier.

Via a colleague at NCsoft, Nielsen is yet again trying to get into game demographics to sell to marketers. They’ve tried this before, but failed. The old plan, in addition to the antiquated diaries they already use for television :

Nielsen intends to roll out two products this year, the more formidable one being a “tag” that PC, console, and online game developers can build into their software to be used by Nielsen to measure all sorts of in-game activity, especially response to advertising. This includes how people navigate through games, what levels they reach, and how long they spend on each level.

But, because tag placement requires the participation of game developers, Nielsen doesn’t expect the first tagged game to be released until the second half of 2005.

I suspect this also failed because the developers didn’t give a shit about “[measuring] all sorts of in-game activity.”

The new plan is a People Meter-style “audio scanning device.” I didn’t know how People Meters work, the devices they use to track television habits outside of the diaries, and Wikipedia’s article sucks, so I Googled it. Here’s a description of a portable one.

The Portable People Meter, developed by Arbitron Inc., is a pager-sized device that is carried by a representative panel of television viewers. It automatically detects inaudible codes that broadcasters embed in the audio portion of their programming using encoders provided by BBM and Arbitron. At the end of each day, the survey participants place the meters into base stations that recharge the devices and send the collected codes to BBM for tabulation. The Portable People Meter can measure exposure to any electronic media, which has audio that can be encoded – television, cable, and radio, even cinema advertising and in-store media.

I am not making this up, and Nielsen is planning to do the same thing with games: “the equipment detects each game’s unique ‘audio signature,’ compares it to the reference library of audio signatures compiled by Nielsen, and determines what games are being played when and where.”

Alrighty then!

They’re also getting data from Sony, which is much less bizarre: “which Sony games are being played, by whom, and for how long — ‘to enhance the data we are collecting and to give us the opportunity to take a hybrid-based approach to measurement.’”

They’re not specific, but I imagine they’re talking PS3 data. I’ve curious about how seriously Sony and Microsoft are taking their mountains of free data from online consoles — really, the Xbox 360 knows everything I play, for how long, what I do in game, when I log out, what TV and movies I watch, which DVDs I purchase. One would like to think that they’re using it to make the games better, instead of selling it to marketers so we might one day enjoy the wonders of dynamic advertising in everything we play, but hey.

Most World of Warcraft server forums maintain a list of which guild has gotten how far in each noteworthy raid instance. Here’s one, for example. They require a representative from the guild to post every time the guild manages to down another boss.

Via the Elitist Jerks forums comes an automated guild progression site, WowJutsu. According to the FAQ, it runs a script on the Armory and looks at guild members’ gear. If it sees my character wearing a drop from a specific raid boss, it assumes my guild has killed it.

This isn’t entirely accurate — at the low end of the raiding scale, players sometimes participate in multi-guild raids. Sometimes players don’t log out in their raid gear. The layout is cool for top-tier guilds with more than one raid icon to show, but for most crappy guilds on most crappy servers, most people only care how many bosses they’ve killed in Karazhan, and that requires a mouseover. The individual guild reports are pretty cool, though.

I was going to rant here about WoW guild recruitment tools and the lack thereof, but I’m just bitter that I ended up in a terrible guild when I was shopping a few months ago. Clearly, I didn’t read the progression thread. And while some kind of raid epeen rating in an EQ2-style in-game guild recruitment tool would make things even harder on these crappy little guilds, I sure wish that I would have had some more obvious warning.

– Edit, July 5
Brandon has some discussion on the percentage completion stats on the main page’s sidebar.