Evidence-Based Design

I’m home with a cold. Thanks a lot, fellow AGDC attendees, for getting me sick. I’m home drinking Nyquil on this fine Monday morning. Maybe too much Nyquil.

I got up this morning to email my boss and send out a few mails I had planned to take care of at work this week, one of which was to mail out a link to an interesting website a World of Warcraft player put together: Changes from Live Patches to World of Warcraft Classes, via the Elitist Jerks forums. He’s listed every class change made since the beginning of recorded history (i.e. early beta), and he’s categorized it by buffs, nerfs, bug fixes, changes, new features, and overhauls. (If you read the Elitist Jerks link, you’ll see that he’s been revising the classifications with their feedback.)

As I reviewed the list, I realized that if I sat back and squinted, I could see patterns in the category colors, and it occurred to me that I could write a Perl script to parse the data and chart it. (Thanks, Nyquil!) But I’m home, and I don’t have Office on my new desktop, and my laptop’s screen is broken. Enter Google Docs and their (new-to-me) charting feature.

Class-specific charts after the jump.

(more…)

From the “I don’t know what it means, but it looks cool” department:

• Superbowl win probability by betting activity. (In baseball, they do this with historical data. In prior games when the home team’s been up by 2 in the 5th inning with no one on base and no outs, how often have they gone on to win? 83% of the time in the last five years!)

• Superbowl strategy metrics. The writers’ company is applying the aforementioned historical data method to football coaching decisions.

After a year of the Station Exchange, Sony released a white paper to tell everyone how much money they made. Here’s the actual .doc. (Warning: really awful 3D charts!)

There’s a brief section on differences in play patterns between Exchange servers and regular servers. Characters on the Exchange servers tend to have more money.

One result of the easy availability of money on The Bazaar is that players own more in general. The broker board on The Bazaar shows double the number of sales for house items and triple the number of sales for house pets. Perhaps players on The Bazaar have more cash, but nothing much to spend it on.

Huh.

The author also discusses race and class distribution in sales, but doesn’t say if they adhere to actual play distribution. The designers should be keeping an eye on play distribution, looking for balance issues; I wonder if Exchange data is more helpful. When the author says “warriors, summoners, rogues, druids, clerics and brawlers are among the classes sold just once,” I, as a designer, would be thinking there’s a problem. (I’m just talking; I’m not familiar with EQ2’s class balance issues.)

Sony collects a flat listing fee of $10 for characters, and when the character sells, they collect an additional 10% of the final price. The author is proud to point out that characters are worth a lot of money. “Of the top 20 auctions of characters, none was for less than $1,000. The top four character auctions were worth $2,000 each.”

One of the author’s final conclusions:

Station Exchange is not an extension of game play. It is a utility. It offers a fundamentally different approach to play: a means of skipping the boring parts.

That’s the most interesting part of the whole paper. He’s admitting that they have a financial incentive to create a boring game, and that’s horrifying.

Via Kotaku, Information Architects’ Web Trend Map 2007 is a website network presented as a subway map. Go look.

More colors does not equal more cool, largely because it does equal less usable.

Information Architects does give us an awesome article debunking Technorati’s blog stats from early last year. Yay!

When people look at web pages, what are they really looking at? Give Eyetools some money to answer the question for you. They’ll do a little study and send you some cool visualizations.

Check out the heatmap, which reports the behavior of many users. The “individual session images” are equally fascinating.

They do all the work at their office. I guess that’s so you don’t get too scared of the tools. Unfortunately, it reduces their outreach into fields outside of web design — like, say, game design.

UI usability is normally totally ignored. If Microsoft is your publisher, they’ll send your game through the Usability Lab (and if you live in Seattle and are suitably multicultural, you can be a tester, too!). If you’re anybody else, you’re lucky if a single designer on the team gives enough of a shit to say something about it.

Eye tracking is important. Minimal eye movement should give the player access to all the information they need for the task at hand (during combat, for example, maybe your power cooldowns and the opponent’s health shouldn’t be on opposite sides of the screen). Unfortunately, without the crazy Eyetools lab, you’re not going to have any hard data on that front.

Intuitiveness is something that you might have more luck with. Is it intuitive to find your character’s inventory the first time you log in? You’d need some way of tracking clicks (log each open-HUD event, for example). Alternatively, send the peon junior designer into the newbie chat channels during the first few days after launch, and find out what people are asking about. They probably won’t go to the boards to say that they can’t find the inventory. They’ll ask in chat and wait for a beta player to answer.

To validate the default UI layout, you need to know how players have rearranged theirs. This requires you save UI options on the server — probably not a bad thing anyway, as it means they don’t have to arrange their UI just right every time they log into a new machine (just be sure you account for potentially different resolutions). You get a mountain of fun data to play with, too.

Option C is doable by just about every average MMO. Why don’t you?