Evidence-Based Design

Yesterday, I snarked on food diary websites who could make money with their data.

In terms of determining the “average American diet,” like NPD food diary findings strive to do, diet websites like The Daily Plate won’t be useful. People who use those sites are Internet-savvy dieters. They almost certainly aren’t representative.

However, it’d be fascinating to compare the average eating habits of said Internet-savvy dieters over time — did Michael Pollan and friends really encourage a significant number of people to start eating “food, not too much, mostly plants,” the way that NYT article suggested? They could undermine the NPD there. Muahaha.

Companies like Weight Watchers could benefit tremendously from analyzing their database. The Weight Watchers program depends on social reinforcement, where group leaders are pushing company policy on “the right foods to eat” and (to a lesser extent) consumption of Weight Watchers products. If a significant percentage of Weight Watchers members use the website to track their diets — and track reliably — the company can see how effective those money-making policies are, and can see whether or not it makes a difference, since users regularly track their weight. (Of course, they need to take a shame factor into account — you know, since stuff you eat while you’re standing doesn’t count, and that weight can’t be right, because I drank a glass of water before I went to bed, etc. etc.)

Is Weight Watchers taking advantage of their database? They charge an extra fee for access, so I doubt it. Additionally, they say that group members who use those tools lose more weight; the small print says “weight loss data based on 12 week study comparing people who were instructed to attend Weight Watchers meetings and use eTools to people who were instructed to attend Weight Watchers meetings alone.” If local Weight Watchers groups send data home — I don’t know if they do — they could have gotten that data for free.

To bring it back to games, Magic the Gathering presents a similar case. Wizards of the Coast pushes a huge amount of content out to a public that plays in game stores. Tournament play is regulated to some extent, but (to the best of my knowledge), deck composition is not tracked. Lots of people talk about the leetest decks on the boards, but there’s no real way to tell how popular they are aside from community manager-style gut feel.

Magic the Gathering Online should allow WotC incredible insight into the way that people play their game, access to data that they’ve never had before. However, the old version of MTGO saved decks on the client. You could save “net decks” to the server, but if I remember correctly, it defaulted to local storage. Data on the client does you no good.

I’d write a witty conclusion, but comparing Weight Watchers to Magic the Gathering is probably enough levity for tonight.

This is creepy.

This is funny.

While it’s old news by now, this article on the lack of social play in Warhammer is very good. Conan had similar issues, as I wrote about earlier this year. While encouraging social play isn’t as crucial in a PvP game with prebuilt sides, it’s still sad to see any MMO deny its fundamental nature.

Speaking of PvP: in the midst of a discussion on PvP itemization in World of Warcraft, some talk on tracking player performance in battlegrounds. I don’t know what Blizzard’s planning, but I hope they haven’t forgotten that a bunch of people have been trying to solve this problem in other contexts.

Getting back to Warhammer, gamerDNA’s been printing some interesting articles based on Xfire data and surveys. If you’re not familiar with them, they’re a data mining company pretending to be a social networking company, so we can expect lots of intesting stuff from them in the future. Subscribe to the RSS feed if you haven’t already.

In an oddly similar direction, this NYT article suggests that people are eating better food these days because the NPD says so! Somebody call the Daily Plate and Sparkpeople and Fitday and hell, Weight Watchers Online, and let them know about their exciting new business model.

I have joined the DC Universe Online team at SOE Austin. Here’s a recent preview of the game.

As always, I continue to speak for myself and not for Sony Online Entertainment. I signed a form saying so!

In other news, Blizzard has announced that come Wrath of the Lich King, many UI settings and macros will be saved server side. I’ve been meaning to do a blog post on stuff that should be saved server side … you know, stuff that makes players think you’re kind and considerate and care about the plight of players who play on multiple machines, but really just gives you lots of interesting things to data mine. Hard drive space is cheap, it earns goodwill, and you can build charts.

Blizzard’s been taking advantage of the Wrath of the Lich King beta to make major changes to their core RPG. This is a nice situation for them — they get to iterate, they get tons of player feedback, people are actively playing (unlike most games’ test servers), and they don’t destabilize their live service.

One of the latest changes is a change to the way that mana costs are calculated. They’re switching from the normal fixed costs for particular spells (Generic Fire Nuke (Rank 1) costs 17 mana and by god, it always costs 17 mana) to scaling percentage costs (Generic Fire Nuke (any rank) costs 10% of base, unmodified mana no matter what).

A few players are excited, but the majority are unhappy about this because it does away with “downranking” — casting a lower-level version of the same spell because it costs less mana and is more efficient for your purpose. Healers do this quite a bit. If I know that my target only needs 1k HP healed, why should I spend a ton of mana to cast my max-rank heal that heals for 3k, when I could cast a lower-ranked version that heals for 1k and costs much less mana? Smart healers usually play with a couple of different ranks of several of their spells on their cast bars. Hey, it’s strategy!

Usability-wise, in World of Warcraft, where your spellbook displays all the old ranks, it’s not hard to sit there and do the mana-per-HP healed calculation yourself, or you can use a mod like DrDamage to do it for you. (People downranked in Shadowbane too, but they had to rely on oral tradition to get the slash command to access those “obsolete” spell ranks.)

Loss of the strategy of downranking aside, it’s a good change for the designers, in terms of using their time efficiently.

When I’m costing a spell the traditional way, I’ll use a spreadsheet that says something like “okay, this is a level 10 spell, and at level 10, you should have about this much mana and this much mana regen, and I want you to be able to cast this spell this many times before you run out of mana.” I’ll have to do that for every rank of the spell, taking into account how much mana and regen I think the character should have at that level. And later on, I may have to revisit those numbers once I mine the character data to confirm those expected mana values — if that data is available to me.

Generally speaking, I want that number-of-times-cast value to remain consistent as the character gains spell ranks, because it fits the player’s expectations — a lowbie spellcaster knows that she can kill a certain number of mobs with Generic Fire Nuke before she has to rest, and every time she gets a new rank of Generic Fire Nuke, that should remain the same as before. In other words, her downtime shouldn’t fluctuate dramatically as she advances, because downtime is a defining class characteristic.

The percentage cost model makes my life a lot easier. I don’t have to figure how much mana I think you should have at that level, and I don’t have to check the character data to confirm. I just say that I want you to be able to cast Generic Fire Nuke about 10 times before you run out of mana, so I say it costs 10% of your mana. DONE. (Okay, it’s not quite that simple because I need to account for mana regen, but I’m still DONE in a lot less time.)

Now, this breaks down a bit if new spell ranks aren’t granted quickly enough. Lowbie Spellcaster will find that she has to rest more often, killing the same number of mobs, because Generic Fire Nuke does less damage relative to the content she’s doing, so she has to cast it more times, but she can’t cast it more times because it still costs the same percentage of her mana. Theoretically, with traditional fixed costs, if her mana pool was scaling at the same rate as her other stats, she’d be able to cast it more often before resting to make up for doing less relative damage. (However, this means that her mana pool needed to scale properly … but math is hard and it doesn’t always work out that way.) And she still feels slightly less powerful with every level that passes without a new rank, because she has to cast more often.

The thing is, stairstepping is required to make advancement feel meaningful. I wouldn’t do percentage damage to make up for that problem. Damage is usually figured as a percentage of the intended target level’s HP, same as the traditional way of doing mana costs. The Generic Fire Nuke I intend for Lowbie Spellcaster to use at level 10, versus level 10 opponents, is probably supposed to do N% of the average level 10 mob’s health. If Lowbie Spellcaster use that rank of Generic Fire Nuke against level 12 opponents, it’ll do less relative damage. But this is the way it should be. It motivates Lowbie Spellcaster to get off her ass and level to earn the next rank, same as Lowbie Fighter should be motivated to level to use a new sword. When they get their respective rewards, they feel like they’ve accomplished something.

Going back to WoW’s situation: the loss of downranking, to be fair, might increase workload because the designers may want to build and maintain abilities that perform the same purpose (like small, low-cost heals for that poor shaman in the thread I linked). But in the long run, percentage mana costing makes their lives easier. If I were balancing an RPG right now, I’d be inclined to make the same decision.

Netflix is eliminating their profile feature, where an individual account (say, two movies at a time) can be split up into separate movie queues (say, one for my movies and one for my spouse’s, in case our tastes don’t match).

At least, we assume that the reason the feature exists is a) because it’s a nice feature for households with multiple movie watchers who disagree on DVD priorities, and b) because it allows them to data mine individual preferences for said movie watchers. Without this feature, they can do neither, and they piss people off while they’re at it.

I wonder if maybe the data just wasn’t that useful (but I like seeing recommendations driven by my movies!). I wonder if the data storage was getting to be a problem (doesn’t seem like it’d be that much, aside from maybe the individual rating data). Maybe they never used it in the first place (ahh, the world of corporate “business intelligence”).

Like somebody points out in the Metafilter thread, it’s a bad time to piss off subscribers for any entertainment service. Food for thought.

In addition to your friends’ scores, Rock Band song leaderboards provide some interesting business data.

Every time you play a song in Rock Band while connected to Xbox Live, your score is applied to the song’s leaderboard. What’s interesting about that is that the number of entries on the leaderboard can tell us how many users have played the song. And what’s interesting about that is that it tells us how many users have purchased the song, if it was downloadable content. (Not counting people that purchased the song and never played it, and not counting if the users-played count is over 100k because the leaderboards don’t go that far, but hey.)

Rock Band DLC Statosphere adds it all up.

skedastic of the Qt3 boards massaged the numbers a bit to remove time factors and to group by artist. There’s some debate in the thread over his statistical methods. I would like to debate because Rush and Nine Inch Nails are relatively low, and goddammit, I want more Rush and Nine Inch Nails.

Am I not getting more Rush and Nine Inch Nails because the team can see that they’re poor investments? I don’t know, but I haven’t seen any patterns in the DLC releases that would imply they’re trying to provide more songs by proven bands — yet. I wonder what their lead time and content production pipelines look like. Is it harder to get assets out of studio musicians than it is artists?

In the future, you’ll be able to store saves for your Steam games on Valve’s servers.

I don’t know how often people play the sort of games they’d want to save on multiple machines — in other words, narrative games as opposed to multiplayer. I can understand burning some time at a friend’s house with a fast game of Some Multiplayer Shooter, but I don’t think I want to sit down and pick up a story where I left off. Narrative games would seem to lend themselves to longer and less social sessions.

But since Valve knows what you’re doing all the time, they must know how often people play the same narrative game on multiple machines, so maybe they see some demand.

Or … maybe there’s another reason. What can they data mine out of save games that they don’t already know? They already know how far you’ve gotten — if that’s what “Highest Map Played” means. There must be something interesting in there …