We Can Fix That with Data

Let’s stay on this handcrafted content thing. Let’s say you have awesome tools and a great pipeline, you have an enormous and productive team which never complains about having to make stuff they don’t like, and you’re just full of great ideas with what to do with it all.

Your bottleneck is playtesting. Not QA — your awesome tools help ensure that stuff is mostly not broken on the first iteration! You need playtesting to make sure that experiencing that content is everything that it can be — your awesome tools cannot help you.

Let’s say you’re, oh, working on a mature game and making end-game content to shut whiny guild leaders up.

Your QA team is a bunch of minimum wage temps who don’t actually play the game. They can’t help you test raid content.

You can round up the whole office and ask them to play through it a few times a week, but that’s time they could have spent coding or modeling or whatever it is they do. And they’re not top-notch players either.

You can recruit some player volunteers and ask them to please, please not tell anybody about your dumb first iterations. Maybe they will, maybe they won’t! At least they play the game. (And woe if anyone outside of the program finds out. It’s so unfair!)

Alternatively, you can throw it on test and move on. Then everybody knows about your dumb first iterations! It’s so embarrassing.

How can you solve this bottleneck?

Two weeks ago, one of the top World of Warcraft raiding guilds announced that they were leaving the game because they were unhappy with the current risk and reward ratio. Despite the fact that the next patch is probably next week, they’ve followed up with a tangent on the trouble with handcrafted content: (copy-pasting quite a bit because the link doesn’t look stable)

Do you want the truth? Blizzard is giving nobody that pays for a subscription to WoW what they deserve. With the amount of money they are making, they should be putting out at least 4 5-mans and 1 10-man per month and 1 25-man every-other month. Not to mention they should probably be adding 1 new battleground each month, and 1 special tournament every other month. […] To be honest, I’m more irritated with this badge loot bullshit […] No, I really really don’t want to do more Kara, more of the same heroics, or anything else that gives badges. That is OLD CONTENT, regardless of the rewards.

Do people that enjoy 5-mans and heroics not enjoy seeing different encounters in different instances? […] . As was said earlier, -everyone- should be getting content. […] It is all being put out in this game slower than it is in AC1, and AC1 has been out for what, 9-10 years, and has less than 1% of the player-base of WoW.

Asheron’s Call 1 has nine years of monthly content updates. They had what, a live team of ten? But they had a famously awesome toolset, and people said they had an obscene retention rate.

Following up on Joe and Damion’s posts, Raph says “a designer who only works inside of data-driven systems will not have a career path and training path to learn systems design unless they are given access to system modifications.”

Well, shit. Somebody tell my boss.

I design “infrastructure systems,” like items and abilities, and I do so all the better because I have years of experience in data management. Five years from now, the live team will be three guys in a cardboard box. If I’ve done my job right, they’ll do more than tweak numbers — they have room for creativity because the systems are well-thought-out and allow for combinations to last for the service’s lifetime. And if they need something the system doesn’t support, they can get an hour’s worth of code and be on their way.

If the player ends up feeling like they’re “wandering through a database,” then the systems were poorly designed.

There’s nothing wrong with design within constraints. That design will be easier to polish, balance, and maintain in the long run, because it’s easier to run a query than to comb through scripts. And we’re failing if we’re not in it for the long run.

CCP has brought on an economist to monitor Eve’s economy. “His research is designed to provide players with information necessary to make strategic decisions, but is also expected to have an impact on future development of the game.”

He’s maintaining a blog.

I just rerolled on a new server in WoW, and I planned to get rich with skinning and herbalism. It’s only sort of working — ye olde bank alt has about 25 stacks of thick leather that I think I’m going to have to vendor. I wonder if leatherworking would get fixed if a goddamn economist pointed out that there’s no market in this shit.

• WoW talent distribution metrics. Somewhat flawed methodology (he only examines a set of characters with Allakhazam profiles), but interesting nonetheless. 12% balance druids, represent!
• An RMT solution to the Chinese government-mandated anti-addiction system.
• Pictograms and culture. Worth thinking about if you’re planning to localize your game. You can’t bake the word “Blacksmith” into your sign textures. Well, you can if you want to redo all your signage for every version of the game. Good luck with that, and I’ll spend a lot less time a) thinking about universally understood icons, and b) asking the artists to do the textures once and only once.
• Information design is graphic design: improving the usability of information software (via Juice Analytics). MMOs are information-heavy games; we should be working harder to present the players with the information they need in a format they can use. WoW has a decent default interface, but still, how many of the most popular WoW UI mods are ones that improve information display?

Thanks for all the kind comments and emails. My wrist’s recovery isn’t quite on schedule — I was supposed to be able to type by now, but the doctor’s telling me to wait for another freaking month. This has significantly hampered my ability to work, but after a few days learning to target with the keyboard, I’m playing WoW at about 90% efficiency. It helps that I’m a caster druid, so my job is to either “tab to target, hit Moonfire hotkey, hit Insect Swarm hotkey, hit Wrath hotkey a bunch of times, tab to next mob” or “hit f key to target groupmate, hit Lifebloom hotkey, wait … hit Lifebloom hotkey, wait … oh shit, hit Healing Touch hotkey, wait …” About all I ever have to type is the initial “am I healing?” when I join the group.

This may say something about group gameplay and dynamics in WoW — not that I’m complaining. I’m happy I can do something other than watch Star Trek: The Next Generation on DVD.

In “stuff I wish I could comment on further that’s come up in the last few weeks:”

• The discussion on guilds at Will’s is timely, because another one of my recent WoW activities has been shopping for a raiding guild. It’s been an interesting experience, and I plan to rant about it later. See also Kevin’s followup commentary.
• Terra Nova talks PvP balance. I’ve done PvP balance for a long time. I eventually discovered a secret to making changes that feel responsive, rather than change-for-the-sake-of-change or an attempt to jerk players around: wait until the last minute to commit changes. Producers hate it, QA complains, but if you fix a perceived problem and sit on it for two months, nine times out of ten that’s enough time for player ingenuity to solve the problem on their own. Then you post the patch notes and they think you’re an out-of-date idiot, and you are! (Well, regardless, they always think you’re an idiot.) (And assuming that the game mechanics allow for sufficient emergent behavior.) I’ll have lots more to say on this in the future, heh.
• Darius Kazemi’s new metrics middleware company gets off the ground. They have an extremely awesome blog so far. I expect they’ll be an extremely strong competitor to Emergent.
• Holy shit, Rush Limbaugh intelligently defends games. Well, he defends games by way of comparing them to guns, which doesn’t really work on the target anti-game audience, but hey.

Former EQ GM complains that we keep rehashing the same old shit and don’t know how to design gameplay; clearly doesn’t understand why and how the same old shit works.

I was going to say “some single-player designer who played WoW for a while,” but when I checked his company’s website, I realized the “former EQ GM” angle was even better.

Christ, crafting minigames again? You can’t make business deals when you’re playing Tetris. Twitch gameplay? Alas, poor server … and alas, poor non-traditional gamers who make up a significant portion of modern MMO subscribers. Mobs of mobs? Man, you thought that server was on fire before! User-created content? Ha. “Would it really be so bad if player A and player B had different game experiences as a result of the world being altered?” Um, YES. If something is worth pointing out to your groupmates, they better be able to see what you’re talking about, or they’ll a) waste CS time and/or b) think your game is a buggy piece of shit.

Work on a live team for a couple of years before you start making grand proclamations about how terrible we are at game design. I thought that CS would count — I mean, isn’t that Ground Zero for learning about team resource allocation, what with the terrible tools and having to tell players that no, we weren’t able to get around to fixing that bug for the third year in a row? I guess not.