Game Design Concepts: Evaluating the Design
We're continuing our series on game design with insights from Jeff Tidball, a veteran game designer and producer, as well as Atlas Games' COO. Have a question for Jeff? Give us a mention on Twitter @AtlasGames.
[What are] Best practices for testing and evaluating a game. — Richard (Facebook). How do you run play tests? What sorts of prompts net you the most productive feedback. —Aser (@aser_tolentino)
These two questions demonstrate the right thinking: That evaluating and improving a game design requires testing it, gathering feedback from the testers, and iterating the design to improve it.
Begin by considering two different scopes of testing, and two types of testers.
By “scope,” I mean, “How much of the game is being tested?”
The longer scope option is easier to define; the whole game is tested.
But some designers — especially those newer to the discipline — overlook that you can test smaller subdivision of a design. This idea becomes crucial when designing, say, an RPG, where it’s completely impractical to test character creation, combat (and its diverse sub-systems like initiative, magic, tactical maneuvering, healing, etc.), advancement, and more in a single play session.
Testing only parts of a design can also work for less ambitious board and card games, too. And frankly, it’s a good idea to focus on one thing at a time. Testing six endgame outcomes, for example, or running set up and unit placement ten times in a row, will give a designer a lot more insight into that area of their design than running three full games, when their attention is diffused across the entire experience.
By “types of testers,” I mean, “What’s the relationship of the people doing the testing to the designer?” There are essentially two options here, although you could argue that there are three.
Blindtesting is when people who might or might not know the designer personally play the game without any input from the designer or design team. The testers learn the game from documentation, answer their own questions by referring to the components, and so on. (Now, one or more members of the design team might be present to observe and record outcomes. Ideally, they wouldn’t be, because their presence can change the feedback, but sometimes it’s not practical to audio- or video-record the testers’ feedback, or observe a test without being observed. If the designer is present, the important thing is that they remain uninvolved, not answering questions, simply observing whether — or, more likely, when — things go off the rails.)
Non-blind playtesting (most frequently just called “playtesting”) is when the designer is present, perhaps playing themselves, explaining play and answering questions, and perhaps altering rules on the fly as it becomes clear that some new option will be more fruitful. Most playtesting is done in this mode, especially early testing of a newer design, or testing of a design that’s currently undergoing a lot of change.
A sometimes-fruitful type of tester — the arguable third option mentioned above — is when a designer does non-blind playtesting with players who are not personally known to them, such as at a convention or meet-up. When you playtest with your friends, you often get the (unwarranted) benefit of the doubt when some element of your design is confusing, not fun, or complete nonsense. People you’re never going to see again are both more likely to expose you to a wider diversity of viewpoints, as well as less likely to exhibit undue care about your feelings.
What should you ask testers? I’ve used a ton of different questions at various times. I like most of the questions in this Gamasutra blog post by Wesley Rockholz. They get at some key concerns I always have for a new design without asking questions that are too on the nose, like whether the game dragged, what the emotional experience of gameplay was like, whether the game is easy or difficult to understand and play, and so on.
For both scopes and all types of testing, and no matter what you’re trying to learn, here’s the cardinal rule of playtesting: You must not offer rationales and explanations in response to comments and criticisms. Write down everything that’s said, and think about it later, when you’re more detached. Explain a rule if needed, but not in a way that negates or explains away an objection or observation. If you must offer a design perspective that you think may have been overlooked and might undermine a criticism, do so well after playtesting is done, and after you’ve thoroughly internalized the idea that the comment was, in the first place, a legitimate response to the current state of the design.
It’s worth noting that it can be very hard to receive negative feedback about something you’ve worked hard on. Receiving feedback calmly is an emotional skill. Cultivate it. You can’t make a game better without learning what can be improved, and learning what can be improved feels a lot like an attack, and an attack on you, especially at first. If your experience is anything like mine, you’ll eventually arrive at an emotional place where, to the contrary, you’re deeply suspicious when people claim to enjoy your game, and you probe them incessantly for broader and broader critique, because you know that something can always get better, especially in very early prototypes. Designs are never finished, goes the quip, only abandoned.
If you’re having trouble with the idea of negative feedback, consider this: It’s a lot better to have a table of your friends tell you what they think is wrong with your design than read a negative review online, where hundreds or thousands of other people will read it too. Not only does the playtest audience comprise your friends, who’re going to like you in spite of your design’s shortcomings, but with playtesting you’re still in a position where you can do something about an issue. If you’ve already published the game? Not so much.
If you're looking for more on game design, The White Box features twenty-seven essays on topics ranging from box design to accessibility in gaming, plus game components to help bring your game into the physical world.