How To Ask Good Playtest Questions [TATT #50]

If you've ever observed a family member playing a game and wondered why they did something in that game, you are in the shoes of a video game playtester. When running live playtest interviews, we can figure out what's going on inside players' heads and why they behave the way they do by asking questions. Think of it like a detective trying to solve a mystery: we need players to explain their actions so we can make the game better. Without their feedback, we'd be lost in the dark! By asking proper questions and we can get to the bottom of the case.

Playtesting interviews are a crucial part of game development. It provides valuable insights into how players interact with the game and what can be improved.

However, interviews are only as good as their questions. Good playtest questions can yield precise, specific feedback to improve the user experience, while bad ones can produce vague, unhelpful insights. Consequently, excellent playtest questions are necessary to ensure actionable input that can influence development. Playtest questions help games user researchers make informed judgments that improve user experiences and game success.

Research questions may influence players. Picture this: you're a magician and your questions are spells. Spells always influence the player's thoughts and actions. Our magic spells must thus be handled with care. Imagine your questions to be clear like a freshwater lake. The wrong words will contaminate the water and ruin everything. We must choose our words carefully and not pollute the water and ruin everything.

What Makes Good Playtest Interview Questions

A good interview question is clear, specific, and neutral. It should ask about a specific part of the game, like a game mechanic or a level, and not try to lead the player to a particular answer. Good playtest questions should lead to detailed, specific feedback that can be used to improve the player experience. Neutral questions are better than leading ones. Ask, "What's happening?" or "How did you decide?" instead of "How did you know this was correct?" That way, you won't give away any clues about which actions in the game are correct.

Let's walk you through some examples of bad playtest interview questions:

1. Did you think the boss was too hard? (This is a leading question.)
2. Wasn't that level really fun? (This is a leading question and doesn't gather specific feedback.)
3. What didn't you like about the game? (This question is too open-ended and invites players to rant without providing specific feedback.)

Here are some good interview question examples:

1. What parts did you like best about the game? (This is a good open-ended question that allows the player to provide detailed feedback.)
2. How did you feel when you encountered the boss for the first time? (This is a good question about a specific event in the game and doesn't lead the player to a specific answer.)
3. What aspect would you change about the game if you could? (This is a good question asking for specific feedback and provides valuable insights for game developers.)

It's vital to ask questions at the right time and in the right way, so that players feel comfortable and can give honest and accurate feedback.

When Is A Good Time To Check Players' Mental State?

Playing games takes focus, and asking questions can distract players and make them self-conscious. Wait for a quiet moment in the game, like the end of a level, to ask questions about gameplay. But don't wait too long; people forget their actions quickly. Sometimes researchers ask players to review gameplay recordings later, but that can be tricky. Players might guess their motivation and get it wrong. It is preferable to ask players very soon after they do something interesting, so they can explain what they were thinking at the time.

Types of Playtest Questions

Open-Ended Questions. The goal of open-ended questions is to be broad and let players give detailed answers. These questions can help determine what players think and feel because they give players a chance to say what they think in their own words. For instance, "What did you think of this aspect of the game?" is an open-ended question that can lead to detailed feedback.

Close-ended questions. Conversely, close-ended questions are meant to give specific information about a particular part of the game. These questions can usually be answered with "yes" or "no," which can help you get specific information quickly. For instance, "Did you find the game hard?" is a closed-ended question that can tell you that the game was hard. These are used sparingly by games user researchers, because the insight value of these is limited.

Leading questions suggest answers and should be avoided altogether. These questions can lead to biased answers that misrepresent the player's actions. "Was the boss fight particularly hard for you?" is a leading question.

Neutral questions don't instruct the player. Fair questions help you receive more accurate replies. "What was your boss fight like?" doesn't indicate that it was hard and is a much better way to phrase the question.

Finally, interviews can be combined with short surveys after a playtest and multiple-choice questions can swiftly gather precise data. These questions allow players to choose an answer and provide quantifiable data. "How many times did you die during the boss fight?" could a multiple-choice question that might lead to quantitative insights about game difficulty.

Think-Aloud Protocol

Sometimes, we ask players to talk out loud while they play the game. It can be distracting, but it works well for simple games. If you use this method, tell the players at the start. Say, "We want to know what you're thinking as you play. Please talk out loud, and we'll remind you if you forget." This technique works really well with Twitch streamers that are used to talking in front of an online audience. Requesting more information with questions like "Why did this occur?" or "Why did you go and do that?" will yield further explanations.

Move From General To Specific Questions

Start with general questions before moving to specific ones. That way, it's easier for players to talk. If you ask about specific features too soon, it might mess up their answers. Ask open-ended questions such as "What was your overall impression?" or "Is there anything else you'd like to share?" These questions can show you what you're missing and help you ask better questions.

Here is a list of questions that will likely come in handy for any playtest interview you conduct:

• What did you enjoy most about the game today?
• Tell me about how the game went for you today.
• Which game mechanics did you find the most challenging to master?
• Can you think of any ways that the game could be improved?
• How did the game's tutorial or instructions impact your understanding of the game?
• Did you feel a sense of progression while playing the game today? Why or why not?
• What was the most frustrating aspect of the game today?
• So, what do you think about the game?
• How did the game's music and sound effects enhance or detract from the experience?
• Would you keep playing if you were at home? Why and why not?
• What parts of the game did you find the most immersive or engaging?
• What did you think when you first played it?
• Do you play games like this one very often?
• Which character or aspect of the game did you relate to the most and why?
• Were there any moments during the game that you found particularly memorable? If so, what made them stand out?
• Did you encounter any aspects of the game that you found confusing or unclear? If so, can you describe them?
• How did you feel about the pacing of the game today? Did it feel too slow, too fast, or just right?
• What was your overall impression of the game today? Would you recommend it to a friend?
• What game does this remind you of the most? Why?
• Were there any aspects of the game today that you found frustrating or tedious? If so, can you describe why?
• Did the game today meet your expectations, exceed them, or fall short? Why?
• How would you tell another player about this game?
• Can you describe a specific moment or instance where you felt particularly engaged with the game?
• What aspect of the game today did you find the most difficult to master or understand? Can you describe why it was challenging?
• What part of the game's story or narrative stood out to you the most today?
• Did anything about what you saw today surprise you?
• Where would you look in this game to learn how to do something?
• What was the best thing about today's gameplay session?
• What bad thing stood out the most today?
• What about this game will you remember?
• What did the game today make hard?
• What about the game today was hard to understand?

Avoid These Common Pitfalls

There are 5 strategies that are likely to fail in generating insights from your players in playtests. Avoid these at all costs:

1. Vague questions can result in unwanted feedback. "What did you think of the game?" may not elicit particular feedback.
2. Leading questions might prejudice players' responses and give false or unwanted feedback. "Did you find that level particularly irritating?" implies that all players found it frustrating. As mentioned above, avoid these.
3. Overly specific questions may limit players' responses and not capture their whole experiences. "Did you like the character's hat?" may not be related to the user experience or provide useful insights.
4. Asking for opinions, not behaviours: Asking players' ideas or sentiments about the game is less beneficial than asking about their behaviours or experiences. "Do you like the game?" is less helpful than "Did you find the game challenging?"
5. Not listening to players is a common mistake. Players' feedback should inform development decisions. Ignoring or dismissing player feedback can lead to a less engaging and less successful game.

Try out these tips and techniques for asking good playtest questions and see how they can help you get more detailed and valuable feedback from your players. This helps you make better development decisions and give your players a better experience. See how these techniques can help you get better feedback and make games more fun. Don't forget that your game's success depends on the players.

Engaging playtest questions can boost game success. Specific questions and neutral phrasing help developers gain more accurate and unbiased player input. These question-building tips can help you improve feedback, focus adjustments, and create a better player experience.