In today's issue 32 of The Acagamic Tip Tuesday, I will walk you through how to conduct an interview as part of a practical qualitative playtesting session.
Because much playtesting is conducted as a one-on-one session with a UX researcher observing a playtester (or a group of playtesters) while taking notes (sometimes with a separate notetaker), interviews are the perfect supplement to a playtesting session.
Interviews complement observation: Observations can tell you how people played and interviews will answer why they exhibited the observed behaviours.
Interviewing allows you to understand why people behaved the way you observed them. Often these interviews are conducted in a semi-structured format, where much of the questions are prepared in an interview guide, but the UX researchers are free to sidetrack or drill down on questions to help get more accurate answers from testers. Being able to deviate is critical. Interviews can provide rich qualitative data that can form the basis of thematic analysis later.
Unfortunately, many people are afraid to conduct interviews or have no idea what format to follow for their interviews. Junior UX researchers also often struggle to translate research questions (meaning: the questions we want answers to) into interview questions (meaning: the questions we will ask).
Translate research goals into interview questions
Research goals should be set before venturing into an interview session. Often these can be specified directly by the game's designers. An example would be: "Do players understand our reload mechanic?" The interviewer could then ask questions like:
- Can you explain to me how the reload mechanic works?
- How does the reload mechanic work here? Walk me through it in your own words.
- What is unclear about reloading your weapon? Was there anything you struggled with? Why?
- How would you capture the way the reload mechanic works in one sentence?
It is not always easy to translate research questions into interview questions, but it is crucial to understand the difference. Once you've understood the translation process, you can quickly create an interview field guide.
Here's what's in it, step by step:
The playtesting interview field guide
A field guide is an essential document for UX researchers to establish a protocol of what will happen during an interview. This provides structure to the interview session.
It usually consists of:
- Introduction, logistics, and player background
- Main body with subsections for each research goal
- Idealizations and projections
- Wrap up and shot list
Introduction, logistics, and player background
Identify the time of the playtest session when you want to ask questions:
- pre-test interview,
- version comparing interview (A/B tests),
- skill-check interviews (interrupt the game at a pre-determined time), and
- post-test final interview to validate observations.
Give out model release, consent, and non-disclosure (and sometimes incentive declaration) forms and let participant sign.
Start recording (cam or mic) and confirm total interview time with participant.
Explain who you are and why you are doing the interview. Ensure that participants knows that the game is tested and not their performance (say: "there are no wrong answers, we only aim to improve the game").
Ice breaker and background questions:
- What main game are you currently playing at home?
- How do you feel about that game?
- How long have you played it?
- Have you ever played a [game genre of study]?
- What is it that you look for in [game genre of study]?
Main body with subsections for each research goal
Make sure to map out the areas you want to explore in your interview with questions here.
Be open to discuss and revisit concepts, understand the context of play/use and what contextual factors influence gameplay.
If you are doing skill-check questions, drill down on specific concepts like: "What weapons do you have available? How does each one work?"
If you are doing a post-test final interview, start broad ("What did you think about the game?"), then pros and cons ("What did you like best/least about the game?"), then specifics ("How did game mode X work?; You mentioned X, does this always work the same or different? Why do you think that is?").
Idealizations and projections
Specifically, towards the end of longer (for example, 90 minutes) interviews, you have earned the right to ask some more imaginative questions. Something like: "If you could change one particular thing about the game to create a better player experience, what would it be? If you could produce your ideal experience, what would that be like?" or "If we came back in five years and had this conversation again, what would be different?"
This stimulates the participant's imagination and provides a good transition into the wrap-up.
Wrap-up and shot list
Have good closing questions prepped makes it easy for you to end the interview professionally and tie things back together. Questions like: "Is there anything else that you did not get the chance to tell me?" or "Did we miss anything?" or "Is there anything you want to ask us?"
Then, you thank the playtesters and pay them the incentive.
Finally, provide a list of photos you would like to include for your interview report (usually, participant and game, sometimes with the interviewer).
Things interviewers should be aware of
- Repeating questions can alter player answers and influence how they play (some questions only produce accurate answers when first asked)
- Mirroring and rephrasing answers can be powerful in creating a persuasive conversation atmosphere. It also validates that the interview understands the player's point.
- When players ask questions about the game, instead of giving them direct answers, it is good to deflect into their expectations about how things should work in the game.
- Avoiding leading questions is vital to make players feel at ease.
- Opinion questions are lead-ins to other questions but do not identify real issues (filter out during analysis)
- When asking players about events that have occurred a while back, you risk them unintentionally lying because they do not recall correctly. Validate unreliable memories with observation data.
- Drill down to the root problem by asking "Why is that?" several times in different ways until you get actionable information.
A field guide is just a guide
You should always have a field guide to know what you will ask and remind yourself what you should be aware of. However, this is not a script but provides the guidelines for conducting your interview. Share it with your UX team and evolve it together.
The field guide is a thorough outline of what you anticipate will occur during the interview, usually moving from specifics to their significance. When you begin your interviews, having that well-thought-out plan gives you the necessary flexibility.
UX Research App of the Week: Riverside.fm
I know, Riverside is a podcasting app/website. However, it is super useful to create high quality audio and video recordings of remote interview sessions, too. So, in case, you are looking for the best quality audio and video capture when doing remote interviews, Riverside is a great tool to consider.
Games Research Find of the Week
This study reports on interviews with six idle game designers to identify core characteristics that these games share. The paper synthesizes this knowledge into a deeper understanding of what it means to design idle games. This game genre is understood as games where playing is mostly an activity done by the game itself with players merely slowing down or speeding up game processes. This allows players to casually drop in and out of play.
The paper recommends three major design guidelines: (1) Taking care by design urges designers to span narrative beats across larger time intervals for less addictive potential, to encourage players to disengage from the game temporarily, limit transactions within a time frame, give players the option to play for free, and reduce the energy load on the play device. (2) Systematic narrative arcs suggests that designers create a storyline first before setting computational game parameters, understand implemented functions as a code-based storytelling device, let narrative beats guide computational functionality, and ensure a coherent variable representation to players, that adheres to the surrounding systematic game world. (3) Safe places for exploration recommends for designers to combine system variables for game states based on player decisions, allow players to disengage at will without consequences in game progression, enable players to try options via allocating different resources for various purposes over time, and offer players to learn beyond the core game.
The different guidelines come in handy at different game design stages.
Read the full study: Katta Spiel, Sultan A. Alharthi, Andrew Jian-lan Cen, Jessica Hammer, Lennart E. Nacke, Z O. Toups, and Theresa Jean Tanenbaum. 2019. "It Started as a Joke": On the Design of Idle Games. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '19). Association for Computing Machinery, New York, NY, USA, 495–508. https://doi.org/10.1145/3311350.3347180
Join UX rockstars learning about games, research, and writing
Each Tuesday, I will send you tips for UX in game research and design. And once a month, I will send you emails about academic writing.
No hassle, no selling your information, no spam. You can unsubscribe anytime.