How to do a playtest on a small budget [TATT #34]

the acagamic tip tuesday Oct 11, 2022
A playtester chatting with a test player over a coffee.

In today's issue 34 of The Acagamic Tip Tuesday, I will show you how to conduct playtests when you only have a small budget available. It will take about 5 minutes to read. I will walk you through how to run a playtest on a small budget.

Running extensive testing sessions is great, but it's even more important to know how to do UX research when you feel like you don't have the means to do it. Once you know how to streamline a playtest, you will always be able to expand it when resources become available.

Playtesting on a small budget: This provides goal-oriented iterative results that can incrementally improve your game.

The benefits of doing UX research on a budget are worth it:

  • Sharply-focused insights
  • Clarity about execution
  • Easy to learn
  • Easy to set up

Unfortunately, many game companies wait to explore user research until more funds become available. This is not necessary. The best time to playtest is now.

Companies think playtesting takes too much time and too many resources

That is not correct. All you need to decide and know are (1) the single goal/focus of your playtest and (2) when, (3) where, and (4) how you will run it. That's it.

Focusing on a single goal for your playtest will produce better feedback quality and allow you to set up, run, and analyze the test with a minimum budget.

Here's how to run a budget playtest, step by step:

Make a plan for your playtest

Each sub-objective in your playtest needs to relate to the main goal you set at the beginning. For example, you want to know if the new laser sword is fun to use (e.g., ask: "What was new to you in this level? [Answer: laser sword] Could you explain to me how the laser sword worked?"). That's your main objective. Sub-objectives could be if the players enjoy playing the game and if the combat is difficult enough.

You can usually classify your playtest goals related to three broad—sometimes overlapping—areas:

  • Game usability and accessibility (Can players understand and act upon the game?)
  • Satisfactory user experience (Are they having fun?)
  • In-game player behaviour (What are they doing?)

Knowing what bins your goals fall into helps you focus your testing strategy.

Limit resources to keep the test manageable

Fewer players mean less time spent running the test and fewer data to analyze. Keep your test between 5-8 sessions to keep it manageable. Quantitative measures will require more participants (20+ usually), while qualitative measures and usability tests will allow you to run as few as 5 people.

In addition, decide 1-2 methodologies (like questionnaires, interviews, or video observation) and make sure they fit your playtest goal.

Assert audience and recruit players

Remember you're on a budget. Focus on recruiting players familiar with the game genre rather than a diverse set (time to do that when you have the resources, it's better but takes longer, too).

Tap into a game's community if it exists, otherwise rely on your own network or that of your company to recruit the right players.

Universities are great places to find recruits.

Set up a testing space

For the players to run the test, make sure you create a small isolated part of your office for the test. If you don't have an office and want to go full guerilla, find a café or a public University space to run the test. Providing good headphones is essential either way.

Keep the environment as consistent as possible between tests. Make sure to keep your distance while they play, but be available for questions and to make them feel relaxed (because help is available).

Run the playtest session

The sequence of a generic playtest is as follows:

  1. Welcome them warmly and provide a breakdown of the test (and provide snacks and drinks).
    • You are testing the game, not them.
    • They can leave at any point if uncomfortable
    • Make yourself available for questions
  2. Require participants to sign consent forms (and any other legally required forms)
  3. Use a recording tool like Loom on your game computer to capture in-game and face video.
  4. Record your observations while people playtest
  5. Administer any surveys or interview questions right after the playtest session
  6. Thank and pay participants

Compile data and write the report

Make sure to leave time after each playtest session to ensure data has been recorded and your data is labelled for analysis.

Compare your goals to the data you have collected. Create a list of issues where participant feedback does not align with playtest goals.

The easiest way to present the report to the team is with free tools (like Google slides) and to keep it focused on takeaways. Remind yourself to end your presentation with an action plan for the development team.

Games Research Find of the Week

This paper proposes new approaches to automated playtesting. The idea is that playtesters could describe archetypal player behaviour with procedural personas to playtest distinct playstyles. The flaw of these personas (or automatic players) generated via reinforcement learning algorithms is that they don't adapt playstyles as real players would during a game.

The paper proposes a novel artificial intelligence method dubbed Alternative Path Finder (APF) to create a developing persona that essentially is a more realistic player model of a multi-goal-oriented persona. The article compares the developing and procedural personas based on their underlying decision models.

Game designers can embed various personalities in developing personas to generate unique playtests for their games.

Read the full study: Ariyurek, S., Surer, E., & Betin-Can, A. (2022). Playtesting: What is Beyond Personas. IEEE Transactions on Games. 11 April 2022. DOI: https://doi.org/10.1109/TG.2022.3165882

Join UX rockstars learning about games, research, and writing

Each Tuesday, I will send you tips for UX in game research and design. And once a month, I will send you emails about academic writing.

No hassle, no selling your information, no spam. You can unsubscribe anytime.