How to conduct a heuristic evaluation of a game [TATT #33]

the acagamic tip tuesday Oct 04, 2022
The image shows the tutorial level in Fallout 4, where the player practices using V.A.T.S. with the easiest enemy in the game (radroaches). The on-screen text explains that a game approachability principle is that the player knows what actions they are supposed to perform and the consequences for completing or failing these actions.

In today's issue 33 of The Acagamic Tip Tuesday, I will walk you through how to conduct a heuristic evaluation of games.

Heuristic evaluation is one of the most cost-effective UX research methods for games. Many people get their start as UX researchers by conducting heuristic evaluations. It's a skill that is easy to learn and—when mastered—quite powerful for fixing problems early in game development.

Heuristic evaluation is a cost-effective UX method: Playability guidelines let you find major gameplay issues fast.

Unfortunately, most junior UX researchers don't know how or where to start.

Heuristic evaluation enables better game quality

Better game quality leads to acquiring, retaining, and monetizing players.

Heuristic evaluation has been popular within usability evaluation for decades, but good playability is less clearly understood.

A lack of playability in a game can be caused by:

  • Unintuitive interfaces
  • Interfering game base
  • Player does not understand the game
  • Game is too difficult

Heuristic evaluation allows a quick way to assess some of the worst issues in games. And there are already a few heuristics lists worth checking out: Heather Desurvire's Game approachability principles (GAP), Playability principles (PLAY), and Virtual Reality Principles (VR PLAY). Hannu Korhonen wrote his Ph.D. thesis on playability heuristics and has produced the most valid set this far. Newsletter subscribers have received a link to my Notion template for this.

Here's how a heuristic evaluation works, step by step:

1. Prepare inspectors, heuristics, and game

First, you want to select 3-5 heuristic inspectors that have expertise and training with this method. They must be familiar with similar games. Then, select the set of heuristics you want to use.

2. Evaluate individual parts of the game

First, evaluate all the menu screens with different configuration settings. Focus on shell UI only. This would focus solely on game usability heuristics.

Next, play the game. Get familiar with core mechanics and objectives. Evaluate gameplay heuristics. Add modules for multiplayer, mobile, and others if needed.

Looking at the combination of usability and gameplay. How well does the interface support in-game interaction? Document positive findings, too.

3. Debrief among inspectors

Discuss the found playability problems among inspectors. Prepare suggestions for fixing each problem. Prioritize problem with severity ratings. Attach violated heuristics to problems to remove doubles.

4. Report to stakeholders

Present the prioritized findings and solution suggestions to your stakeholders.

Open up a discussion with the development team to fix issues with different solutions than yours.

5. Reflect, share, and debrief again

Finally, if you find problems that your current heuristics set did not cover, expand the collection for your use case. Beware, though, that you must understand the nature of your problems first before creating more (possibly unnecessary) heuristics.

Document all your findings to make it easier for future projects and teams.

That's it—a 5-step process of running through a heuristic evaluation for games, evaluating game usability and playability.

Games Research Find of the Week

This paper presents a set of heuristics called the Networked Game Heuristics (NGH). The researchers created the new heuristics by analyzing problem reports from 382 reviews of networked PC games. Problem reports converged into ten problem categories. The paper tested the heuristics with X evaluators, who found more usability problems for multiplayer game usability.

The heuristics were created from problem reports from many games and provide broad coverage of major game genres. Evaluators could use the heuristics after only a brief training session. They were able to find real usability problems related to multiplayer features.

The 10 networked game heuristics are:

  1. Provide simple session management.
  2. Provide flexible matchmaking.
  3. Provide appropriate communication tools.
  4. Support coordination.
  5. Provide meaningful awareness information.
  6. Give players identifiable avatars.
  7. Provide protected training for beginners.
  8. Support social interaction.
  9. Reduce game-based delays.
  10. Manage bad behaviour.

Read the full study: David Pinelle, Nelson Wong, Tadeusz Stach, and Carl Gutwin. 2009. Usability heuristics for networked multiplayer games. In Proceedings of the ACM 2009 international conference on Supporting group work (GROUP '09). Association for Computing Machinery, New York, NY, USA, 169–178. 

Join UX rockstars learning about games, research, and writing

Each Tuesday, I will send you tips for UX in game research and design. And once a month, I will send you emails about academic writing.

No hassle, no selling your information, no spam. You can unsubscribe anytime.