How to Execute the Rapid Iterative Test and Evaluation Method (RITE) [TATT #41]

the acagamic tip tuesday Nov 29, 2022
An image of Age of Empires 4 in the background. In the foreground, the newsletter title points to a figure showing a record of errors over time as changes to the Age of Empires II tutorial were made using the RITE method.

In today's issue 41 of The Acagamic Tip Tuesday, I will show you how to conduct rapid iterative testing and evaluation (RITE)—a method developed by Michael Medlock and colleagues—on your games to identify issues in the software early on and work with developers to solve them quickly. 

RITE Method: A playtesting method that requires data to be analyzed after each participant or at least after each day of testing. Changes to the game are made immediately after identifying an issue.

Benefits of Rapid Iterative Test and Evaluation (RITE)

The Rapid Iterative Test and Evaluation Method (RITE) is an iterative usability testing method that helps UX researchers detect gameplay or usability problems early in the design process. This method is advantageous because it allows for more effective use of resources and faster feedback from users to ensure any user experience issues are addressed quickly. It can reduce development costs by eliminating costly redesigns. RITE can also save time by allowing developers to identify and address potential usability issues before they become serious problems later in the design process. Most importantly, using RITE encourages collaboration between stakeholders and provides immediate feedback from users, which makes it easier to make changes based on their input. It can result in improved product quality with fewer resources expended overall.

The RITE process involves selecting participants, having them complete tasks on the game or prototype being tested, conducting interviews about their experience and gathering feedback. After each participant has completed testing, any changes identified by the testers can be implemented quickly and efficiently into the design, allowing for an iterative approach to refining designs until all desired goals are achieved.

RITE can save valuable time and resources down the line

So, RITE leads to increased efficiency in the testing process, improved quality of the product, and more accurate estimates of project timelines (meaning your development estimates are more likely to meet customer expectations). RITE allows teams involved in creating video games to rapidly ideate solutions while also validating their assumptions through feedback from actual users.

Unfortunately, RITE may seem like more effort upfront, and many game changes are often made without any form of testing, such as observing a real player interacting with the game. This means designers can make mistakes or overlook potential issues that could have been discovered through playtesting. A lack of knowledge and understanding about RITE is why so many people don't use it in game development projects.

To overcome this issue, it is crucial to become familiar with RITE's core concepts.

The RITE Process Steps

Here's how to execute the RITE process:

Once you identify an issue with the video game, it is essential to address it quickly. Depending on the nature of the problem, there are different ways to approach resolving it. For example, suppose a user cannot find certain features within your game. In that case, you might need to redesign the user interface or provide more detailed instructions or tutorials for them to understand how they can access those features. If a bug has been discovered that affects functionality, then a solution should be tested and implemented as soon as possible so that players will not experience any disruption in their gameplay. After resolving each problem, review what was done and evaluate whether further changes could enhance user experience even more - this is an essential step in creating successful games through RITE.

It is also vital to ensure decision-makers are part of the game UX research team. This allows for a more effective feedback loop between those conducting the research and those making decisions based on its results. Decision-makers can provide valuable insight into potential solutions or changes in direction while also understanding any limitations posed by resources or timelines. Additionally, having decision-makers involved may help accelerate the process as they will be able to quickly assess findings and make well-informed decisions further down the line.

Example Timeline of a Three-Week RITE Test

Notice how a week of planning and scheduling is required in the following timeline. Then tests are run every other day, alternating with implementing changes stemming from test findings.

  1. Plan (3 days)
  2. Schedule Participants (4 days)
  3. Run Participants and Analyze Data (1 day)
  4. Fix Issues (1 day)
  5. Repeat steps 3 and 4 at least five times.

What Makes RITE, Right?

RITE shares the same four principles as usability tests:

  • Improve the product.
  • Real users participate.
  • Participants perform tasks.
  • Observe and record participant behaviour.

Unlike traditional usability tests, which typically involve collecting data from all participants at the end of the test, RITE requires data to be analyzed after each participant or at least after each day of testing. Changes to the video game are made immediately after identifying an issue. This means that instead of waiting until all participants have completed their testing, any potential solutions can be implemented as soon as there is clarity around them. In cases where a problem and its solution are both straightforward, only one participant may even need to complete the test before making the change.

While this approach may mean more frequent modifications to address each individual concern, it ultimately results in a product with improved playability and usability overall. It also involves testing a changed game with subsequent players to see if the changes solve any existing problems without creating new ones.

The Advantages of Using RITE

RITE enables more issues to be identified and fixed in a shorter period of time. This is achieved because the test process is broken into smaller parts and performed iteratively. During each iteration, new tests are conducted on the game being tested so any issues can be quickly identified and addressed. As this process continues over several iterations, more issues can be pinpointed and corrected efficiently, resulting in fewer errors or bugs when the testing phase is complete. Furthermore, with RITE’s iterative approach, it is possible to discover even more issues that may have been overlooked during traditional testing.

RITE has the potential to improve team morale and dynamics, too. Creating an environment where all team members collaborate closely on a project encourages people to work together more effectively as a unit. This can help create a strong camaraderie among team members that would not be possible with other methods. Furthermore, this type of deep collaboration allows for better problem-solving skills within the group as each member contributes their own expertise and ideas toward solving any given issue. RITE can help build a positive and productive atmosphere between colleagues, leading to increased efficiency in completing projects or tasks. To be most effective, the real decision-makers have to be identified for inclusion in the team.

Games Research Find of the Week

This paper discusses the RITE method introduced above, which was then used by some practitioners but had not been formally defined. The paper evaluates the effectiveness of this method. They mention RITE is a testing technique designed to identify problems quickly, to achieve a high ratio of found problems to fixes made. The motivation for this was simple: Once it's determined a discount usability method is needed, the team must prioritize fixing problems and assessing if the “fix” resolved them over uncovering every problem.

The paper also looks at the application of RITE to the tutorial for Age of Empires II, a popular real-time strategy game. After using RITE, developers found that it was highly effective in identifying issues with the tutorial and fixing them quickly. This was done is a study of 16 people (aged 25 to 44). One of the limitations of traditional usability testing is that it can be difficult to tell which "fixes" are not effective due to time or resource constraints.  RITE reveals “fixes” that need reworking.

This resulted in positive reviews from industry professionals about how well the tutorial performed compared to other tutorials on similar games. The success of using RITE on Age of Empires II can serve as an example for others looking to deploy this method when creating tutorials or other interactive software applications. RITE can identify and fix as many issues as possible and verify the effectiveness of these fixes in the shortest possible time.

Read the full study: Medlock, M. C., Wixon, D., Terrano, M., Romero, R., & Fulton, B. (2002). Using the RITE method to improve products: A definition and a case study. Usability Professionals Association, 51, DOC, PDF.

Join UX rockstars learning about games, research, and writing

Each Tuesday, I will send you tips for UX in game research and design. And once a month, I will send you emails about academic writing.

No hassle, no selling your information, no spam. You can unsubscribe anytime.