How To Measure the ROI of Game UX [TATT #46]

the acagamic tip tuesday Jan 03, 2023
AI generated image of a professor ruminating over measurements.

In today's issue, I will explain to you measure the return on investment (ROI) of Game UX. Measuring the return on investment (ROI) is vital for determining the success of UX work on a game. Calculate ROI by subtracting the cost of an investment from its gain and then dividing it by the cost. To measure ROI accurately for game UX projects, researchers need to understand how players interact with their games to determine what improvements are effective.

Measuring the ROI of Game UX: Identify core KPIs, collect data, convert to metrics, calculate ROI.

You want to understand the value of investing in game UX to be able to communicate this to your company. Improved game UX can lead to increased conversion rates, higher player retention or reduced customer churn, improved brand loyalty and player advocacy, cost savings, reduced development risk, increased in-game purchases, and, ultimately, more revenue for your game.

Unfortunately, many researchers struggle to measure the ROI of their UX efforts effectively.

UX researchers lack clear goals and metrics

Researchers struggle to measure the ROI of game UX because...

  • They don't have a clear definition of what to track to measure good UX for their game.

  • They don't have a system in place to track key UX metrics.

  • They don't know how to connect UX improvements to business outcomes.

  • They don't have the necessary resources or support from their publisher.

Fret not. I'll help you understand how you can overcome these challenges and measure the ROI of game UX, so you can articulate this to management.

Define your UX goals and objectives

Start by clearly defining your UX goals and objectives for the game. Define measurable goals to measure ROI. Even better: make them SMART (Specific, Measurable, Achievable, Relevant, Time-Bound).

These should be aligned with your development studio's overall business goals and be measurable. For example, your goal might be to increase player retention (specific, relevant) by 25% (measurable) within the first month of launch (time-bound, achievable).

With this goal in mind, you can then define your UX research objectives. You could include improving the tutorial experience for new players, streamlining the in-game navigation, or increasing the visual appeal of the game. Just put them in the same SMART framework I used above.

Choose the right metrics to track

To measure the ROI of game UX, you need to have a system in place to track key metrics.

Start with a list of the most important key performance indicators (KPIs) that finance and sales usually track. These things relate to game UX, but also—and more importantly—to the business value of your game.

  • Revenue and Cost (Profit Margin). High revenue indicates user engagement and spending, key indicators of success. Cost is also a key metric to monitor because it can be used to calculate the profit margin achieved by your game. Calculate like this: Revenue = Total Sales - Associated Costs. For example, if you sold a game for $100 and had $20 in associated costs, your revenue would be: $100 - $20 = $80.

  • Conversion rate (of in-game purchases). It shows how effective the user experience is in enticing players to take the desired action (e.g., for free-to-play games). It is calculated by dividing the number of players who convert (e.g., purchase an in-game item or buy an add-on) by the total number of players exposed to that action. Conversion Rate = (# of Conversions / Total # of Players Exposed to Action) x 100. For example, if your game had 1000 people seeing an in-game item and 10 of them purchased it, the conversion rate would be: 1% (gets even trickier with in-game currencies like V-Bucks in Fortnite). Segmenting your data by different player groups will help you understand which types of players are most likely to convert and tailor your game UX accordingly.

  • Customer Lifetime Value (CLV). This measures the total value of a customer over their lifetime as an engaged user. Track current/future purchases, referrals & user actions for ROI. To calculate CLV, use this formula: CLV = (Average Value of a Sale) x (Number of Repeat Transactions) x (Average Retention Time in Months or Years). For example, if your game has a subscription value of $100, an average of 5 repeat transactions, and an average retention time of 2 years, the CLV would be: ($100) x (5) x (24) = $12,000. Understanding this metric can help inform strategies on how best to engage players for optimal long-term growth.

  • Net Promoter Score (NPS). Many things are wrong with NPS and really it's an outdated and inaccurate customer satisfaction measure you SHOULD NOT use. It basically asks customers how likely they are to recommend your game to others, from 0-10. You score it like this: NPS = (% of Promoters) - (% of Detractors). Promoters are customers who rate your game 9 or 10, while Detractors are customers who rate your game 0-6 on the same scale. Many people have written in many places online how useless it is. I list it here for its current popular adoption only.

  • Churn rate (vs. player retention). This measures the percentage of players who have stopped playing your game over a specified period of time. The formula is simple: Churn Rate = (# of Customers Lost / Total # of Customers) x 100. This can be used to determine how engaged players are with your game, as well as where improvements need to be made to increase engagement and reduce player attrition. Tracking churn rate will also help you understand which features are working best for your players, and which need more development or improvement to become more attractive to players.

  • Player engagement. This indicates how long and frequently players are playing your game, as well as how much they interact with it. To track player engagement, you should look at metrics such as average playtime per session, total playtime, number of sessions per day/week/month. A simplistic tracking formula could be: Session Engagement = (# of sessions played / Total # of players) x 100. For example, if your game had 100 players and they played a total of 500 sessions, their session engagement would be: (500 / 100) x 100 = 500%. It's vital to consider qualitative feedback from players, too, regarding their experience with the game. This is the only way you can tell why players may be engaging more or less than expected.

  • Overall satisfaction. You can track this by surveying players and having them rate their overall experience playing the game based on user interface design, gameplay mechanics, and ease of use. You could build your own surveys for this. The results from these surveys will help you determine how much value players are getting out of your game. Ideally you want this to be more fine-grained than NPS and allow for qualitative feedback.

  • Brand sentiment. You want to track how players feel about your game brand. It is especially useful for tracking user loyalty over time, because a positive sentiment towards your brand will lead to increased customer retention. I would track brand sentiment across social media, qualitatively monitor mentions and reviews of the game, track player engagement, and monitor your sales and customer loyalty. If the numbers improve after you make UX changes, you have your evidence.

  • Cost savings. Hard to measure for companies that only have one game. For all other companies, I would assess UX for each games and then calculate as: cost of game with worse UX - cost of game with better UX. Of course, costs savings can come from lower development costs because of more efficient production processes (not UX), reduced customer service costs because of fewer user complaints (UX) and better onboarding experiences (UX), or decreased marketing expenses as a result of higher engagement levels in-game (UX).

  • Risk reduction. Less risk could mean decreased abandonment rates or fewer users experiencing technical issues during gameplay, for example. It's hard to calculate because you need to assign a value to the risk first. Then subtract the value of the risk before playing the game from the value of the risk after playing the game. For example, a game company purchases a game engine. The engine is expected to reduce the development time by 50%. The company estimates that the average cost of developing a feature is $10,000. The risk reduction would then be calculated as follows: Risk reduction = ($10,000 * 50%) - $10,000 = $5,000. UX research can inform this process by looking into development tools (more market research than UX probably though).

It's important to choose the right metrics to track, as they should be directly related to your UX goals and objectives. You should also make sure to track these metrics over time, as UX improvements may not always have an immediate impact (as some mentioned above take several game iterations and years to compare).

Step 3: Analyze and interpret the data

Once you have a system in place to track your UX metrics, it's important to regularly analyze and interpret the data. Look for trends and patterns, and try to connect UX improvements to changes in your metrics. Identify core KPIs, collect data, convert to metrics, calculate ROI.

For example, if you implemented a new tutorial experience and saw an increase in player retention, you can attribute some of that increase to the improved tutorial experience.

Step 4: Calculate the ROI

With the data you have collected and analyzed, you can now calculate the ROI of your UX efforts. To do this, you'll need to consider both the costs associated with improving your game's UX and the resulting revenue or business outcomes.

For example, if you spent $10,000 on UX improvements and saw a 25% increase in player retention, which resulted in an additional $50,000 in revenue, your ROI would be 400%. You formulas will vary depending on your business priorities.

By following these steps and regularly measuring the ROI of game UX, you'll be able to make informed decisions about how to allocate your resources and maximize the value of your UX efforts. Tracking and calculating this regularly will also help you advocate for game UX within your company.

Games Research Find of the Week

When it comes to UX design, measuring the return on investment can be a tricky task. This paper focuses on the importance of metrics for designers and how to effectively use them to demonstrate to decision-makers the connection between user experience and customer value. Key takeaways are that good design produces customer value, Usability ROI metrics should be tied to Total Cost of Ownership, and good metrics should be strategically aligned, drive action, be important to stakeholders, and have a process in place to change things. 

Traditional ROI measurements for UX don't typically address business concerns. The author urges us to identify objectives driving company action and measure UX's contribution to them to calculate ROI. They advocate the use of the Balanced Scorecard (BSC) approach to align UX activities with company goals. BSC's four categories of measures (customer, internal business, innovation and learning perspectives) link to financial performance through strategy maps. UX designers can contribute to strategic decisions using all four perspectives. There are many ways to measure the return on investment of UX activities, not just through metrics. Design and design process can solve business problems, including UX.

Read the full study: Carl W. Turner. 2011. A Strategic Approach to Metrics for User Experience Designers. J. Usability Studies 6, 2 (February 2011), 52–59. 





Join UX rockstars learning about games, research, and writing

Each Tuesday, I will send you tips for UX in game research and design. And once a month, I will send you emails about academic writing.

No hassle, no selling your information, no spam. You can unsubscribe anytime.