Biometrics, Psychophysiology, Research

Caution with Biometrics, Game Evaluation and UX

My col­league Steve Fairclough recent­ly post­ed an arti­cle on in which he dis­cuss­es the poten­tial pit­falls of bio­met­ric research and how it is cur­rent­ly being sold to the game indus­try. I will present some of his ideas here.

Steve out­lines that “psy­chophys­i­o­log­i­cal meth­ods are com­bined with com­put­er games in two types of con­text: applied psy­chol­o­gy research and game eval­u­a­tion in a com­mer­cial con­text.  With respect to the for­mer, a researcher may use a com­put­er game as a plat­form to study a psy­cho­log­i­cal con­cept, such as effects of game play on aggres­sion or how play­ing against a friend or a stranger influ­ences the expe­ri­ence of the play­er.”

Similar to Mike Ambinder’s pre­sen­ta­tion of user research and game design at Valve (PDF), he makes the point that games in this con­text are analysed using prin­ci­ples of exper­i­men­tal psy­chol­o­gy.

They are used as tasks or vir­tu­al worlds with­in which a research can study the behav­ior of play­ers (you might recall John Hopson’s Gamasutra arti­cle on behav­ioral game design).

He char­ac­teris­es the exper­i­men­tal psy­chol­o­gy approach by 4 fea­tures:

  1. Comparing con­trolled con­di­tions
  2. Importance of sta­tis­ti­cal pow­er (large N)
  3. Controlled par­tic­i­pant sam­ple
  4. Counterbalanced design (remov­ing order effects)

He makes a point about the sen­si­tiv­i­ty of phys­i­o­log­i­cal data as being volatile, vari­able and dif­fi­cult to inter­pret with­out a high lev­el of exper­i­men­tal con­trol. He warns that the use of think aloud pro­to­col might influ­ence the phys­i­o­log­i­cal data being record­ed because it influ­ences heart rate and res­pi­ra­tion con­sid­er­ably.

He also warns about the over­sim­pli­fi­ca­tion regard­ing the inter­pre­ta­tion of phys­i­o­log­i­cal data (some­thing I have seen way too often), regard­ing its one-to-many rela­tion­ship to psy­cho­log­i­cal impact.

For exam­ple, gal­van­ic skin response is used too often to infer emo­tion­al qual­i­ties although it is a high­ly ambigu­ous mea­sure regard­ing emo­tion­al labels.

Steve clos­es with a dis­cus­sion of the ques­tion of how to make phys­i­o­log­i­cal data and exper­i­men­ta­tion valu­able to the game indus­try. Meaning, what ques­tions can we answer with this type of data that the game indus­try does not already know?

This sum­ma­ry was first post­ed on my Gamasutra blog.

(Please read Steve’s orig­i­nal post here, too)

Advent Calendar 2009

21 - Game User Research: Making Games Better

Today, we have more of a slide col­lec­tion. But the main fea­tured pre­sen­ta­tion is the one from Graham McAllister, who is a researcher in video game usabil­i­ty at the University of Sussex in the UK and also runs the com­pa­ny Vertical Slice that spe­cial­izes in User Experience (UX; human per­spec­tive, not qual­i­ty assur­ance) test­ing for games. Most of what I have been research­ing in the past 4 years is already start­ing to be employed in prac­tice by them (quite fas­ci­nat­ing, real­ly).

First, he explains the dif­fer­ent mean­ings of UX jar­gon, such as usabil­i­ty (can I do it?), user expe­ri­ence (do I like it?), user inter­face (how does it look?), inter­ac­tion design (how is the inter­face used?). Then he men­tions that UX is a key fac­tor dri­ving review scores of games (not the tech­ni­cal func­tion­al­i­ty alone), which then dri­ve the sales. He backs up his claims with sales data. However, some games with good reviews may still fail finan­cial­ly. On the oth­er hand, games with bad reviews are not very like­ly to sell well. He then dis­cuss­es two case stud­ies (Assassin’s creed and Bioshock) in terms of suc­cess­ful design intent or game­play flaws. The rise of episod­ic gam­ing demands a high­er lev­el of qual­i­ty even for ver­ti­cal slices of games. He goes on to ana­lyze UX flaws of games defaced by gam­ing mag­a­zine reviews. Continue read­ing