Biometrics, Psychophysiology, Research

Caution with Biometrics, Game Evaluation and UX

My col­league Steve Fairclough recent­ly post­ed an arti­cle on in which he dis­cuss­es the poten­tial pit­falls of bio­met­ric research and how it is cur­rent­ly being sold to the game indus­try. I will present some of his ideas here.

Steve out­lines that “psy­chophys­i­o­log­i­cal meth­ods are com­bined with com­put­er games in two types of con­text: applied psy­chol­o­gy research and game eval­u­a­tion in a com­mer­cial con­text.  With respect to the for­mer, a researcher may use a com­put­er game as a plat­form to study a psy­cho­log­i­cal con­cept, such as effects of game play on aggres­sion or how play­ing against a friend or a stranger influ­ences the expe­ri­ence of the play­er.”

Similar to Mike Ambinder’s pre­sen­ta­tion of user research and game design at Valve (PDF), he makes the point that games in this con­text are analysed using prin­ci­ples of exper­i­men­tal psy­chol­o­gy.

They are used as tasks or vir­tu­al worlds with­in which a research can study the behav­ior of play­ers (you might recall John Hopson’s Gamasutra arti­cle on behav­ioral game design).

He char­ac­teris­es the exper­i­men­tal psy­chol­o­gy approach by 4 fea­tures:

  1. Comparing con­trolled con­di­tions
  2. Importance of sta­tis­ti­cal pow­er (large N)
  3. Controlled par­tic­i­pant sam­ple
  4. Counterbalanced design (remov­ing order effects)

He makes a point about the sen­si­tiv­i­ty of phys­i­o­log­i­cal data as being volatile, vari­able and dif­fi­cult to inter­pret with­out a high lev­el of exper­i­men­tal con­trol. He warns that the use of think aloud pro­to­col might influ­ence the phys­i­o­log­i­cal data being record­ed because it influ­ences heart rate and res­pi­ra­tion con­sid­er­ably.

He also warns about the over­sim­pli­fi­ca­tion regard­ing the inter­pre­ta­tion of phys­i­o­log­i­cal data (some­thing I have seen way too often), regard­ing its one-to-many rela­tion­ship to psy­cho­log­i­cal impact.

For exam­ple, gal­van­ic skin response is used too often to infer emo­tion­al qual­i­ties although it is a high­ly ambigu­ous mea­sure regard­ing emo­tion­al labels.

Steve clos­es with a dis­cus­sion of the ques­tion of how to make phys­i­o­log­i­cal data and exper­i­men­ta­tion valu­able to the game indus­try. Meaning, what ques­tions can we answer with this type of data that the game indus­try does not already know?

This sum­ma­ry was first post­ed on my Gamasutra blog.

(Please read Steve’s orig­i­nal post here, too)