Three Cool Things You Wish You Knew Before About Bibliometrics

by The Acagamic

Biblio... WTF?
Yeah, you think, here is the sci­ence guy with his words again, but actu­ally bib­lio­met­rics is a a good set of meth­ods to under­stand and study research in your field of inter­est. Usually, all it requires is a pub­li­ca­tion search engine, like Scirus, ISI Web of Science (you need an insti­tu­tional login), Google Scholar, BioMed Central, DOAJ, OAIster, or Faceted DBLP. And then you search your favorite terms and count the results in dif­fer­ent clus­ters. You put it in your favorite spread­sheet soft­ware and plot a graph to see a gen­eral trend and compare.

Example: HCI & Game Bibliometrics
I was actu­ally inspired to do this by Gareth over at Game Player Interaction, who always finds way more pub­li­ca­tions than you are able to read and who recently posted a DBLP search query on games. So, I sat down and gave my 5 favorite search terms a try in Faceted DBLP (which focuses on com­puter sci­ence). First, I put in “game* play” (note that the aster­isk usu­ally indi­cates any kind of affix), usabililty, hci (human-computer inter­ac­tion), game­play, and finally “dig­i­tal games”. The fol­low­ing chart shows how they compare:

Game Bibliometrics

First, we notice of course the ris­ing trend in the amount of research papers pub­lished that con­tain the key­words game and play. Second, we see some sort of sim­i­lar trend for usabil­ity papers, also ris­ing more slowly. Then, when look­ing at the hci data, we instantly notice two things: (1) the trend is not a straight line, but vibrates peri­od­i­cally (we call that oscil­la­tory), (2) which also means it shows spikes, one of which is rather large for 2007. Then again, if you look at the curves usabil­ity, games and play you also see a spike in 2007. I was puz­zled. Something must have hap­pened in 2007 that must have shook the games inter­ac­tion and espe­cially the HCI com­mu­nity that year. Something big.

Exploring the Statistics
To find out what it was, I had a look at more sta­tis­tics related to the search queries. For exam­ple, what pub­li­ca­tion venues were listed for dig­i­tal games? It is clear that this search term is usu­ally used by the DiGRA com­mu­nity, so as no sur­prise, 86.7% of all listed dig­i­tal games pub­li­ca­tions where pub­lished at a DiGRA con­fer­ence. Then, I looked at my game­play key­word, at it turns out — to my sur­prise actu­ally — that the main pub­lish­ing venue for this was the International con­fer­ence on Advances in Computer Entertainment Technology (ACE) with 21.7% of all pub­li­ca­tions listed there. This was inter­est­ing, but did not give me the answer I searched for: What hap­pened in 2007?

The Secret of HCI
You would of course nat­u­rally pro­ceed and look at the hci and usabil­ity key­words next, which I did. Turns out, the two biggest HCI con­fer­ences are HCI International with 41.5% of all hci pub­li­ca­tions listed there and INTERACT con­fer­ence with 11% pub­li­ca­tion share. For usabil­ity 7.6% of all pub­li­ca­tions come from the CHI con­fer­ence, which is only a bit more than 7.4% that come from HCI International (which also shows that there is no estab­lished sci­en­tific venue for usabil­ity research as all other pub­li­ca­tions are shat­tered over dif­fer­ent jour­nals and con­fer­ences). The HCI International con­fer­ence was founded in 1984 and is held every two years, then in 1995 the INTERACT con­fer­ence was estab­lished and is also held every other year. And this is why we have this wave­form of the curve for the hci key­word (- does this also mean that human-computer inter­ac­tion researchers pro­duce their research out­comes in waves?). Further inves­ti­ga­tion on the HCI con­fer­ence web­site yielded an final insight to the ques­tion what hap­pened in 2007!

Something Chinese
Yes, you remem­ber the olympic games — that minor event there last year. Turns out, China (Beijing) was also host­ing the HCI International con­fer­ence in 2007 jointly with a bunch of other related con­fer­ences. To quote them “this event was one of the biggest ever orga­nized in the fields related to Human-Computer Interaction and Information Society Technologies”. 2300 par­tic­i­pants sounds truly olympic to me for a sci­en­tific con­fer­ence. But there it is, the answer to a ques­tion, we never would have asked had we not looked at some sta­tis­tics. :)

3 Cool Things You Know Now About Bibliometrics

  1. You are able to see a research trend in num­bers of pub­li­ca­tions (games and usabil­ity have both ris­ing pub­li­ca­tion numbers)
  2. You under­stand what con­fer­ence venues relate to what key­words (we know that HCI International is the biggest HCI conference)
  3. You under­stand a gen­eral ten­dency in pub­li­ca­tion fre­quency in your research field (HCI research is pub­lished every other year)

Readers Who Liked This Post Also Read