Measurements for player experience

K.M.K Dissanayake University of Colombo School of Computing

June 13, 2011

Author :

K.M.K Dissanayake 08000352 2008CS35 kalanadis@gmail.com +94726108706

Supervisors name : Mr. Manjusri Wickramasinghe

Measurements for player experience

Declaration
I hereby declare that this literature survey report has been prepared by Kalana Mahendra Krishantha Dissanayake based on mainly the reference material listed under the bibliography of this report. No major components (sentences/paragraphs etc.) of other publications are directly inserted into this report without being duly cited. Name of Candidate: K.M.K Dissanayake Signature of Candidate: .................................................. Date: 13 June, 2011

K.M.K Dissanayake

Measurements for player experience

Abstract Lately the term Game experience has been emerged among the research community. Understanding and dening game experience is one of the biggest challenges that research community must face. Measurements taken within the game software (eg: speed of a player in a car game) and Psychophysiological measurements taken from the player (eg: skin conduction level) are used to craft player experience of a playable game. The aim of this survey is to identify the measuring techniques to craft player experience.

K.M.K Dissanayake

Measurements for player experience

Acknowledgements
I would like to thank my supervisor, Mr. Manjusri Wickramasinghe, whose encouragement, supervision and support from the preliminary to the concluding level enabled me to develop an understanding of the subject and guiding me on writing the Literature survey . Also I would like to thankful to Coordinator of the Literature Survey, Dr. H.L. Premarathne for providing a great support during the survey and would like to thankful to UCSC library for giving me chance to refer past surveys which are done by previous year students. Lastly, I oer my regards and blessings to my parents, colleagues and all of those who supported me in any respect during the completion of the project.

K.M.K Dissanayake

Measurements for player experience

Contents
1 Introduction 1.1 Player experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Playability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Towards Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Measuring Playability 2.1 User Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Expert Review Method (Heuristics) . . . . . . . . . . . . . . . . . . . . . 3 Measuring Experience 3.1 Measurements form Games (Gameplay Metrics) . . . . . . . . . . . . 3.1.1 Anatomy of a Gameplay metrics suite of an online game . . . 3.1.2 Analysing Gameplay Metrics . . . . . . . . . . . . . . . . . . 3.1.3 An Application of Game metrics : dening personas in games 3.2 Measurements form Player (Bio Metrics) . . . . . . . . . . . . . . . . 3.2.1 Measuring Emotions . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Measuring Attention . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Measurement tools . . . . . . . . . . . . . . . . . . . . . . . . 4 Discussion 5 Bibliography 6 6 7 7 9 9 9 11 11 12 12 14 15 15 15 15 16 18

. . . . . . . .

. . . . . . . .

List of Figures
1 2 3 4 5 6 7 8 9 Player experience . . . . . . . . playability and experience . . . player experience . . . . . . . . Mobil heuristics . . . . . . . . . Gameplay Metrics . . . . . . . . Anatomy of a Gameplay metrics GIS . . . . . . . . . . . . . . . . Gameplay metrics . . . . . . . . Measuring Emotions . . . . . . . . . . . . . . . . . . . . . suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . of an online . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 8 8 10 11 12 13 13 16

K.M.K Dissanayake

Measurements for player experience

1
1.1

Introduction
Player experience

As computer games moves in to new dimensions designers are moving towards the players to create innovative games. As a larger number of video games mushrooms in the market, user experience is beginning to be the main component which determines the success of a game. Marc Hassenzahl,a leading researcher in UX(user experience) denes user experience as a momentary, primary evaluative feeling (good-bad) while interacting with a product or service[1].John Ferrara in UX magazine talks about ve components in player experience[2].(Figure 1)

Figure 1: Player experience

K.M.K Dissanayake

Measurements for player experience

John states Motivation (driving the player forward by giving rewards while keeping things interesting ) , Meaningful choices (making things easy to the players who plays using strategies and tactics ), Balance (it cant be too easy or frustrating),Usability (easy to master and control and players should understand whats going on ) ,Aesthetics (sounds , graphics, etc.) as the elements of player experience. Flow and Immersion are internal factors that aect player experience. Flow is dened by Csikszentmihalyi (1975) as holistic sensation that people feel when they act with total involvement [3].In order to keep the ow in a game there must be a balance between challenge (which the player is facing while playing) and skill (of the player to face the challenge)[4]. Still the term player experience is not well dened. Though there are many denitions, still there is no commonly accepted denition. In order to understand and improve player experience, it should be measured and more research should be done.

1.2

Playability

Usability of a software product is eectiveness,eciency, and user satisfaction in a specied context of use.Even we dont have a commonly agreed denition for playability, we can say it is closely related to Usability even they are not the same. A game is said to be playable when the game is understandable, balanced, suitably dicult and engaging[13]. The initial bond between player and game depends on playability. As literature suggests [5, 6] playability depends on players former experience and players present experience while playing. If a player wants to play a certain game its because of his past experience and if he wants to keep playing its because of his present experience .Still the term playability is not well dened.

1.3

Towards Measurements

To understand player experience researchers are looking for answers to what and why. By linking these two facts together a clear picture of the player experience can be created[7, 8]. Lennart Nacke at el. suggests that playability is a prerequisite for evaluating a game for player experience[7]. Playability must be kept when designing a game. Evaluation of playability is used to improve design while player experience evaluates players to improve gaming. Even though player experience and playability are not clearly dened or divided from each other, playability and player experience can be considered as two separate things K.M.K Dissanayake 7

Measurements for player experience

Figure 2: playability and experience which depend on each other[5, 7, 8]. Player experience can be evaluated using Qualitative and Quantitative measurements taken from players (eg: Heart rate) and game software (eg: number of kills) .There are number of measurements we can get from player and games (eg: physiological measurements, expert evaluation (heuristics etc.), subjective, self-reported measures and usability tests).

Figure 3: player experience Section 2 is focused on measuring playability discussing user studies and expert reK.M.K Dissanayake 8

Measurements for player experience

view methods. Section 3 is focused on measuring experience using game metrics and biometrics.

2
2.1

Measuring Playability
User Studies

By involving user throughout the development cycle to evaluate the playability of a game, designers can build playable games. Data are gathered mainly by using direct observations, verbal reports (think aloud protocols), Q and As and Surveys[9, 10]. Even though user studies are the best place to get accurate results, because of high cost and time its not widely used.

2.2

Expert Review Method (Heuristics)

Heuristics are a set of design guidelines which used by designers and usability professionals as an evaluation tool. Because of cost eectiveness and eciency, heuristics are widely used in software industry to evaluate user interaction.When it comes to games, a special set of heuristics are needed to evaluate playability. In user interaction design designer focuses on making users to do their tasks quickly and easily. In game designing its not the case. The goal of UI designers is to make the interface easy to learn, use and master. In games the interface should be easy to learn but dicult to master[11]. Desurvire et al. [11] and Korhonen and Kovisto [12] are two of the most advanced Heuristic sets at the moment[13]. Heuristics Evaluation for Playability or HEP[11] had been developed using current literature and had been reviewed by several usability experts and designers. Four categories are dened as game play (problems and challenges a user must face to win), game story (plot and character development), game mechanics (programming and interaction), game usability (interface). Playability Heuristics for mobile games[12] is dened as three modules as Game play, Game Usability, Mobility. Mobility module can be excluded or included depending on the game (eg: Mobility module is excluded when evaluating a computer game). Game play and Game usability are common to all games. Playability experts use heuristics as a evaluation tool for playability of a game. The methodology is simple. The playability expert plays the game and check for violated or supported features using the pre dened heuristics set. For an example the 7th heuristic of the usability category in HEP[11] says Sounds from the game provide meaningful feedback or stir a particular emotion if the evaluator feels like the heuristic is preserved he will say mark that as supported.

K.M.K Dissanayake

Measurements for player experience

Figure 4: Mobil heuristics Though Expert review method is a cost eective way to evaluate playability the results depends on experts skill and experience.

K.M.K Dissanayake

10

Measurements for player experience

3
3.1

Measuring Experience
Measurements form Games (Gameplay Metrics)

Gameplay metrics are a sub set of game metrics(Figure 7)[14].

Figure 5: Gameplay Metrics In order to analyze player behavior designers are focusing on Game play metrics. Gameplay metrics can be dened as numerical data obtained from player interaction with the game software. Gameplay metrics can be used during the development phases and after the game launch. Using game metrics original player behavior can be analyzed without interrupting the player. In a typical user study at least the players actions are observed directly or indirectly (using cameras) and typically their natural environments are changed, this could aect the result. By integrating game metrics with the game, accurate results could be obtained. Game metrics can be used eciently to measure online games. Because data can be collected remotely this can be easily used to evaluate a large online player community and accurate results could be obtained. Gameplay metrics can be sub-divided as Interface ,In-game and System( all the actions of game engines to respond player actions )[14].Interface metrics is used to measure interface of the game by recording players interactions with the interface and menus(eg: setting monitor brightness).In-game metrics is used to record all the actions and behaviors of the player in the game(eg: Navigation ).System metrics are used to record the K.M.K Dissanayake 11

Measurements for player experience

actions of the game engines in respond to the player(eg: Automated events).Analyzing these three metrics the game play of a player can be evaluated.

3.1.1

Anatomy of a Gameplay metrics suite of an online game

Following is a anatomy of a Gameplay metrics suite published by AGORA Informatics[15]. A Gameplay metrics suite is a combination of client library, server library, and Front end tools.

Figure 6: Anatomy of a Gameplay metrics suite of an online game Client library is used by the game developers to establish the communication between the game and the data hosting server.The connection of the game can be establish using Fire and forget method and/or Reliable metrics method. In Fire and forget method data are stored locally as queues.Data are transmitted to the server without receiving a conrmation from the server.And when the queues are full new data will overwrite old data.Even though memory consumption is limited data losses can occur. In Reliable metrics method data are transmitted on conrmation of the server.In case of a connection lost,data will be kept with the client. Server library collects and stores data transmitted by clients.Because of high input rates data cannot be directly saved to a database rst they are stored in a le system. Front end is used by the game companies to analyze and visualize data. Data is analyzed using spread sheet tools and more advanced tools like SPSS. Simple spread sheet tools and advanced Geographic Information Systems (GIS) are used for visualization .GISs can be easily used to multi-layered multi-variable analyses and visualizations.(Figure 9)

3.1.2

Analysing Gameplay Metrics

Data from the gameplay metrics can be continues (eg: movement of an avatar) ,recored using specic frequencies (eg: position of avatars in every 2 seconds ) or triggered (eg: K.M.K Dissanayake 12

Measurements for player experience

Figure 7: GIS

Figure 8: Gameplay metrics every time an avatar shoots).They can be visualized in spatial domain or non-spatial domains.There are many types of metrics,some examples are Navigation metrics,Health metrics,Crouch and cover metrics.Speed metrics[17]. Since there are no commonly known ways to output data from game play tracking software various ways are used.Following is an example of a set of triggered metrics. 00000: 00001: 00002: 00003: (x, y, z), (t), (w), (, ), (m), (i) (135, 451, 00), (0079), (111), (100, 75), (i1) (155, 422, 05), (0148), (126), (110, 90), (i3) (193, 420, 10), (0210), (141), (180, 110), (w3)

Data about position, time, camera angle, metric type is recorded in these metrics. K.M.K Dissanayake 13

Measurements for player experience

Typically recoded data are parsed to structure and ltered. Then patterns are located using data mining or statistical methods. Then data are represented in visual models (eg:GIS).

3.1.3

An Application of Game metrics : dening personas in games

Players in the real world interact with game world using avatars. Avatars can have various features such as appearance, animation, movements, etc. and even a personality. A avatar with all or some above features are here referred as a character. Player character features can be tracked using Anderss and Alessandros elements[16].Player character is a combination of Navigation (records about characters navigation) ,Narrative(deals with story and how the character navigates through it), Interaction(triggered by character actions), and Interface metrics. To make things ecient we must focus on three aspects of the character as play mode(eg:In Hit Man weapons used by the player lethal or non-lethal),play style(a set of composite play modes) and play persona(a pattern of play styles).

K.M.K Dissanayake

14

Measurements for player experience

3.2

Measurements form Player (Bio Metrics)

Psychophysiological measurements are physiological measurements taken in order to study psychological phenomena. There are two types of Psychophysiological measurements as tonic (aggregate signal levels during a long period of play) and phasic (event based).Studies are mainly focused on emotions and attention of a player[19].

3.2.1

Measuring Emotions

All emotions lie on a two dimensional coordination system of valence(pleasant to unpleasant) and arousal(exited/energized to calm/sleepy)[18]. Electro dermal activity (EDA) or skin conductance is a psychophysiological index of arousal. Arousal activates sympathetic nervous system resulting increased sweat gland activity and skin conductance. Facial electromyography (EMG) is a psychophysiological index of hedonic valence. Though cardiac activity (eg: heart rate) is a well known and widely used physiological measure because of the sensitivity to many bodily process its not easy to interpret the measurements.Figure 11 illustrates mapping of emotions using EDA and EMG[9].

3.2.2

Measuring Attention

When mental resources are allocated towards a game, physiological changes occur. By using measurements of these changes (eg: EMG,EDA).Electroencephalography or EEG (data about the brains electrical activity) can be used eectively measure attention. Eye tracking can be used to measure visual attention.

3.2.3

Measurement tools

Electromyography (EMG) can be used to capture emotion and blinking of a player. By placing sensors on muscles activation of muscles can be determined. Mostly sensors are placed on forehead to determine the mood of the player.Though hard ware is expensive EMG is an eective measuring technique which is more sensitive than image processing techniques and the signals are easy to interpret. Galvanic skin response (GSR) can be used to capture excitement, arousal and engagement of a player. By placing two electrodes on two close ngers electrical resistance between the two electrodes are measured.GSR is an inexpensive easy and non-instructive method. Electroencephalogram (EEG) can be used capture the brain waves of a player. By placing 20 to 256 electrodes on scalp EEG can be used to capture brain waves of a player. K.M.K Dissanayake 15

Measurements for player experience

Figure 9: Measuring Emotions Table 1 describes the brain waves. Eye tracking can be used to capture visual attention, blinking and cognition of a player. Measurements are taken about the movement of eye(fast movements, dwell times),pupil dilation and blink rate.

Discussion

Traditionally Game testing has been a similar process to software testing. Most methods and concepts for deigning and testing games were borrowed from HCI and user experience community. Because of the incompatibilities of game development and software development aspects, new set game developmental disciplines are developing within the game research community. Even the HCI community is moving towards gamication. In some literature playability is dened more related to player experience which doesnt seem to be acceptable. Because the complexity of both playability and player experience concepts its better to keep them separated in order develop enjoyable games. Merging usability and playability is seemed to be acceptable. K.M.K Dissanayake 16

Measurements for player experience

Table 1: Brain waves of a player Brainwave Frequency band Describes Delta 1-4 Hz trance, sleep Theta 4-8 Hz emotions, sensations Alpha 8-14 Hz calm, mental work Low beta 14- 30 Hz focus, relaxed Mid beta 14- 30 Hz thinking, alert High beta 14- 30 Hz alert, agitated Gamma 30-50 Hz information processing

Clearly a new set of playability heuristics must be developed which the result will be independent from evaluator. As Hannu Korhonen et al.[13] shows from a study, both HEP[11]. a Mobil heuristics[12] should be improved. Because of the large number and unclearness of the heuristics accurate results cant be obtained. In order to evaluate a game with heuristics the playability expert must play the game focusing on supportive and violated featured of the game, therefore the expert cant experience the game as a player. So its obvious that we cant use expert review method to evaluate player experience but can be used to evaluate playability. Although the player has found his own identity in the user world still he hasnt distinguish or dened his area. Even the most crucial component of the success of a game player experience isnt following a common denition. More research on player experience must be done. A vocabulary and units must be identied to describe player experience more clearly. Marc Hassenzahlsdenition[1]for user experience is too narrow for player experience because he states about a primary feeling of good or bad which isnt true for a game, a player can even experience fear. Although most of the John Ferraras components of experience [2].(Figure 1) is acceptable usability component should be eliminated as its been evaluated under playability. Eective techniques to integrate games with Gameplay metrics must be introduced. In order to record continues data in online games communication techniques and protocols must be developed. Gameplay metrics must be developed as an eective game evaluation tool for online games. Psychophysiological measuring techniques must be studied and developed. The main problem with psychophysiological measurements is the complexity of interpreting. Experts from physiological, psychological and gaming areas must work together in order to develop a standardized bio metric system. If physiological measuring equipment could be integrated in day to day playing devices (eg: game pad, wheel, mouse) it would provide an eective way for evaluating online games and standalone games.

K.M.K Dissanayake

17

Measurements for player experience

The player must feel the game and he must believe the game. In other words if the player is driving a car in a virtual environment he must feel the risk of speeding as he was in the real environment. The challenge of the developer community is to make the game experience more natural. Path for natural games is through player experience therefore understanding and measuring player experience is important towards the development of Gaming.

Bibliography

References
[1] Leitner Jakob Haller Michael Geven Arjan Koeel Christina, Hochleitner Wolfgang and Tscheligi Manfred. using heuristics to evaluate the overall user experience of video games and advanced interaction games. 2009. [2] John Ferrara. kuxmagazine the elements of player http://uxmag.com/design/the-elements-of-player-experience, 2011. experience.

[3] Karolien Poels Audrius Jurgelionis Wijnand IJsselsteijn, Yvonne de Kort and Francesco Bellotti. characterising and measuring user experiences in digital games. ACE Conference07, 2007. [4] Lennart Nacke and Craig A. Lindley. ow and immersion in rst-person shooters: Measuring the players gameplay experience. Computer Systems Laboratory, Stanford University. [5] Julian Kcklich. play and playability as key concepts in new media studies . [6] Francisco Lus Gutirrez Vela Jos Lus Gonzlez Snchez. bility: What a video game needs to be http://www.sciencedaily.com/releases/2011/03/110310082132.htm. playasuccessful.

[7] Kai Kuikkaniemi Joerg Niesenhaus Hannu J. Korhonen Wouter M. van den Hoogen Karolien Poels Wijnand A. IJsselsteijn Lennart E. Nacke, Anders Drachen and Yvonne A. W. de Kort. playability and player experience research . Digital Games Research Association (DiGRA), 2009. [8] Anders Tychsen. crafting user experience via game metrics analysis. 2008. [9] Canossa A. Mandryk R. Nacke L., Ambinder M. and T. Stach. game metrics and biometrics: The future of player experience research. Future Play 2009, 2009. [10] Lennart E. Nacke. next generation testing:biometric analysis of player experience. Game Developers Conference Canada, 2008. K.M.K Dissanayake 18

Measurements for player experience

[11] Martin Caplan Heather Desurvire and Jozsef A. Toth. using heuristics to evaluate the playability of games. CHI 2004. [12] Elina M.I. Koivisto Hannu Korhonen. playability heuristics for mobile games . MobileHCI06, 2006. [13] Janne Paavilainen Hannu Korhonen and Hannamari Saarenp. expert review method in game evaluations comparison of two playability heuristic sets. MindTrek 2009. [14] Carsten Srensen Alessandro Canossa Anders Drachen, Thomas Hagen and Morten Heiberg. introduction to game metrics. 2010. [15] Carsten Srensen Alessandro Canossa Anders Drachen, Thomas Hagen and Morten Heiberg. anatomy of a gameplay metrics suite. 2010. [16] Anders Tychsen and Alessandro Canossa. dening personas in games using metrics. FuturePlay 2008. [17] Anders Drachen and Alessandro Canossa. towards gameplay analysis via gameplay metrics. MindTrek 2009. [18] Mikko Salminen Jari Laarni Niklas Ravaja, Timo Saari and Kari Kallinen. phasic emotional reactions to video game events: A psychophysiological investigation . 2006. [19] Guillaume Chanel Simo Jrvel Ben Cowley Mikko Salminen Pentti Henttonen J. Matias Kivikangas, Inger Ekman and Niklas Ravaja. review on psychophysiological methods in game research. DiGRA 2010.

K.M.K Dissanayake

19