Mental Workload and Eye Movement 2004 6 28

Mental Workload and Eye Movement 2004. 6. 28 (Mon) NICIELab seminar Chang Hoon Ha 1

Contents • • • Introduction Properties of Workload Measures Evaluation of Approaches to Measuring Mental Workload Comparison of Subjective Approaches Eye Movement Parameters Characteristics of Eye Movement Parameters Discussion & Conclusion Further Study References 2

Introduction • • Mental workload has both static and dynamic attributes, which reflect respectively, the mental workload at a single moment and within a time interval. (1) All the subjective measures most widely used and accepted cannot reflect the dynamic aspects of mental workload. So, complementarily physiological measure should be used to show the dynamic aspects of mental workload. • And physiological measure cannot tell us about the nature of the mental workload. This measure is naturally lack of diagnosticity giving only the implicit clues of mental workload. • If there is some peaking load point in overall task, physiological measure may nor reveal the cause of that peak workload. Then, we have to use the diagnostic subjectivemethod to know the causes of that peak. Instantaneous workload over time Peak workload Average workload Accumulated workload 3 Attributes of mental workload Time

Properties of Workload Measures Sensitivity How well a measure detects changes in the mental workload experienced by an operator Diagnosticity How precisely a measure can reveal the nature of mental workload Intrusiveness If taking the measure interferes with the performance of the task, then a contaminated workload measure is obtained Validity Whether the workload measure is measuring mental workload Reliability Whether the workload measure is stable and consistent over a period of time Ease of use How easy it is to collect and analyze the data associated with a measure Operator acceptance Crucial factor in the successful use of mental workload measures (8) 4

Evaluation of Approaches to Measuring Mental Workload Method Merits/ Demerits Available case Unavailable case - High face fidelity, Operator acceptance, Ease of use, high transferability / - Susceptible to memory problems, operator bias, and operator’s experience & degree of familiarity - Delays of up to 30 minutes in workload reporting don’t lead to significant differences. (9) Almost all the cases - Use operator behavior to determine workload (Primary & Secondary task measure) Subjective Operators rate the level of mental effort that they feel (immediately or retrospectively after the task) Primary task measure: - Face validity, Direct objective measure / - Insensitive to extremely low workload level Secondary task measure: - Sensitive to low workload level, Diagnosticity / - Intrusive, Requiring high background knowledge & experience Primary task measure: when the primary task is specific to the system evaluated Primary task measure: Highly automated system, Monitoring and supervising role, Very low workload level Measure changes in the operator physiology that are associated with cognitive task demands - No-interference, Continuously available measure / - Implicit measure - - Performanc e Psychophys iological 5

Evaluation of Approaches to Measuring Mental Workload Sensitivity Diagnost icity Intrusiv eness Validity Reliabili ty Ease of use Accep tance Subjective - Relative, retrospective tech. - One-D. (MCH scale) technique is more sensitive than multi-D. technique (NASA-TLX) Multi-D. measure Partly Relative, retrospective technique Relative, retrospectiv e technique Easy Excelle nt Performance Secondary task measure - - Hard - Psychophysio logical Some parameters Poor Non Some parameters Hard Poor Subjective measure is superior to performance-based measure in diagnosticity, sensitivity and other properties, being available in almost all the cases. (2) 6

Comparison of Subjective Approaches Unidimensional VS Multidimensional Absolute VS Relative Immediate VS Retrospective Bedford U A I MCH U A ( R) I or R Psychophysical U R R SWORD U R R NASA-TLX ** M A ( R) I ( R) SWAT M A I Workload Profile M A R Subjective rating instruments ** Relative ratings: Operators are asked either to compare the task condition of interest to a single standard or to multiple task conditions ** Immediate: Rating immediately after performing the task Retrospective: Rating after having experienced all of the task conditions (8) Relative judgments retrospectively after having experienced all task conditions are superior to rating each task condition immediately after performing it because of; (2) - High sensitivity - Concurrent validity with performance - High test-retest reliability 7

Eye Movement Parameters • • Blink – Since visual input is disabled during eye closure, reduced blinks help to maintain continuous visual input when high levels of visual attention are required. – There is a tendency to blink as the eye moves from instrument to instrument. If the task requires gathering information from one source, then blink rate would be expected to decrease. However, if the task requires scanning in order to obtain the information, then increased blink rates would be expected. – Measurable Data • • • Blinks per minute Blink duration Blink interval time Fixation – Where the eye locks on a target while the perceptual systems encodes the image • Fixation duration : 100 ms ~ over 1 second (depending on the difficulty of the task) (1) – The higher the cognitive mental workload is, the longer the proportion of the fixation duration for instrument observation. (6) – Measurable Data • • • Fixations (Mean Frequency) on a single target Fixation dwell time (ms) Number of fixations Fixation fraction (%) Gaze direction – Aggregating consecutive fixations over a single target or region becomes a gaze. – According to the task difficulty, gaze direction is changed. – Measurable data • • Horizontal/Vertical gaze direction variability 8

Eye Movement Parameters • • Pupil – Pupillary dilation increases with workload (e. g. difficulty level of tracking task). Beyond workload, pupil constricts. (4) – Pupil dilation ∝ Rate of presentation, # of transformations – Pupil dilation is more reflection of mental activity in storage & retrieval of information than in mental effort. – Pupillary response measures were sensitive to the information processing demands of the search task. (7) – There is high correlation between pupil diameter and pilot rating. face. Lab. TM Unfortunately, can measure only the coverage of the iris, not pupillary movement. Pupillary response is so sensitive to the environment that this measure is suitable only to the laboratory environment, not to the real situation. Saccades – A series of discrete, jerky movements that jump from one stationary point in the visual field to the next • During saccades, no information can be encoded. • Saccade duration and peak velocity increase with the saccade’s magnitude: – Average saccade duration is approximately 30 ms for a 5 degree saccade and 2 ms longer for each additional degree. – Peak velocity can vary from 200 – 700 degrees/sec for saccades of 5 – 30 degrees. – Methods to distinguish saccades from fixation • Velocity-Threshold Identification (I-VT) • Dispersion-Threshold Identification (I-DT) 9

Characteristics of Eye Movement Parameters Measures Blink ** Fixation Pupil Gaze direction Saccadic eye movements Merits/ Demerits Effective case Ineffective case - Blinks per minute - Blink duration - Blink interval time - Easy to collect and analyze the data - Revealing task complexity - Visually monitoring task - Euros task (arithmetical calculation) - Memory task - Cognitive processing task - Fixation time (dwell time) - Number of fixations - Fixation fraction (%) - Excellent tool to track the operator’s cognitive process (e. g. scanning route or pattern) - Showing the difficulty of information extractions, difficulty level of task - Finding out the critical instrument -Memory task with detection task - Very low level load task - Euros & Memory task w/o detection task - Dilation - Constriction-Dilation difference - Diameter - Sensitive to both the extent of processing capacity as well as the breakdown of capacity / - Susceptible to experimental conditions (e. g. light) and noise - Only suitable for workload measure in laboratory work - Euros task - Memory task - face. LABTM cannot measure - Horizontal gaze direction - Vertical gaze direction - Horizontal gaze variability - Vertical gaze variability - Easy to collect and analyze the data - Showing the important component to specific task / - Ambiguous relationship with mental workload - Euros task - Memory task - Detection task - Saccade magnitude - Saccade duration - Velocity (degrees/sec) - Not to deteriorate significantly with prolonged scanning activity. / - Giving only implicit clues -No significant changes in task complexity - Monitoring task - 10

Characteristics of Eye Movement Parameters Sensitiv ity Diagnosti city Intrusiveness Validity Reliability Ease of use Acceptance Blink ** Good (3) Poor Non Good (5) Excellent (4, 5) Good Non Fixation Medium (3) Poor Non Poor (4) Good Non Pupil Poor (3) Poor Non Medium Good (4) Poor Non Gaze direction Good (1) Poor Non Good (4) Medium (4) Good Non Saccadic eye movements Poor Non Poor Good Poor Non All the parameters of eye movement are not suitable for operator’s monitoring and supervisory control task. Parameters related to the blink are superior to the other parameters. 11

Discussion • • • In order to measure mental workload of operators, performance task measure is not suitable, especially primary task measure, because the main role is monitoring or supervisory control. And also even if the workload changes continuously, operator’s performance may not be deteriorated in monitoring tasks. Secondary task measure have the largest degrading effect on the primary task and is the most intrusive method to measure mental workload. Subjective measure reveals – Static aspect of mental workload – Good diagnosticity (especially, multidimensional measure) Modified NASA-TLX • Physiological measure reveals – Dynamic aspect of mental workload – Poor diagnosticity Blink-related measures • The combination of subjective and physiological measures (e. g. Modified NASA-TLX and blink) is supposed to reflect both static and dynamic aspects of mental workload. 12

Conclusion • Subjective method is superior to performance measurement technique. – Unidimensional method (e. g. 10 -point scale Modified Cooper-Harper (MCH) method) is more sensitive than multidimensional (e. g. NASATLX) in the operator’s mental workload rating, but the unidimensional method could not give us the diagnosis of mental workload which is thought to be the best merit of the subjective measure. – It is desirable that we could compare the results of subjective and physiological measure in the sense of sensitivity using MCH method. – Retrospective judgments are superior to immediately judgments for rating each task condition. Therefore, it is necessary to develop 1)multidimensional, 2)relative, and 3)retropective subjective measure. • Measures related to the eye blink (e. g. blinks per minute, blink duration, blink interval time) are more appropriate than other parameters. 13

Further study • • • Modify NASA-TLX. Study about collecting and analyzing the eye movement data using Fa. T (Face. LABTM Toolbox) is needed. For the experiment, operational conditions sampling should be reviewed according to the Nureg/CR-0711 in three dimensions such as; – Plant conditions • Normal operational events • Failure events • Transients and accidents – Personnel tasks – Situational factors 14

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Dario D. Salvucci, “Mapping eye movements to cognitive processes”, Doctoral thesis, Carnegie Mellon University, Pittsburgh, PA, May 10, 1999 Pamela Tsang and Michael A. Vidulich, “The roles of immediacy and redundancy in relative subjective workload assessment”, Human Factors, 1994, 36(3), 503 -513 Walter W. Wierwille, Mansour Rahimi, and John G. Casali, “Evaluation of 16 measures of mental workload using a simulated flight task emphasizing mediational activity”, Human Factors, 1985, 27(5), 489 -502 Luis Nunes and Miguel Angel Recarte, “Cognitive demands of hands-free-phone conversation while driving”, Transportation Research Part F 5, 2002, 133 -144 J. A. Veltman and A. W. K. Gaillard, “Physiological indices of workload in a simulated flight task”, Biological Psychology, 1996, 42, 323 -342 Yasuko Itoh and Yoshio Hayashi, “The ergonomic evaluation of eye movement and mental workload in aircraft pilots”, Ergonomics, 1990, 33(6), 719 -733 Y. Lin, W. J. Zhang, L. G. Watson, “Using eye movement parameters for evaluating humanmachine interface frameworks under normal control operation and fault detection situations”, Int. J. Human-Computer Studies, 2003, 29, 837 -873 Pamela Tsang and Glenn F. Wilson, ‘Mental Workload’, ‘Handbook of human factors and ergonomics – 2 nd ed. ’, Gavriel Salvendy (Ed), Purdue University, 1997 De Waard. D, “The measurement of drivers’ mental workload”, Ph. D. thesis, University of Groningen, Haren, The Netherlands: University of Groningen, Traffic Research Centre Glenn F. Wilson, “An analysis of mental workload in pilots during flight using multiple psychophysiological measures”, The International Journal of Aviation Psychology, 12(1), 3 -18 15