Heterogeneous hidden Markov models for context modeling through eye gaze observations

Document Type

Conference Proceeding

Publication Date


Publication Title

AAAI Spring Symposium - Technical Report


SS-17-01 - SS-17-08

First Page


Last Page



Eye gaze patterns are known to have correlation with cognitive context, such as cognitive understanding, difficulty, fatigue and inattention. Traditional eye-gaze metrics that are developed for such analyses, such as fixation duration, saccade length, the nearest neighbor index (NNI), fail to accommodate the dynamic nature of mental states. In a recent work, a hidden Markov model-based observation model was suggested, where the gaze-patterns on the Cartesian plane, which correspond to each cognitive state, are modeled as Gaussians. However, we recognized that a single observation model is not sufficient to represent diverse gaze patterns that correspond to different cognitive states of the brain. In this paper, we assume a heterogeneous hidden Markov model to represent such observations and demonstrate a modified Baum-Welch approach to train such a model. The effectiveness of our approach is demonstrated using eyetracking data collected from volunteers engaged in a simulated task that required varying levels of cognitive inputs.



This document is currently not available here.