Object Category Understanding via Eye Fixations on Freehand Sketches

IEEE Trans Image Process. 2017 May;26(5):2508-2518. doi: 10.1109/TIP.2017.2675539. Epub 2017 Mar 1.

Abstract

The study of eye gaze fixations on photographic images is an active research area. In contrast, the image sub-category of freehand sketches has not received as much attention for such studies. In this paper, we analyze the results of a free-viewing gaze fixation study conducted on 3904 freehand sketches distributed across 160 object categories. Our analysis shows that fixation sequences exhibit marked consistency within a sketch, across sketches of a category and even across suitably grouped sets of categories. This multi-level consistency is remarkable given the variability in depiction and extreme image content sparsity that characterizes hand-drawn object sketches. In this paper, we show that the multi-level consistency in the fixation data can be exploited to 1) predict a test sketch's category given only its fixation sequence and 2) build a computational model which predicts part-labels underlying fixations on objects. We hope that our findings motivate the community to deem sketch-like representations worthy of gaze-based studies vis-a-vis photographic images.

MeSH terms

  • Adult
  • Algorithms
  • Female
  • Fixation, Ocular / physiology*
  • Humans
  • Image Processing, Computer-Assisted / methods*
  • Male
  • Neural Networks, Computer*
  • Visual Perception
  • Young Adult