Australasian Science: Australia's authority on science since 1938

Are You Looking at Me?

eye spy

Observers have a tendency to believe that someone's gaze is directed towards themselves.

By Colin Clifford & Isabelle Mareschal

Is that person wearing the sunglasses looking at you? Or are we programmed to anticipate that we are being watched even when we’re not?

We rely on our vision to provide us with moment-by-moment information on the world around us. But the apparent effortlessness with which we enjoy a rich, seamless visual experience belies the amount of work going on in our brains. Roughly one-third of the human brain is devoted to vision, but what is all that grey matter doing?

Our brains are not merely passive receivers of sensory input from our eyes. Indeed, the visual information that reaches our brains from our eyes can be quite poor, yet we perceive a coherent and very detailed view of the world. How is this possible?

One idea is that the incoming stimulation to our brain is actively interpreted on the basis of specific expectations about the structure of the environment. For example, our vision is biased to assume that illumination comes from above, that objects are convex (solid) and not concave (hollow), and that contours tend to be oriented horizontally and vertically rather than diagonally. Thus, our brains somehow represent implicit expectations about the structure of the world around us.

Our research is looking at the rules of thumb our visual systems employs in social situations, such as looking at someone’s face. We have recently published several studies investigating how we perceive the gaze of another person – that is, the direction in which someone else is looking.

We are particularly interested in gaze because it is said that the eyes are the windows to the soul. The direction of a person’s gaze shows where they are looking and what they are interested in, providing useful clues about their state of mind. For example, when people are thinking hard they often look up and slightly to one side. Direct gaze can indicate approach emotions such as anger and joy, whereas averted gaze can indicate avoidance emotions such as sadness and fear.

Figure 1. Neither the boy’s head nor his body are oriented towards the camera and his eyes are barely visible, but we still have a strong sense that he is looking at us.

To explore the expectations implicit in interpreting another’s gaze, we carried out a simple behavioural experiment. The logic of the experiment was that biases in our visual processing should be more evident when sensory input is uncertain. In dark conditions or when the other person is wearing sunglasses, for example, any prior expectation of the direction of gaze is likely to be given more weight than when the eye region is clearly visible (Fig. 1).

Subjects in our experiment viewed computer-generated faces on a monitor. Their task was to discriminate the direction of gaze of two faces presented one after the other, and then record which one was looking more towards the right.

The two faces differed in the quality of the images of the eyes. In one face the eyes were clearly visible but in the other they were degraded by reducing the contrast and adding random texture. We reasoned that any bias to perceive gaze in a particular direction should be more evident in the degraded condition where the sensory input was more uncertain.

Figure 2. (a) Computer-generated female face looking slightly leftwards. (b) The same face with noise added to the eyes to degrade the image. (c) Most observers experience the following: adding noise to the eyes (here shown looking leftwards, but this is true also for right-gazing eyes) makes them appear to be looking more direct.

Figure 2 is an example of the type of faces we showed observers. In Figure 2a, the eyes are clearly visible and most observers judge the gaze to be slightly to their left. However, when we degrade the eye information (Fig. 2b), most observers judge the gaze to be more directed towards them even though the direction of gaze in the two images is the same. Figure 2c is a schematic illustration of the how a prior expectation would work: adding noise (or glasses) to the eyes makes them look like they are directed more towards us.

Our results revealed that observers have a tendency to believe that gaze in the degraded condition is directed towards themselves. This tendency manifested not as a simple default response but as a graded interaction between the direction of the eye gaze and a prior expectation that the gaze was directed at them. So, when gaze deviations were extreme, observers correctly identified that the noisy gaze was directed to their left or right.

Observers were also not simply using a different cue to perform the task when noise was added to the eyes. For example, they did not just report the direction of the head instead of the eyes.

Why might we have a bias to assume that gaze is directed towards us? One possibility is that it is better to be safe than sorry. If we are the object of someone (or something) else’s gaze then some action on our part is likely required.

If someone is looking at us then we should establish whether they are happy or angry, friend or foe. If our caveman ancestor ignored a pair of eyes staring at him from out of the undergrowth then he might find himself dinner for a sabre-tooth tiger – in which case he might never have got to be our ancestor at all!

The cost of not realising we are being looked at could be far greater than the cost of a “false alarm”. If so, it is a sensible strategy to assume that, when in doubt, we are being looked at.

This evolutionary perspective on why we may have a prior expectation for direct gaze may reflect a more generalised strategy that applies to most animals living today. Although it remains an open question as to whether other animals expect that gaze is directed at them, the finding that many animals respond to direct gaze differently than to averted gaze could provide a basis for such a scenario.

Interestingly, even young infants show a preference for gaze directed at them. If the tendency to think that gaze is directed towards us is indeed something inherited from our ancestors then we might expect it to be evident even in young babies. If not then it might be a tendency that develops through experience with social interactions. These are alternative hypotheses that could be tested in future experiments.

The experiment we carried out is an example of basic, curiosity-driven research: knowledge for its own sake. Are there practical applications of this knowledge?

One direction is towards improved interactions between people and computers. I have often thought how much easier life might be if my computer only knew what I wanted it to do! That day might still be far off, but if it is ever to come then we will need to equip computers with simple rules of thumb to interpret the social signals present in our facial expressions and gestures.

Our results could also play a pivotal role in the development of tools for people suffering from neurodegenerative diseases or strokes that prevent them from communicating directly, such as “locked-in” syndrome. Currently most means of communication rely on tracking the person’s eyes and translating their blinks into a message. A computer that could not only track blinks but also decipher subtle emotional cues from the eyes could be of great benefit.

Another potential area of application is in the understanding of various clinical conditions where the perception of gaze appears to be disordered. People suffering from social anxiety, for example, have a strong tendency to feel that they are being watched. The results of our experiment suggest that this is an exaggerated form of a tendency that we all share. If this tendency does develop through experience then it might be possible to improve their interpretation of social signals through practice with facial stimuli such as those used in our research.

We also found that not just the direction of the eyes but also the orientation of the head provides a cue to the direction of gaze. Thus, making as simple a judgment as where someone else is looking involves the combination of multiple sources of sensory information– eye direction and head orientation – with prior expectations.

This makes the task of understanding the visual interpretation of social situations all the more daunting. It does, however, suggest that gaze perception is a good place to start – a model system in which to establish more general principles.

Colin Clifford is a Professor and Australian Future Fellow at The University of Sydney’s School of Psychology, and a Chief Investigator of the Australian Centre of Excellence in Vision Science. Isabelle Mareschal is a Lecturer in Biological and Experimental Psychology at Queen Mary, University of London.