Australasian Science: Australia's authority on science since 1938

Your Face Is Your Fortune

Doreen Salcher/Adobe

Doreen Salcher/Adobe

By John L. Bradshaw

Beauty is in the eye of the beholder, but there is an additional treasure-trove of information still to be mined from new ways of looking at people's varying faces.

Her face may have launched a thousand ships in the punitive raid to retake the errant Helen, and a picture of a particular face is certainly worth a thousand or more words – if you have ever tried to describe one unambiguously. Indeed few other pictures, objects or pictures of objects can match the nuanced complexity of a particular person’s portrait or actual living countenance. Apart from what might be the case with occasional pairs of identical twins, one would have to search through many thousands of mug shots to encounter an even remotely similar match, and even then the search would likely prove fruitless.

It is no accident that the latest generation of security software seeks to address the problem of user identity via face recognition. Your face is practically unique, and usually is conveniently exposed to full view.

Faces are in this respect the equivalent of a motor vehicle’s registration plate, but they are also much more than that. They convey a person’s mood changes from moment to moment and, along with postural alterations and body language generally, are an important adjunct to the spoken word. The blind are thereby doubly penalised in daily intercourse.

Humans have long been considered unlike all other species in our ability to make and deploy tools, use language to communicate and have a sense of personal identity, but these bastions of uniqueness have long since fallen. Crows are incredibly ingenious, and can manufacture hooks from bits of wire to snare otherwise inaccessible items of food; various mammalian species communicate by calls between themselves that are akin to a form of language; and the great apes certainly have a concept of the self, recognising themselves and their own faces in a mirror.

What about the ability of non-human species to recognise other individuals or even humans from their facial characteristics? Dogs and sheep can definitely recognise particular people, whatever they are wearing, and the Australian magpie is known to recognise, and hold grudges against, people who have at some time in the past treated them badly, again irrespective of changes in the person’s clothing.

Recently there have been claims that even the humble bumble bee, with its minute brain, can learn to distinguish human faces from similar non-facial objects or patterns if their training had involved exposure to facial stimuli that resembled some form of floral configuration. As yet, however, there is no evidence that they can recognise individual people, let alone if they can visually identify certain other bees.

Neurologists term the loss, often after head injury, of the ability to recognise objects as an agnosia. There are many subtypes of agnosia depending upon what is lost, preserved or altered. The loss of the ability to recognise faces, in the flesh or as represented in photographs, pictures or sculptures, is termed prosopagnosia.

Even this deficit may be further subdivided. The individual may no longer be able to tell if a pattern represents a face or not; or if two faces not previously encountered, and presented side-by-side or successively, belong to the same person; or they may be unable to recognise old, familiar, famous or previously encountered faces. They may be unable to identify them while admitting that they are somehow familiar, or they may be able to comment appropriately about certain aspects of the unidentified person (e.g. she was a prominent politician), or they may claim they have never previously encountered them. It is likely that subtly different but closely adjacent processing regions are affected.

In our own species there is a certain brain area in the right cerebral hemisphere called the fusiform gyrus that is strongly active when we are processing or responding to facial stimuli. Damage here usually results in impaired capacities for facial recognition (prosopagnosia). The fusiform gyrus is adjacent to regions known to be involved in processing complex visual patterns and object recognition.

The fusiform gyrus is also close to the lingual gyrus, which seems to mediate aspects of letter and word recognition, particularly on the left side. Early developmental damage to a child’s left hemisphere in these regions can lead the corresponding structures in the right hemisphere to take over the linguistic functions of the left, and this can be detrimental to the child’s face-processing abilities. It’s as if language functions are favoured developmentally over certain aspects of pattern recognition.

Does this mean that face processing is just a special case of pattern recognition generally, and the fusiform gyrus is where general aspects of pattern recognition are elaborated? It is certainly true that some prosopagnosics may simultaneously present with problems recognising other types of generic objects. Thus there are reports of a farmer who could no longer recognise individual cows in his herd, and motoring enthusiasts who lost their ability to identify particular marques of motor car.

However, it may be the case that damage was not confined to the face-processing regions, which lay in or adjacent to general pattern recognisers in the brain. Indeed, a related phenomenon is known as category-specific deficit. Here, an affected individual may have problems naming, identifying or describing members of a particular category, such as vegetables (e.g. carrots, mushrooms, sprouts), tools (e.g. saw, screwdriver, wrench) or items of furniture (e.g. table, chair, sofa) while continuing quite happily with other similar categories, such as items of clothing. It is unclear whether what is lost is the actual engram (or memory “store”) containing the particular category members, or whether what is affected is the relevant access routes into the engram.

Until recently, prosopagnosia has been seen as an acquired disorder resulting from brain trauma, but I had long speculated whether, just as dyslexia may be acquired from injury or may be developmental, there might be whole kindreds with an inherited, developmental form of the disorder. This has indeed proven to be the case.

It has long been held that most things that we encounter may be processed, differentiated or identified via two quite different but complementary routes or processes:

  1. detailed, analytic, bottom-up breakdown into fundamental, basic or component features; versus
  2. holistic or synthetic top-down matching of the unitary whole, or large parts thereof, to a stored engram, where it is more a matter now of how the whole fits together than a laborious analysis of component features.

The never-ending debate as to whether reading should be taught on the basis of letter-by-letter analysis or by learning to recognise the word, or most of it, as a whole neatly captures this debate. Of course, both processes should operate simultaneously, “talking to each other”. Unfamiliar and difficult words would initially rely on the analytical process, and when familiarity or facility is achieved there would be a shift towards the synthetic direct-recognition process. Exactly the same issues arise when designing machine-learning systems.

There is strong evidence that the analytical aspects are represented more in the left cerebral hemisphere while the more synthetic, holistic aspects are in the right. Neurological studies of the effects of damage to lateralised brain regions have revealed that reading scripts employing phoneme coding (e.g. the Japanese syllabary hiragana) employs an analytical symbol-by-symbol coding mode while reading logographic kanji (where the symbols more directly represent the concept as a unitary whole, as in our road signs) involves direct-recognition.

The question of processing mode bears in an interesting way upon the question of conscious awareness, as we can often recognise a briefly exposed face yet subsequently find it quite difficult, when it is no longer before us, to say exactly what it was about the face that made it uniquely recognisable. Likewise, with the common and irritating “tip-of-the-tongue” phenomenon, we may be quite unable to recall a name with which we were previously familiar while retaining the ability to recall its initial letter or number of syllables. Usually, if one makes no further effort to recall, the whole will unexpectedly “pop up” into memory, usually when we are doing something quite unrelated.

To return to faces, a ready demonstration of the effects of these two modes of processing comes from presenting portraits in their normal, upright fashion or inverted instead. In the latter instance, all sense of familiarity is immediately lost. Try it yourself! You will initially be forced to rely on an analytical, feature-by-feature process. Eventually, however, if you persist and practise identifying inverted faces you may recruit additional processing capacity, often from corresponding regions in the other hemisphere, or by a process of brain plasticity that enlarges existing processing structures. We have all heard of the professional London cab-drivers who, after years of navigating the labyrinthine city’s maze of streets, possess enlarged brain regions dedicated to spatial navigation.

Much has been written about the breakdown of function in prosopagnosia, but what about the opposite situation: unusually high functioning? Recently, attention has turned to an unusually enhanced capacity to recognise faces in a very small percentage of otherwise normal individuals. Such people are able to recognise a face they’ve perhaps only seen once, very fleetingly, years ago. These individuals are now even being employed in a forensic capacity in several of the world’s major jurisdictions.

Faces are commonly assessed in a largely automatic fashion to determine the other person’s age, sex, likely country of origin and even attractiveness. An unexpected and frequently confirmed finding came from digitally averaging the images of several hundred or more randomly chosen males and, separately, females. The resultant images were seen as faces of a rather extraordinary beauty or attractiveness. So, a good-looking person would seem to be someone who possesses the group average for all relevant features and dimensions. Beauty is indeed in the eye of the beholder; more correctly, beauty is a function of a long prior exposure to very many faces from your own culture, varying across the entire range of features.

Beauty may also be perceived in faces that possess a high degree of left–right symmetry. It may be that both an overall symmetry, and an approximation to population mean in the general facial configuration, separately signal to the observer that such a person possesses a high degree of genetic fitness, and by having overcome many possible environmental challenges, would therefore be a good candidate for generating progeny.

We can further enhance left–right symmetry in a face by cutting a photo down the midline and constructing a composite from the two left or the two right sides, flipping one of them over to join the two halves. Viewers tend to see the facial composite constructed from the two left halves as more closely resembling the original than the composite made from the two right halves. This effect is partly in the eye of the beholder, as our asymmetrical neural organisation pays slightly more attention to the left field, which in this case is the left side of the original, unmodified face. Indeed, unilateral brain damage is far more likely to result in unilateral left neglect – not noticing events on the left side of visual space – than neglect of things on the right.

A slight leftwards attentional bias in normal healthy individuals attending to the spatial properties of a display can be easily demonstrated with the grey-scales task. Take a fairly short strip that grades via progressive shades of grey across its horizontal extent from white at one end to black at the other. Beneath (or above) it, place an exact copy that is now reversed, so that the gradation now runs in the opposite direction. People asked to judge which strip, top or bottom, looks overall darker or whiter tend to choose the strip where the darker, or lighter, end respectively is on the left.

If a silent video of someone speaking masks lip movements, masking of the left side (as viewed) has a more of a detrimental effect on attempts at comprehension than masking the right side. Indeed, when watching a TV newsreader one can often see that the asymmetry of the lip movements is somewhat greater on the left side (as viewed).

Note that the sensory and motor pathways mostly cross in the mammalian brain, possibly because during our evolutionary trajectory from a worm-like forbear, the head end rotated 180° with respect to the rest of the body. If we add this neural cross over to the left–right reversal occurring between the speaker’s and the observer’s points of view, the effects cancel each other out.

This all goes to show that there’s more to the face than meets the eye.

John L. Bradshaw is Emeritus Professor (Neuropsychology) in the Monash Institute of Cognitive and Clinical Neurosciences.