Australasian Science: Australia's authority on science since 1938

Watching the Detective

Credit: dynamosquito (CC BY-SA 2.0)

By Tim Hannan

Studies of neural activity in viewers of Sherlock reveal how we connect story elements.

Over the past century, research in cognitive neuroscience has enabled the development of a consistent and testable theory of how we perceive and remember individual pictures and words. Yet one question has continued to trouble researchers: how does the brain enable us to understand and recall a series of discrete words and pictures as a continuous story? A team of American researchers believes it may have solved this problem, with the assistance of Sherlock Holmes.

The paradigm employed in most research into human perception and memory has required participants to perceive and memorise lists of words or photos. Yet human experience involves much more than the perception and recall of discrete elements: we experience, perceive and remember our lives as a continuous, meaningfully connected stream of events. Somehow the brain is able to process these elements into a coherent and memorable whole.

The process we use to achieve this task is loosely termed “chunking”. Although this is a long-held and generally accepted theory, the neural mechanisms for it have remained little known. Recently, researchers have begun to employ functional imaging techniques such as fMRI to observe how the brain reacts during perception and recall of experiences, especially those requiring the processing of related events.

In a series of studies, a team of researchers from Princeton has used episodes of the BBC series Sherlock to observe how the brains of viewers react to connected scenes. While undergoing an fMRI scan, one group watched the episodes as usual while the other listened to an audio description of the storyline. Some of the participants had viewed the show previously while others were seeing it for the first time.

The first interesting finding was that the ongoing neural activity in the brains of participants was interrupted by irregular “jumps”. These jumps occurred intermittently during viewing of the episodes: some after a few seconds, and others after several minutes.

The researchers understood these different time scales to reflect participants’ attempts to connect various story elements together. This occurred quickly on occasions, with more complex associations taking longer periods.

Notably, the same pattern was observed in both groups of participants, whether they were viewing the episode or listening to the audio description. Thus it seemed to relate not to variations in the perceptual input but to higher-level processing of the story itself. Further analyses demonstrated that the jumps corresponded to the points in the episode that participants later identified as changes in scenes.

A further finding was that the patterns of activity differed between those who had seen the show previously and those who had not: the participants who were familiar with the storyline had slightly earlier “jumps” than those who had not, suggesting that they were able to anticipate the connections between story elements.

Finally, the researchers also asked the participants to describe the storyline from memory, while continuing to have their brain activity monitored. The same sequence of jumps were observed during this recall phase as had been observed in the initial presentation of the show – and in the same order.

The fMRI indicated activation of the hippocampus at the end of each of the jumps, and that the strength of the activation was associated with the quality of recall of details of the storyline. This finding is consistent with the previously established view that the hippocampus is essential for learning new information, and supports the claim that it does so by assisting the chunking of information into memorable elements.

The studies elegantly demonstrated that it is possible to identify the neural processes underlying the chunking of information to facilitate understanding and remembering.

So why did the American neuroscientists use the BBC’s Sherlock? Well, for the researchers it’s elementary: the methodology called for material that was sufficiently complex to activate the brains of participants, and this ruled out many of the television shows produced on their side of the Atlantic.

A/Prof Tim Hannan is Head of the School of Psychology at Charles Sturt University, and the Past President of the Australian Psychological Society.