Australasian Science: Australia's authority on science since 1938

How Reliable Is an Eyewitness?

Credit: auremar/adobe

Credit: auremar/adobe

By John Dunn

Eyewitness identification of criminals is notoriously unreliable, but a new study based on police records has identified factors that can determine which witnesses are accurate and which are guessing.

Eyewitness misidentification is the single greatest cause of wrongful convictions in the United States, having played a role in more than 70% of original convictions later overturned by new DNA evidence (www.innocenceproject.org).

This is consistent with a great deal of psychological research using simulated crimes and lineups. This research shows that our memories can be surprisingly fallible – we forget important details of events, even whole events themselves, and we remember things that have never happened. What’s worse, we can make these mistakes even when we seem very confident that our memories are correct.

Findings like these have cast doubt on the reliability of eyewitnesses. Even when witnesses appear to remember something, or someone, strongly and with high confidence, they can still be wrong.

Our research challenges this widespread view. Rather than relying on laboratory studies, we were interested in the reliability of eyewitnesses in actual police lineups and tested two important conclusions drawn from laboratory research: that confidence is not a good guide to accuracy, and that sequential lineups are better than simultaneous lineups.

In a simultaneous lineup, the eyewitness is given all the photos, usually laid out in a grid, and asked to identify the person they remember committing the crime. In a sequential lineup, each photo is presented one-by-one and the eyewitness is asked to identify whether or not each photo is the culprit.

There is laboratory evidence that accuracy can be greater in a sequential lineup than in a simultaneous lineup. This is surprising, because more information is available to the witness in a simultaneous lineup. While this could aid their decision-making, it may instead make the decision more confusing.

The data for our study was collected by a member of our team, William Wells, in collaboration with the Robbery Division of the Houston Police Department, where 45 police officers had presented photo lineups to more than 700 eye­witnesses over a 12-month period. Each lineup included a photo of a suspect that had been identified by the police as possibly responsible for the crime, as well as photos of five innocent “fillers”. Half the lineups were presented simultaneously and the other half were presented sequentially.

The lineups were conducted fairly – the administering officer was unaware which photo was of the suspect. Eyewitnesses who identified a photo as the culprit were asked to rate their confidence on a three-point scale: low, medium or high confidence.

Our first question was whether confidence was a reliable indicator of accuracy. Unlike laboratory studies, we didn’t know if the suspect in the photo lineup was innocent or guilty so we weren’t able to measure accuracy directly. We therefore approached this problem in two different ways.

First, because the suspect could be the culprit but none of the innocent fillers could be, high accuracy would be reflected in a high level of suspect identification coupled with a low level of filler identification. This is exactly what we found.

At the lowest level of confidence, most identifications were of fillers – as we would expect if witnesses were simply guessing. However, identifications at the medium level of confidence were evenly divided between suspect and fillers. At the highest level of confidence more than 80% of identifications were of suspects.

This shows that while eyewitnesses do make mistakes, and falsely identify innocent people, they usually have little confidence in such decisions. On the other hand, when their confidence is high they make many fewer mistakes.

We also found that the number of identifications of suspects with corroborating evidence increased systematically from low to high confidence, reaching more than 90% in the latter category. This result further supports the idea that when eyewitnesses have high confidence they have a high probability of identifying the culprit.

Although our findings provide strong indirect evidence that eyewitness accuracy is related to confidence, we were unable to measure accuracy directly because we didn’t know the proportion of lineups in which the suspect was guilty of the crime. However, the mathematics of choosing from a lineup can provide useful insights.

If the suspect is not the culprit and the witness identifies a photo, there is a one-in-six chance that the identified individual will be the suspect. On the other hand, if the suspect is the culprit and the witness identifies a photo, the probability that they choose the suspect is a measure of their accuracy. If accuracy is zero (the witness is guessing) then this probability will be one-in-six. On the other hand, if accuracy is 100% then they will always identify the suspect.

However, to measure this probability we have to know the number of lineups conducted at the Robbery Division of the Houston Police Department in which the police suspect was indeed the culprit. To solve this problem we turned to a mathematical model of memory used by other members of our team (John Wixted, Laura Mickes, John Dunn and Steven Clark) to understand the relationship between confidence and accuracy.

According to the model, the different levels of low, medium and high confidence simply reflect whether memory strength exceeds a low, medium or high criterion. This is analogous to the bar of a high jump: strong, medium and weak jumpers can all get over a low bar, but only strong and medium jumpers can get over a medium bar, and only strong jumpers can get over the high bar. Similarly, weak memories that are likely to be inaccurate may be still be strong enough to exceed a low criterion, but only strong memories that are likely to be accurate can exceed the high criterion.

As well as helping us to understand our results, a remarkable feature of the model is that we could use it to estimate the proportion of lineups in which the suspect was also the culprit. However, before we applied it to our data, we first tested the model against a large-scale simulation conducted by a different group of researchers in Australia.

Because it was a simulation, the researchers arranged the test so that the suspect was the culprit in half of the lineups. We used the results from this study to construct eyewitness identification rates for lineups in which the culprit was present or absent. When we applied the model to these different combinations it determined the proportion of culprit-present lineups with a very high level of accuracy.

Armed with this knowledge, we then applied the model to our real-world data and estimated the proportion of culprit-present lineups. Unlike almost all laboratory-based studies, where usually 50% of lineups are culprit-present, our results suggested that only 35% of lineups at the Robbery Division of the Houston Police Department contained the culprit. In many ways this is not surprising, as it is likely that the police would want to know if a person responsible for one crime was also responsible for similar crimes.

More importantly, we replicated our earlier analyses suggesting that accuracy increased with confidence. In particular, we found that identification of the culprit when present in the lineup was close to 100% at the highest level of witness confidence. In other words, when viewing lineups at the Robbery Division of the Houston Police Department, if an eyewitness identified the suspect with high confidence, that person was almost always the culprit.

This result does not completely depend on our 35% base rate estimate. We also estimated accuracy based on different estimates of the proportion of culprit-present lineups (25–75%). Identification accuracy was unaffected and remained close to 100% when witnesses were highly confident.

Finally, we used the mathematical model to compare memory strength between simultaneous and sequential lineups. Unlike many laboratory studies, we found that memory strength was always greater for simultaneous lineups than for sequential lineups.

The results of our study are important for two reasons. First, we were able to investigate eyewitness reliability for crimes conducted in the real-world rather for simulated crimes in laboratory settings. Second, in contrast to the conclusions reached from many laboratory-based simulations, we found that eyewitness identifications are highly accurate when they have high confidence in their identification.

Furthermore, and again in contrast to the conclusions reached from many laboratory-based simulations, we found that a simultaneous lineup led to higher levels of memory strength than a sequential lineup.

How do our results square with the Innocence Project, which found that a high number of false convictions were based on unreliable eyewitness identifications? The answer to this question lies in where and when confidence is measured.

Our results support the view that confidence expressed at the time of identification from a fair lineup is a reliable index of accuracy. However, confidence can increase over time as more information is learned by the witness, reducing its usefulness as an index of accuracy.

This effect has been known for a long time. In a well-known psychological study conducted in the 1950s, people were asked to judge whether two straight lines were continuous or slightly disjointed. Most people were not initially confident in their original decision, but when surrounded by other people who all appeared very confident in one decision or the other (because they were coached by the experimenter to say this), most people increased their stated confidence in that decision even when the evidence of their own eyes stayed exactly the same.

The story for lineups is much the same – only initial confidence counts.


John Dunn is Professor of Psychology at The University of Adelaide, and Chair of the Australian Psychology Accreditation Council.