We have upgraded our news system. To view the old one please follow this link.
|The Psychologist News - Rows over replication|
Rows over replication
A row has erupted online after an eminent social psychologist in the USA reacted angrily to a failed replication of one of his classic stereotype priming studies. John Bargh, Professor of Psychology and Cognitive Science at Yale University, used his blog on Psychology Today to launch a stinging criticism of the researchers who failed to replicate his 1996 study, the journal they published in, and the British science blogger who reported on their new research.
In a post that extends to several pages, Bargh implied that Stéphane Doyen (Université Libre de Bruxelles) and her colleagues are 'incompetent and ill-informed'; he claimed that the open-access journal PLoS One allows researchers to 'self-publish' their studies without appropriate peer review so long as they are willing to pay the $1350 fee; and he described Ed Yong's Discover magazine blog coverage of the failed replication as 'superficial online journalism'.
The new paper by Doyen et al. 'Behavioural priming: It's all in the mind, but whose mind?' (PLoS One) attempted to replicate Bargh's highly cited 1996 article, co-authored with Mark Chen and Lara Burrows, which showed that participants primed non-consciously by the elderly stereotype walked away from a psychology lab more slowly (Journal of Personality and Social Psychology.
Doyen's team made some changes to Bargh's methodology, including doubling the number of participants and using infra-red beams to time participants' walking speed (as opposed to a research assistant with a stop-watch). Their attempt at replication failed - participants exposed to ageing-related words in a scrambled sentence task didn't walk away any more slowly than control participants.
However, when the study was repeated with the experimenters knowing the expected results of the study and which condition participants had been allocated to, the slowing effect was observed. In another twist, experimenters told to expect participants to walk away faster actually obtained data supporting this reverse-effect, but only if they used a stop-watch. A final important detail is that there was evidence that some participants in the prime condition had noticed the ageing-related words they'd been exposed to, thus casting doubt on the scrambled sentence task as a way to deliver primes non-consciously.
Based on their results, Doyen's team concluded that 'experimenters' expectations seem to provide a favourable context to the behavioural expression of a prime.' They argued further that it was important to consider the limitations of automatic behavioural priming: '...it seems that these methods need to be taken as an object of research per se before using it can be considered as an established phenomenon.'
In his blog post, Bargh argued there was no way that experimenter expectancies could have interfered with the results he and his colleagues obtained. He blamed the replication failure on 'gross' methodological changes made by Doyen's team. For example, he quoted them as having instructed participants to 'go straight down the hall when leaving', in contrast to his study, which he said let participants 'leave in the most natural way'. In fact, as Yong has pointed out in a response on his blog, Doyen's team wrote that 'participants were clearly directed to the end of the corridor'; similarly, Bargh and his colleagues wrote in their study that the experimenter told the participant that 'the elevator was down the hall'.
Bargh concluded his blog post by arguing for the robustness of the concept of stereotype priming, which he said has been replicated 'dozens if not hundreds' of times and is solidly embedded in several theories across multiple scientific fields. 'I am not so much worried about the impact on science of essentially self-published failures to replicate,' he wrote, 'as much as I'm worried about your ability to trust supposedly reputable online media sources for accurate information on psychological science.'
As we went to press the controversy was playing out online with several psychologists contributing their views: Matt Craddock commented on Bargh's Psychology Today post; Matthew Lieberman has written a piece on his blog Social Brain, Social Mind; and Daniel Simons posted his views on Google+ as 'A primer for how not to respond when someone fails to replicate your work'.
In a related incident, the failed replication attempt of Daryl Bem's 'precognition' study, by Chris French (Goldsmiths, University of London), Stuart Ritchie (University of Edinburgh) and Richard Wiseman (University of Hertfordshire), has finally been published, also in PLoS One, with Bem responding in the comments. Their report was rejected by several journals including the Journal of Personality and Social Psychology (see 'News', June 2011), which originally published Bem's findings along with his appeal for attempted replications. Writing in The Guardian, Chris French said: 'Although we are always being told that "replication is the cornerstone of science", the truth is that the "top" journals are simply not interested in straight replications - especially failed replications. They only want to report findings that are new and positive.' cj
FuseTalk Standard Edition - © 1999-2014 FuseTalk Inc. All rights reserved.
The Psychologist Home | Accessibility | Text Only | Site Map | Contact Us | BPS Website
© Copyright 2000-2014 The British Psychological Society
The British Psychological Society is a charity registered in England and Wales, Registration Number : 229642 and a charity registered in Scotland, Registration Number : SC039452 - VAT Registration Number : 240 3937 76