Matthew Lieberman’s response and solution

There was a solution posted in a recently started blog by Matthew Lieberman that focuses on direct/conceptual replications. His solution is indeed a very interesting one; add to the curriculum of graduate students in their first or second year that they replicate findings of studies previously nominated to be so. While comments on it are pessimistic (with justified reasoning), I do hope it resonates within the scholarly psychology community.

A personal take on Lieberman’s response is that I probably would not have minded to see this added to my own curriculum. I may not be all too pleased but considering how much one would learn by replicating something that has worked before, I may not be too peeved about it. Also, getting a name on a publication would be a pretty sweet bonus. Of my severely limited insight into other universities ‘caring and nurturing’ of aspiring scientists, some are better (and some worse) in picking up their students and involving them in the ongoing research. Maybe Lieberman’s idea would go a decent amount of the way to attenuate this issue as well.

Ed Yong’s response and a few comments

Ed Yong’s initial coverage* of Doyen’s** and Bargh’s*** study was, in my opinion, quite brutal. I have been taught through my undergraduate to criticise constructively and I do not think the initial post has the depth to do so. For example, a close look at Doyen’s study indicates that one of the few last alternatives at explaining participant’s slower walking speed was experimenter expectation (and a very well conducted piece of research to demonstrate it). The difference in the walk-fast/expect-fast condition was explained by the difference between manual and automatic measuring, not so in the walk-slow/expect-slow condition. I wrote this in my previous blog entry too, but with a different emphasis. This finding means that an environmental stimuli (experimenter expectation manifested in subtle behaviour) was internalised by the participants and subsequently affected observable behaviour (walking slower). This entails that the Doyen study, in fact, supports the original proposition of the Perception-Behaviour Link. This mitigates my criticism of Bargh’s work, since, the theory from which he based his 1996 experiment was conceptually replicated in the Doyen study. The PBL is not mentioned in Ed Yong’s initial coverage.

In Ed Yong’s reply**** to Bargh, he mentions Doyen to “[have] timed volunteers with infrared censors rather than a stopwatch” But they timed both with sensors and manually. This was one of the central reasons that they came to the conclusion that experimenter expectation was the only alternative left to explain their result.

It does strike me from having reviewed large parts of the literature surrounding priming that the published articles are all conceptual replications. The studies following Bargh et al. (1996) have differences in methodology to that study. The issue that has been raised in comments to Bargh’s reply to Ed Yong is that “purer” replications that have not given the same results are subject to the file-drawer phenomenon. I.e. publishers have not accepted them and so they’ve been put in the file-drawer. The issue with this statement is obviously that it is very hard to know (for an outsider like myself) if publishers have denied these studies because they show null-results (not very exciting and from comments it seems there are other rather valid reasons for them not to publish these) or if they contain errors of various types (making them unpublishable).

In either case, I believe I argue in my literature review, strongly, for the theory underlying priming (the Perception-Behaviour Link) but at the same time believe that researchers are getting ahead of themselves and testing advanced hypotheses, when really what this theory needs is the grunt-work of establishing even its simplest tenets. Be that an actual replication of the methodology in Bargh et al. (1996), even though I believe there exist other research more suitable to exemplify the Perception-Behaviour Link.

I should have chosen another topic to do my 30-page literature review on.

*http://blogs.discovermagazine.com/notrocketscience/2012/01/18/primed-by-expectations-%e2%80%93-why-a-classic-psychology-experiment-isn%e2%80%99t-what-it-seemed/
**http://dx.doi.org/10.1371/journal.pone.0029081
***http://www.yale.edu/acmelab/articles/bargh_chen_burrows_1996.pdf
****http://blogs.discovermagazine.com/notrocketscience/2012/03/10/failed-replication-bargh-psychology-study-doyen/

Bargh, Doyen and conclusions thereof.

I have just witnessed an interesting phenomenon.

I am concurrently to writing this blog-post, writing a literature review on priming and the Perception-Behaviour Link*. Another paper whom closely replicated the findings of Bargh et al. (1996)*, Doyen et al. (2012)**, were unable to replicate findings as well as provide an interesting demonstration of experimenter expectation. It is however presumptious to assume that the original literature* then also is explained by this bias. The reason is given in a reply by Bargh*** to the Doyen study, basically stating that the experiment was run as a blind study and so experimenter expectation can be safely ruled out. While Bargh’s reply unfortunately contains personal attacks and evidence towards being technologically unwilling, he is defended by others in that his experiment _has_ been replicated successfully and that one study cannot refute several made on the same topic. The studies given (by others than Bargh -he did not give any studies as support in his criticism) as support for this claim are these; Elderly prime effect on simulated driving speed, Gay male prime effect on hostility, elderly or youth prime effect on walking time and accessibility as precursor to goal-fulfilment and high vs. low self-conscious difference in being primed by an elderly stereotype on walking speed (xps 2 & 3). I have reviewed one of these in my literature review and it is at best a conceptual replication with several issues in methodology and statistical interpretation (specifics available on request).

In my book, it is not enough to conceptually replicate, since one is then stuck with reviewing another piece of research with its own flaws and fallacies. Granted, Doyen comes close to replicating but did differ on the point of blind experimenters (it was one of the manipulations in Doyen). The assumption was that Bargh’s work was non-blind (something I came to the conclusion of as well, although I’ve read the paper a gazillion times). This was not the case, and hence, it is also a conceptual replication. What _is_ interesting with the Doyen study is that it still supports the PBL, albeit unintentionally. The most important statistic presented is the believe-slow-walking speed comparison between automatic and manual measurement. Believe-fast-walking’s significant difference was removed when considering both automatic and manual, not so for believe-slow. A last alternative for this significant difference is then experimenter expectation. The thing is though, experimenter expectation is also an environmental stimuli that is unconsciously internalised and evidently had an effect on observable behaviour! Well, this is what the Preception-Behaviour Link strictly posits.

Again, it is a bit of a shame that many of the arguments Bargh uses in his criticism of the Doyen study are arbitrary, unsupported and, on occasion, false in light of other research (even some from the area of priming)****. It does however not reflect on his prior research. In conclusion, Doyen does not specifically cut Bargh’s research down, but rather, introduces another concept able to be accounted for within the Perception-Behaviour Link’s framework.

On a second note, if you have unpublished research on the replication of Bargh et al.’s 1996 study, I would very much like to read it. I believe it is of central importance to be open-minded to one’s own fallacies and others’ criticism (even if I very much like the Perception-Behaviour Link theory), the only way is forward and it is only obstructed when self-preserving opinions and values are set before empirical research.

*Bargh, J.A., Chen, M., & Burrows, L. (1996). Automaticity of social behaviour: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology, 72, 230-244.
**Doyen, S., Klein, O., Pichon, C., & Cleeremans, A. (2012). Behavioural priming: It’s all in the mind, but whose mind? doi:10.1371/journal.pone.0029081
***Bargh, J.A. (2012, March 5). The natural unconscious: Nothing in their heads. [Web log post]. Retrieved from http://www.psychologytoday.com/blog/the-natural-unconscious/201203/nothing-in-their-heads.´
****See comment section of Bargh’s blog post, specifically the one referring to Assimilation and Contrasting (which is found in Dijksterhuis, A., Spears, R., Postmes, T., Stapel, D.A., van Knippenberg, A., & Scheepers, D. (1998). Seeing one thing and doing another: Contrast effects in automatic behaviour. Journal of Personality and Social Psychology, 75, 862-871.)