Recent claims cast doubt on legitimacy of behavioral science

In a recent series of blog posts, they alleged four studies involving Harvard Business School Professor Francesca Gino used falsified data to produce positive findings. In an ironic twist, some of the experiments were on dishonesty and ethics.

The accusations have led some to question whether the field of behavioral science can be trusted. Given governments and regulators across the globe are using science as the basis of policy- and rule-setting, such accusations have serious implications for healthcare, financial markets, law enforcement, and the judiciary.

It is worth taking a closer look at Data Colada’s claims and weaknesses in behavioral science generally. We need to ask whether we should be concerned about the application of psychology to risk and compliance.

The claims

In August 2021, the Data Colada team said they believed data in an influential behavioral science paper was falsified. The landmark 2012 study was undertaken by Lisa Shu, Nina Mazar, Gino, Dan Ariely, and Max Bazerman.

There were three studies reported in the paper. The third study used data from a car insurance company to conclude signing an honesty declaration at the front of a claim form makes individuals less likely to lie about the mileage they drove. However, the base mileage data used to judge dishonesty was suspicious. One would expect the mileage driven by a large number of people to be extremely varied. A small number of people won’t drive much, a small number will drive vast distances, and most will drive an average distance.

What the Data Colada team discovered was the base data showed a uniform pattern of mileage with a sharp cut off at 50,000 miles. In the 2012 study, the distances driven by individuals didn’t vary at all. An equal number of people seemed to drive across a range of distances—but never beyond 50,000 miles.

There were other indicators of falsification identified by the reviewers, such as suspiciously precise reported mileages (e.g., 1,743 miles rather than 1,750) and changing fonts indicating data had been copied poorly.

Data Colada researchers also identified similar red flags in data used in other papers co-authored by Gino, who is on leave from Harvard University. All articles mentioned were either retracted or have retraction requests associated with them.

Psychology

Psychology as a discipline has faced methodological challenges throughout its history.

Cyril Burt, for instance, was an eminent psychologist whose experiments were used to promote the theory intelligence was inheritable. After his death, it was claimed he not only falsified data from identical twins who never existed, but he also invented two co-researchers he claimed had worked with him. His data and methods were declared untrustworthy, but Burt’s work was used as justification for the U.K.’s “11-plus” examination—a selection test used to channel children into academic or technical schooling around the age of 11.

Psychological science has also been criticized for other failings, which can be grouped under the following broad headings.

• Cultural bias: Psychology is primarily based on experiments involving white, European subjects with little regard to other cultural contexts.
• The “replication crisis”: The findings of some studies have been impossible to repeat.
• Publication bias: Practitioners are under pressure to publish often and only those with positive results reach the light of day.
• I-frame, not s-frame: It has been suggested “nudges” often only produce small effects and that the focus of science on changing individual behavior is less effective than policies aimed at changing societal rules.
• Oversimplification: Concepts such as “System 1” (instinctive, subconscious thinking) vs. “System 2” (cognitive, conscious reasoning) are an overly simplified characterization of the full complexity of human cognition.

There can be no doubt psychology has had its instances of fraud, misrepresentation, and overgeneralization. Yet, the same is true of other, more “concrete” sciences. When genetic scientists at University College London were found to have fabricated results over many years, no one demanded a rejection of the entire field of genetics.

Any science is an inherently social process, subject to the same human biases, flaws, and predispositions as any activity. This is a reason to be cautious and thorough when applying scientific findings, but it is not a reason to reject them outright.

Careful consideration, and much experimentation, is required when deploying interventions. There are many successful examples of the application of behavioral science to significant human problems. That science might sometimes be problematic should not lead us to reject the positive solutions it can offer.

This article contains excerpts from the full story by author Paul Eccleson for the International Compliance Association. The ICA is a sister company to Compliance Week. Both organizations are under the umbrella of Wilmington plc.