Dishonesty study was based on fraudulent data

IF YOU ARE WRITING a book called “The Honest Truth About Dishonesty,” the last thing you want to be associated with is fraud. Yet that’s where Dan Ariely, behavioral economist at Duke University, is with his four co-authors of an influential study on lying.

In 2012, Mr. Ariely, along with Max Bazerman, Francesca Gino, Nina Mazar and Lisa Shu, published a study on how to get people to be more honest. They concluded that when asked to say the information is true before giving it out, rather than afterwards, people are more likely to be honest. The results are from three experiments: two conducted in the laboratory (led by Mr. Bazerman, Ms. Gino and Ms. Shu), and a third based on data from an auto insurance company (led by Mr. Ariely and Ms. Mazar ).

Several researchers have tried and failed to replicate the results of laboratory tests. But it is the study on auto insurance that raises the most serious doubts. He asked the insured to declare themselves the number of kilometers traveled. Customers were asked to sign a statement on the report form that read, “I promise the information I am providing is true”; half of the forms had this statement at the top, half at the bottom. All car owners had previously reported their odometer readings to the insurance company, giving a baseline for the data (the time between baseline readings and experience varied for each customer). Mr. Ariely and Ms. Mazar found that when customers were asked to sign the declaration at the top of the form, there was a 10.25% increase in the number of self-reported miles, compared to the miles reported on the forms where the statement was signed at the bottom. The more kilometers a car has traveled, the more expensive the insurance will be. The researchers concluded that signing the statement of truth at the top of the form made people more honest (and therefore on the verge of paying higher insurance premiums).

With over 400 citations on Google Scholar, these findings have spread widely. But on August 17, Leif Nelson, Joe Simmons and Uri Simonsohn, who run a blog called Data Colada, published a item, based on the work of a group of anonymous researchers, dissecting what they believe to be evidence of fraud. There are several frowning concerns, although two in particular stand out: the number of kilometers reported by policyholders and how the numbers would have been recorded.

In a random sample of cars, one would expect the number of kilometers traveled by each vehicle to follow a bell-shaped curve (like a “normal distribution”). Some cars run a lot, some barely, but most fall somewhere in between these extremes. But in the 2012 experience, the number of kilometers driven follows a uniform distribution: as many cars have driven less than 10,000 miles as driven between 40,000 and 50,000 miles, and not a single car has driven more than 50 000 miles. MM. Nelson, Simmons, and Simonsohn suggest that a random number generator was used to add between zero and 50,000 to the original readings submitted by customers.

The theory of the random number generator is supported by the second problem with the data. Many people, when asked to write large numbers, round to the nearest ten, hundred or thousand. This can be seen in the data from the original odometer readings: almost 25% of mileage ends with a zero. But in the experiment, every digit between zero and nine is also represented in the last digit of the mileage reports. Humans tend to round numbers, but random generators don’t.

The five members of the original research group admit that their study data was fabricated. But all say they have been duped rather than dishonest. “We started our collaboration from a place of supposed trust, rather than earned trust,” Ms. Shu said on Twitter. However, she declined to comment further The Economist. Mr. Ariely’s name is listed as the creator of the Excel spreadsheet with the original data. But he says he has no recollection of the format of the data he received, speculating that he could have copied and pasted the data sent to him into the spreadsheet. One explanation is that the insurance company, or a third party that collected data on its behalf, tampered with the numbers. The Hartford, the Connecticut-based insurance company that allegedly provided data for the experiment, could not be reached for comment. Mr Ariely has called for the study to be withdrawn, as have some of his co-authors. And he is convinced that his mistake was honest. “I did not fabricate the data,” he insists. “I’m ready to take a lie detection test on this.”