A review of Horace Judson's "The Great Betrayal"

Horace Judson published a book on fraud in scientific research about two years ago (Horace Freeland Judson, The Great Betrayal: Fraud in Science. Orlando: Harcourt, 2004. $28.) I wrote an essay review of it some time ago, and I thought I'd post it here.

Judson's book is certainly the best available overall view of the problem of fraud in science. It's very well written, and poses a lot of interesting questions for further research. The discussion of the relationship between law and science (Chapter 9) is outstanding. The discussions of peer review and the role of the Internet are very good.

Much of my essay deals with two issues. The first is the problem of replication and robustness. Replication consists of producing the same results in the same way; it is useful for detecting coincidences and accidents. Robustness comes from the convergence of multiple different lines of activity-- different methods, different lines of evidence, different researchers, different laboratories. Robustness is one of the principal goals of research, and it is also the primary means of detecting fraud.

The second issue follows directly from the first: it is the problem of checkability, i.e., making the work of achieving robustness easy, convenient, cheap and (to the maximum extent feasible) automatic. Our conventions for making research accessible can be changed to improve checkability. Improved bibliographic standards, making data available, and other reforms can be used to reduce the cost of checking by a substantial amount without greatly increasing the workload of scientists. This approach contrasts sharply with using traditional forensic and auditing methods after suspicion has been aroused.