That’s from Richard Smith, who was the editor of the BMJ when that editorial was published. And in this enlightening article just published by Mercator (“Most medical research is flawed, says leading medical editor”), Smith goes on to explain why the situation is as bad as ever.
...Sadly, the BMJ could publish this editorial almost unchanged again this week. Small changes might be that ethics committees are now better equipped to detect scientific weakness and more journals employ statisticians. These quality assurance methods don’t, however, seem to be working as much of what is published continues to be misleading and of low quality. Indeed, we now understand that the problem doesn’t arise from amateurs dabbling in research but rather from career researchers.
The Lancet has this month published an important collection of articles on waste in medical research. The collection has grown from an article by Iain Chalmers and Paul Glasziou in which they argued that 85% of expenditure on medical research ($240 billion in 2010) is wasted. In a very powerful talk at last year’s peer review congress John Ioannidis showed that almost none of thousands of research reports linking foods to conditions are correct and how around only 1% of thousands of studies linking genes with diseases are reporting linkages that are real. His famous paper “Why most published research findings are false” continues to be the most cited paper of PLoS Medicine.
Ioannidis’s conclusion as to why so much research is poor is similar to that of Altman’s:“Most scientific studies are wrong, and they are wrong because scientists are interested in funding and careers rather than truth.” Researchers are publishing studies that are too small, conducted over too short a time, and too full of bias in order to get promoted and secure future funding…
Chalmers, Glasziou, and others identify five steps that lead to 85 percent of biomedical research being wasted. Firstly, much research fails to address questions that matter…Secondly, the methods of the studies may be inadequate…Thirdly, research is not efficiently regulated and managed…Fourthly, the research that is completed is not made fully accessible…Fifthly, published reports of research are often biased and unusable…
I reflect on all this in a very personal way. I wasn’t shocked when we published Altman’s editorial because I’d begun to understand about five years’ before that much research was poor. Like Altman I thought that that was mainly because too much medical research was conducted by amateurs. It took me a while to understand that the reasons were deeper. In January 1994 at age 41, when we published Altman’s editorial, I had confidence that things would improve. In 2002 I spent eight marvelous weeks in a 15th century palazzo in Venice writing a book on medical journals, the major outlets for medical research, and reached the dismal conclusion that things were badly wrong with journals and the research they published. I wondered after the book was published if I’d struck too sour a note, but now I think it could have been sourer.
My confidence that “things can only get better” has largely drained away, but I’m not a miserable old man. Rather I’ve come to enjoy observing and cataloging human imperfections, which is why I read novels and history rather than medical journals.
I'd suggest you read the entire article. It really is fascinating. And alarming.
