There's joyful news reports all over the place today describing great news about a new Alzheimer’s drug. But, as you'll see in this Wall Street Journal report, when carefully examined the results of the study are not very impressive after all....The companies had already revealed data showing the drug, called bapineuzumab, helped people who were free of a form of a gene that’s a risk factor for Alzheimer’s. But patients with that form, called ApoE4, showed only a trend toward improvement — not a statistically significant one.
Now, in another tough year for Alzheimer’s drugs, the companies presented more detailed data today at a big Alzheimer’s conference. The results showed that none of the patients — with or without the genetic issue — saw more benefit from a higher dose of the drug than they did a lower dose...
WSJ reporter Sarah Rubenstein asked Ronald Petersen, a neurologist at the Mayo Clinic who serves as the chairman of the medical and scientific advisory council of the Alzheimer’s Association, about the news. Responded Petersen, “I can’t tell if the compound’s efficacious or not at this point in time, but (Wyeth and Elan) learned some important information going forward about both the possible efficacy and safety.”
There are several interesting comments appended to the original story, one defending the enthusiasm over the new drug but several reflecting skepticism and even cynicism. For instance, an anonymous physician wrote in, "This will be JUST LIKE ARICEPT - a huge expense to taxpayers (ie. medicare part d) and patients (ie. copays etc.) for NO BENEFIT. Except this will be infinitely more expensive, because it ends in “mab” ie. monoclonal antibody. Any honest physician actually treating patients with dementia will attest to the total lack of efficacy of Aricept."
Most valuable in the comments section, however, was a note from David Hamilton with a link to a pretty compelling piece he had written for Pharma Industry hosted by BNET. Hamilton, by the way, wrote for the WSJ himself for a 14 years but now freelances. He most recently founded the LifeScience section of VentureBeat. Hamilton has covered many issues over his career but specializes in science and technology. He is a two-time winner of the Overseas Press Club award.
I suggest you take a look at Hamilton's review -- it's a quick read -- and you'll see that there are several important reasons to beware the hype about Bapineuzumab. Here are some excerpts:
In what is becoming a sadly common ritual, Wyeth and Elan are pressing forward with an expensive, large-scale “phase III” trial of a risky drug based on wishful thinking and shoddy statistical analysis...
Earlier today, Wyeth and Elan disclosed detailed results of the drug’s phase II trial, which found that bapineuzumab failed to improve cognitive function in a test of 234 Alzheimer’s patients after 18 months of treatment. You could be forgiven for not gleaning that from the companies’ joint press release, however, as Wyeth and Elan chose instead to highlight post-hoc analyses that purported to demonstrate the drug’s efficacy in a subset of patients who don’t have a gene variant called ApoE4, which increases the risk of Alzheimer’s.
To put it bluntly, this is magical thinking on a truly impressive scale...
Hamilton then gives four specific criticisms of the study, four points to remember when evaluating the news story about this "breakthrough."
* Prospective measures of success are the only accurate way to judge trial results. Honest clinical trials require researchers to specify in advance what they’re looking for — and by that measure, the bapineuzumab trial was a failure.
* Post-hoc subgroup analyses amount to lying with statistics. By contrast, a post-hoc analysis involves mining the trial data in order to identify some group of patients who appeared to benefit from the drug. It’s tantamount to moving the finish line after the race is over — or, as FDA’s Richard Pazdur memorably put it, firing an arrow into the wall and then drawing a target around it.
* Such subgroup analyses rarely hold up under further study. Or, as the old computer-science saying goes, “Garbage in, garbage out.”
* Drug companies will do and say almost anything to boost the promise of a potential blockbuster. Wyeth and Elan don’t expect data from the phase III trial in patients without the ApoE4 gene variant until 2010. A lot can happen in that time, including the possibility that the FDA will once again warm to the idea of approving drugs based on marginal evidence. It’s like the old joke in which a prisoner staves off execution by promising to teach the king’s horse to sing within a year, reasoning: “A year is a long time. The king might die. The horse might die. I might die. And maybe the horse will learn to sing.”...
Hamilton concludes his report with this warning, "Whenever companies start talking up after-the-fact subgroup results, it’s time to hold onto your wallet."
