Original article from wired.com
LAST WEEK, THE medical journal theLancet published preliminary results on the efficacy of an Ebola vaccine in Guinea, and everybody got really excited—especially about one particular figure. The vaccine, the results suggested, was 100 percent effective at protecting against Ebola, a thrilling prospect in the face of an epidemic that has killed more than 11,000 people. That number is why Doctors Without Borders is recommending distribution of the vaccine begin as soon as possible in the west African countries where Ebola is still killing people.
But that number probably means less than you think it does. It’s based on incomplete data, so it doesn’t have the statistical clout it should. And it never will. Based on the vaccine’s early success, the trial’s runners decided that all participants in the study should get it immediately after exposure. That’s a perfectly reasonable, humane reaction, but it also means that the researchers will never be able to collect better data on the vaccine’s efficacy, which is what regulators look for when they’re deciding to approve a drug. In other words, the vaccine’s early success could make it harder for people to get it down the line.
The main thing that makes the new vaccine, rVSV-ZEBOV, seem so great? There’s simply nothing else available. Trials of Ebola vaccines have been almost impossibly difficult to pull off. In the midst of a massive public health crisis, international and local health organizations had to design studies that they could carry out in unstable, sometimes remote regions while producing useful data about safety and efficacy. “It’s a high-risk thing to take on a large scale vaccine trial,” says Anne Schuchat, director of the CDC’s National Center for Immunization and Respiratory Diseases. “It usually takes a few years to lay the groundwork.”
With that in mind, this clinical trial’s speed—moving from Phase I safety to Phase III efficacy tests in under a year—is impressive. But by the time efficacy trials started around April, the epidemic had already begun to calm down. That was a great thing for west Africa—but not such a good thing for statisticians. With fewer people exposed to Ebola, it’s harder for studies like the one in Guinea to find community clusters where it makes sense to test the vaccine. The problem was even more pronounced for a parallel study being carried out in Sierra Leone by the CDC and their partners, which focused on testing the vaccine’s ability to prevent infection in health care workers.
Here’s why: The Guinea trial of the vaccine tested efficacy by comparing infection rates in two groups. One got vaccinated one to three days after confirmation of an Ebola infection nearby; the other got vaccinated three weeks later. The Sierra Leone trial, in contrast, delayed vaccination to one of their groups by a full six months. Because the epidemic subsided in the intervening months, it’s hard to know what the resulting data actually means. Did the vaccine stop infections, or did the infections just stop? “As the Sierra Leone epidemic has come under control, we think it’s not likely that we’re going to be able to measure efficacy in our trial,” Schuchat says.
That makes the Guinea trial even more important. Typically, regulators like the US Food and Drug Administration require very high standards of proof to approve a vaccine, especially for preventive use—like getting a shot for Hepatitis B before traveling. In terms of efficacy data those regulators will look at, Guinea is it. And now there’s an extra wrinkle: Because the vaccine’s interim results looked so good, the Guinea trial is no longer randomizing the recipients of the vaccine. Everybody who gets placed into one of the study’s clusters will get it immediately, no delay. Again: great for west Africa, not so good for the statisticians.
What the statisticians do have to work with—that 100 percent efficacy—isn’t as mind-blowing as you might assume. Of the immediately-vaccinated people, nonedisplayed symptoms 10 or more days after vaccination, compared to 16 infections in the group of 2,380 that was assigned to get vaccines three weeks late. (It take two to 21 days for symptoms to manifest after infection, so a person who came down with symptoms in those first 10 days could have been infected before the vaccine took effect.)