We now have the official clinical trial designs for the three biggest and most advanced coronavirus vaccine trials: Moderna, Pfizer/BioNTech and Oxford/AstraZeneca. Now, as for you, J&J, Novavax, Merck, and all the others that are pushing into efficacy trials as fast as possible – don’t think for a moment that you won’t be expected to do the same. But this is a good start. So how do these trials differ?
Let’s start off with some background, starting with “vaccine efficacy” (VE), which is pretty much what it sounds like: the percentage difference of infection by the virus between the vaccinated group and the unvaccinated control group. The FDA has already said that they want to see 50% at least (i.e., the vaccine cuts the rate of coronavirus infection at least in half). All of these trials have different subgroups, especially broken out by age and/or pre-existing conditions, and there will be not only an overall VE for the trial but separate VEs determined for all of the predefined subgroups. As is standard, the trials will be using what’s called a Cox proportional hazard regression model to assess the differences between the various groups – I won’t go into the details here, but that’s how you get the eventual figures of “Group A is 2.1x less likely to get infected than Group B” and so on. The Cox model actually gives you the “hazard ratio” (HR) which is just the opposite of the VE, so a 60% HR would mean a 40% VE. Update: this is one way to do it – the various trials have different statistical approachs.
The trials will be doing a similar analysis for different endpoints as well. For example, rather than calculate what effect the vaccine had on just sheer infection by the coronavirus, they will also be checking how many of the infected cases were severe (as opposed to those cases in the placebo group), how many asymptomatic cases there were, how these categories might look under alternate diagnostic criteria, the comparisons between these figures and different immunological readings (neutralizing antibody titers, for example), how many deaths took place, and so on. The same sorts of calculations will be going on for adverse events, looking for differences between the vaccinated and placebo groups and how these differences might change across the various subgroups. We’re talking a lot of data for any given trial, and with the number of trials going on, it’s going to be an absolute downpour of information.
The next thing to go into is statistical power. I will again avoid breaking out any equations, but the idea is that given a certain rate of infection in a population that you need enough patients in such a trial to demonstrate a given level of VE within given statistical bounds. The key to a controlled clinical trial is to lay out your statistical landscape in detail before you start, defining what success and failure will look like, and making sure that you have enough patients (and will see enough events) to distinguish them to an acceptable amount of rigor after a given amount of data gets collected.
All three trials have basically the same definition of failure, which would be a VE of 30% or worse. And they are all designed to have greater than 90% chance of demonstrating that the VE is better than that (if in fact it really is, and it had better be!) That’s the first thing you have to make sure of: to be able to say if the vaccine has succeeded or failed, and it should go without saying that if such a trial just enrolled (say) 100 people that it would be impossible to make such a call. And it would be even more impossible to say anything about safety in such a small study – adverse events are expected to be rare (but by no means impossible!) and you’ll thus need a sample size that can at least let you set some lower bounds on their frequency.
So all these trials are enrolling at least 30,000 people. Moderna is dividing their enrollees into 15,000 treatment patients and 15,000 controls. Pfizer is also 1:1, but AstraZeneca is going 2:1 treatment/placebo (more on that in a moment). That sample size is also related to how quickly you’ll be able to determine success, as you would figure. Remember, in a trial like this you’re waiting for people to get infected with the coronavirus – the number of people infected in the treatment group versus the control group is the most important data point in the entire trial. That’s why you want not only to have a lot of people, but to run the trial in areas where the virus is actively spreading.
Another important thing is the how you specify the interim analyses. Clinical trials often aren’t run “straight through” to the final data collection while flying blind the whole way. There are independent groups that keep an eye on the data (and on the adverse events), variously known as Data Monitoring Committees or Data and Safety Monitoring Boards, etc. These people are kept separate from the investigators, obviously. You need to lay out your plans for such interim looks at the data in advance, to avoid the temptation to move the goalposts once you’ve seen what’s going on. Note that none of these trials going to stop the study if they see good results early on – they’ll continue to collect data, but in the knowledge that they’re on to something effective. That’s both to get better statistics on the efficacy and to keep an eye out for safety: all these trials have prespecified alerts and stopping criteria if the DSMB sees a given number of adverse events as well.
So for Moderna’s trial, they have estimated that they are likely to see about 150 infection events during the whole trial. They have specified an interim analysis at 53 events (35% of the way through) and another at 106 events (70% of the way). At that first IA, the study will be considered to be already declared a success (rejecting the null hypothesis that the VE is only 30% or below) if the p-value for such a rejection is less than 0.0002. That would mean that the vaccine itself would be at least 74% effective. If the efficacy doesn’t meet the cutoff at the first IA, success will be declared at the second one if the p-value for rejecting the null hypothesis is less than 0.0073, which would mean a VE of at least 56.5%. If they have to go all the way to the end, then they’ll need a p-value of less than 0.0227 to reject the null, and that would mean a VE of 50% (the FDA’s floor, and it is no accident whatsoever that these two coincide).
As for Pfizer/BioNTech, they have a somewhat more aggressive approach. They calculate that they’ll hit 164 cases by the end of the study, and they have four IAs planned, at 32, 62, 92, and 120 cases. If they can reject the null hypothesis at that first one, that will be considered “overwhelming efficacy”, with a VE of at least 77%. The VEs if they clear the bar at the later analyses instead are 68%, 63%, and 59%, and if they make it only at the end of the study that would be a VE of 52%.
And the Oxford/AstraZeneca trial also estimates that they’ll see 150 cases by end of the study, and they have one interim analysis set at the 75-case mark. They estimate that will give them > 70% power to detect a VE of 70% and > 90% power to detect a VE of 75%. By the end of the trial, they believe that they’ll have a 90% chance of being able to say if the VE is 60% or greater.
So how quickly will these trials hit these readouts? That depends completely on the attack rate of the virus in the study population, as mentioned above. The more you are testing in viral hotspots, the faster you will collect data. If you decide to test in New Zealand, on the other hand, you will probably never hit the cutoffs at all. Pfizer has said several times that they expect to get a first readout by the end of October, and Moderna has said that they expect to get a look by the end of November. AstraZeneca, with only one interim analysis, will probably have to wait a bit longer from the start of their trial, although they did start on the early side. I would expect the companies involved to announce a positive result if they do make any of these interim analysis hurdles, though (wouldn’t you?)
So that’s roughly where things stand, and I hope this gives people a clearer picture of what’s going on. Now, as with all trials, we just have to wait for the data to come in. . .