In September, scientists from the Thailand Ministry of Public Health, in collaboration with the U.S Army, hailed a breakthrough in research towards a HIV vaccine….a positive result. Now, the full results by Rerks-Ngarm and colleagues, published on 20th October in the New England Journal of Medicine, reveal that the outcome of the trial was not so positive after all…..in fact it was statistically insignificant (i.e. the results could have happened by chance).
The $105 million RV144 HIV/AIDS vaccine clinical trial included more than 16,400 volunteers aged 18 – 30 yrs old. Half were randomly picked to receive a combination of two HIV vaccines (ALVAC and AIDSVAX) whilst the rest received a placebo vaccine. The volunteers were monitored after a 6 month vaccination programme and every 6 months for a further 3 years. On the 24th September, the world media reported that the ALVAC-AIDSVAX combo was the first experimental vaccine to have a statistically significant effectiveness (~31%) at reducing the risk of HIV infection.
However, following the publication of the full analysis of the results last week the response from the scientific world has been somewhat muted. The scientists from the study themselves state that the results show only a “modest benefit” and reveal a 26% effectiveness at reducing the risk of HIV infection which was statistically insignificant (could have happened by chance). Also, the protective effects of the vaccine reduced after 12 months and the vaccine did not protect those at high risk of HIV infection (intravenous drug users and sex workers)
So how come there are two very different results within the same study? Basically, it is all down to which volunteers you include in the analysis of the results. The study started with 16,402 people who were randomly assigned to receive the vaccine or the placebo over the course of six months. However, 7 people were found to be already HIV-positive at the beginning and so were removed from the results. This meant that 8197 people received the HIV vaccine whilst 8198 received the placebo (16,395 total). During the course of the study, 2021 people were excluded from the results from the vaccine arm of the study and 1832 people were excluded from the results from placebo arm of the study. People were excluded for a number of reasons such as they were not given the full 6 doses of the vaccine during the course of the study, they were given the wrong dose of vaccine or were not given the vaccine in the correct time period. This meant that the final number of people that match all the experimental criteria for the study analysis was 6176 in the vaccine group and 6366 people in the placebo group (12,542 total).
The level of success of the vaccine depends on which set of people you pick to analyse:
- the “modified intention to treat” set are the 16,395 people at the beginning of the trial (discounting the 7 people who were HIV-positive). Analysis of their results gives you the statistically significant 31% effectiveness of the vaccine
- the “per protocol” set are the 12, 542 people at the end of the study who have received the correct dose and number of vaccines, and were HIV-negative during the course of the vaccination programme. Analysis of their results gives you the statistically insignificant 26% effectiveness of the vaccine.
The full results demonstrate just how difficult it is to interpret results from large clinical trials and the complexities in using statistical significance for confirming evidence-based research. The full analysis of the people who received the correct number of vaccines and the correct dose during the correct time period suggests that the ALVAC-AIDSVAX vaccine trial was not successful and more work is needed to find that elusive protective HIV vaccine.