Oct 21
Posted by: benvincent  

Today saw the release of the paper published in the New England Journal of Medicine containing the hotly awaited data concerning the HIV vaccine trial that took place in Thailand. There was already some discussion of the initial results, which were reported in September and discussed by Colin, amongst others. As has already been discussed, there is a very cautious consensus due to the statistical analysis of the trial only *just* falling on the side of significant. Also there were the issues that the efficacy (capacity to provoke an effect, in this case protection from HIV infection) was only calculated as 31.2% overall, and the non-intuitive fact that each of the two components of this vaccine – ALVAC (which contains a modified canarypox virus) and AIDSVAX – have been shown to offer no significant protection when administered alone.

Commentary on the full report is already being written of course (by people paid to do it, unlike me…) with the interesting points that vaccine efficacy may not only fall after the first year following inoculation but also seems to have a higher efficacy in trial members who are in low risk groups.  The point is an obvious one – a whole bucket more questions have been raised by this trial than have been answered. Even if we cautiously accept the significance of the findings and that the vaccine’s efficacy is real, we have sod all idea why or how.

But, as disheartening as this may sound, particularly to those people who have watched every other HIV vaccine project attempted over the years fail, this is certainly a positive finding. The reason for this is that in terms of how science is done and applied, the methodology doesn’t exist in a bubble. When important things like vaccine work get noticed, they also get noticed by activists, journalists, politicians. Policy can be affected, along with public opinion which can have more affect on an academic enterprise than may be obvious.  Funding into HIV vaccine research has been dropping which obviously has not been helped by all round economic belt-tightening at the moment, so this study may provide a much needed boost to a previously bleak field. Awareness is also raised amongst those people who sign the cheques. The global vaccine market is big money whichever way you look at it, though obviously investing in 16,000 strong trials that span years with no product to show at the end of it makes a sad face for any investor who might think it purely profitable. Big cash source, big cash sink. This study is the first indication that work on an HIV vaccine might not just be the latter. Many HIV activists are likely to want to push this too, as the head of IAVI Seth Berkley says:

Years of investment and dogged science are providing leads for solving one of today’s most pressing research challenges. Some 7,400 new H.I.V. infections occur daily throughout the world. Clearly we need better methods of preventing the spread of H.I.V., and no public health intervention is more powerful or cost-effective against infectious disease than a vaccine.”

Though cautious optimism should be present for communicators and advocates this is definite justification that HIV vaccine research is not flushing cash down some impossible non-route. it is a long and tricky path…but finally, a little evidence that it is a path. Paths have ends!

Further reading:





Sep 24
Posted by: colinhockings  


When HIV was discovered as the causative agent of AIDS in 1984 (and has since been repeatedly proven, despite denialists claims – see Ben’s post) scientists were relieved: “It’s a virus! Phew, viruses are something we can deal with. Alright, somebody go cook up a vaccine: problem solved”. According to wikipedia, senior figures in the US health system claimed that a vaccine would be ready in as little as two years. 25 years later HIV is getting the better of us like no other disease, and the vaccine front has seen failure after failure. It seemed to me that it was truly impossible to build a vaccine against HIV and our only hope would be some form of procedure where T cells are removed from the patient, ‘cured’ of HIV (not quite as easy as it sounds, surprisingly enough) and reimplanted into irradiated patients – or something similarly novel (read: impossible and/or impractical). Imagine my surprise then when I finished work to find a BBC article saying that an HIV vaccine had ‘reduced infection’, and after a little bit of surfing I check Twitter to see that ‘HIV’ is trending. Even the Huffington Post had some (relatively) positive things to say.

So what’s all the fuss about? It’s hard to give much detail because the study – beautifully named RV144 – hasn’t yet been published (as far as I can tell, and I’ve looked pretty hard [Update: see footnotes]). There’s several press releases around, from all the various organisations that were involved, including:

  • MHRP (U.S. Military HIV Research Program)
  • UNAIDS (Joint United Nations Programme on HIV/AIDS)
  • NIAID (National Institute of Allergy and Infections Disease)

The study had two (blinded) groups, one control group (receiving placebo injections) and one experimental group (receiving four ‘prime’ doses of ALVAC HIV and two boost doses of AIDSVAX gp120 B/E), with over 8,000 volunteers in each group, lasting from 2003 until now. AIDSVAX has been tested before, but showed no effect, and the ALVAC has only been evaluated for safety. The researchers were quite careful to make the study as ethical as possible, giving lots of good safe sex advice and promising free medication to anyone who contracted HIV during the course of the study (as well as testing the participants for HIV every six months). The impact this has on the infection rate is clear to see in the results: 74 people in the control group and 51 vaccinated volunteers caught HIV – in a country where approximately 1 in 100 people is HIV positive. Before you say ‘well that’s a tiny difference’, remember that there were 16,402 people, which gives the study a fair amount of power. One would, however, need to take a look at the original data to be sure that the statistical significance is really significant. On a side note, I get really annoyed at studies that are released to the press before anyone is allowed to check the data. In this case it’s a large, well-known, highly-anticipated trial run by several reputable institutions so I’m sure their methodology isn’t intentionally dodgy, but it’s still very bad scientific practice.

I found an interesting 2004 release from TAG, the Treatment Action Group, which campaigns ‘for larger and more efficient research efforts…towards finding a cure for AIDS’, saying that the Thailand trial (called RV144) is flawed for several reasons.

  • A single experimental arm won’t allow the relative effects of the two vaccines to be tested. At the time, there had been no clinical trials of ALVAC vCP1521 efficacy and I don’t think that that has changed since then. If it’s still true then they have a very valid point: the experimenters cannot tell how much of the effect is from the ALVAC vaccine alone. TAG also had some concerns that the AIDSVAX gp120 boost may neutralise the other vaccine, based on experiments in macaque monkeys. It is therefore quite possible that ALVAC vCP1521 worked much better, but we won’t know until another large trial is completed.
  • TAG also questioned the ethics of recruiting volunteers – who mostly claimed altruism as their motivation for joining – to a trial that would, at best, lead only to further trials (based on the arguments above).

The reason why such a large trial was performed that won’t actually answer any questions is fairly stupid. It was planned well before two clinical trials for AIDSVAX came back negative, and a trial for the ALVAC vaccine (called HVTN 501) was cancelled. Despite the drastically changed circumstances, RV144 went ahead, leaving us with tantalising evidence that we’re making progress, but not really changing the game much.

On the bright side,  at least we have proof that HIV isn’t invincible. This study follows hot on the heels of the first case of a man cured of HIV (published in February 2009), so I think the entire AIDS research field is looking a little more optimistic. While the number of HIV positive people in the world is still increasing, I think it’s fair to say that we have good cause to be hopeful.


Martin Robbins of the Lay Scientist is not so impressed. Apparently the results are only barely significant (p = 0.048 i.e. there is a 4.8% chance that the difference between the groups is down to random fluctuation)

ERV has an interesting description of thesoft of immunological reaction they were trying to elicit with the Prime-Boost strategy


Katie Stover (Media Representative for NIAID) told me that the study will probably be published in October, in NEJM, to roughly coincide with the AIDS Vaccine conference in Paris

Aug 26
Posted by: colinhockings  

If you’re a skeptic, particularly if you’re interested in the fight against pseudoscience and fuzzy logic in health care, Edzard Ernst is a man you should listen to closely. And if the name rings a faint bell you’ve probably heard of him as the co-author of ‘Trick or Treatment? Alternative Medicine on Trial‘ with Simon Singh. While there are hundreds of skeptical bloggers making a lot of noise about alternative medicine, Edzard Ernst’s full time job is to do it properly. With his group at the Peninsula Medical School (at the Universities of Exeter and Plymouth) he writes systematic reviews and meta-analyses of complementary medicine, as well as carrying out clinical trials (although apparently it’s been a while due to budget constraints) and writing books and articles for various newspapers. In short, he’s a very busy man, doing lots of important work – he’s published over 700 articles in peer-reviewed scientific journals!

In the last few weeks Edzard Ernst published two systematic reviews, one examining the utility of chiropractic spinal manipulation for asthma in children, and the other on chiropractic spinal manipulation for infantile colic. Those of you who are familiar with the ongoing trial of Simon Singh at the hands of the British Chiropractic Association will understand the significance. In short: Simon Singh is being sued because he pointed out that there was no evidence that chiropractic could help children with asthma or colic – among other pediatric complaints – and that the BCA promote it’s use for these conditions anyway.

I’ll give a quick summary of the two studies but I recommend reading them yourself (if you have access to the journals) because they are so short.

‘Spinal manipulation for asthma: A systematic review of randomised clinical trials’

They looked for every relevant study they could find on several databases as well as hand-searching references: a total of 35. Then they took out all the studies that weren’t randomised, didn’t study human patients (of any age), didn’t study chiropractic spinal manipulation (as opposed to other auxiliary treatments that are performed by chiropractors, or spinal manipulations performed by other professionals such as osteopaths), and didn’t study a clinical outcome. These criteria seem quite reasonable to me, but despite the bar being set so low only three studies were usable. Two of the studies concluded that there was no effect, and the third did not compare the control group with the experimental group ‘because of the high risk of committing type I and type II errors’. I don’t quite know if that is sufficient justification to avoid what most would consider the entire point of the experiment, but apparently they claimed to be a preliminary trial to determine the feasibility of a larger study. They did, however, deign to say that there was ‘little or no change’ between the patients before and after the manipulation.

Professor Ernst went on to say that the review wasn’t powerful enough (read: there aren’t enough strong studies) to prove chiropractic as being ineffective, but ’science in general and the RCT [Randomised Clinical Trial] in particular are not good tools for proving a negative’. Thus, and I think this is a very important point that applies to all claims by alternative medicine enthusiasts:

“[It is] the responsibility of those who claim spinal manipulation to be effective to demonstrate this beyond reasonable doubt. In the absence of such proof, any claim that spinal manipulation (or indeed any other therapy) is effective seems unjustified and irresponsible”

‘Chiropractic spinal manipulation for infant colic: a systematic review of randomised clinical trials’

This study was performed in much the same way as the other, however the trials that were considered to be of sufficient quality (3 out of 52) were still fairly poor, with low sample sizes and insufficient controls, as well as rubbish measures of outcome. Thus the conclusions that can be drawn are correspondingly shakier. The two slightly better trials reported no significant differences between the groups, and the methodology of the third trial, which reported a small change, was pretty dire to say the least. They didn’t report on the recruiting process, so it presumably consisted of asking parents to join a trial of chiropractic (and so probably selecting parents who believe that chiropractic will work), then they didn’t blind the parents as to whether or not their child had recieved chiropractic manipulation or a placebo drug, and then they relied solely on the parents evaluation of severity.

Colic is particularly interesting with regard to alternative medicine because we have very little idea about what causes it and no real idea about how to treat it. This sets the stage for lots of worried parents feeling powerless to help their children who are wracked with inconsolable crying, flushed faces, flatulence and ‘meteorism’ (drawing up their legs). The kicker is that it usually resolves spontaneously. It is easy to imagine parents who go their pediatrician being told not to worry, to just sit tight and wait, and when the disease has gone on for long enough they start hunting for alternative ‘remedies’ (here’s your regression to the mean). They may try a few different things and suddenly their child gets better. If I was an evil businessman representing chiropractic I would want to make sure that lots of parents come to see me for their baby’s colic because they’ll probably be converts for life once the child gets better on its own. To give you an idea of the magnitude of this effect, one retrospective study reported that 91% of colic patients showed a positive response to chiropractic.

One final note about these systematic reviews: anyone who would dismiss them as being written by Simon Singh’s friend to stack the evidence against the BCA has a tough case to argue. The point of a systematic review (as opposed to a run-of-the-mill, garden variety review) is to look at all the evidence on a certain topic, and evaluate it according to pre-defined criteria. In addition, the data-extraction and Jadad scoring (an assessment of the methodological quality) were performed by two independent reviewers.

Also highly recommended: Edzard Ernst published one of the best take-downs of the BCA ‘plethora of evidence’ in the BMJ a few months ago.


This post was chosen as an Editor's Selection for ResearchBlogging.orgErnst E (2009). Spinal manipulation for asthma: A systematic review of randomised clinical trials. Respiratory medicine PMID: 19646855

Ernst E (2009). Chiropractic spinal manipulation for infant colic: a systematic review of randomised clinical trials. International journal of clinical practice, 63 (9), 1351-3 PMID: 19691620

—— Update ——

This post was chosen as an Editor’s Selection for ResearchBlogging.org

Jun 28
Posted by: colinhockings  
Green Tea (Wikimedia Commons)

A BBC article on a study (PubMed) by a group, led by Dr. James Cardelli at the Louisiana State University Health Sciences Center, reports that green tea extracts slow the progression of prostate cancer. It has also been reported on by Reuters and (briefly) in the Daily Mail, as well as various blogs such as Attorney-at-Law and BloggingStocks and a health information network that seems to focus on alternative medicine called HealthNews.

The study itself is not perfect. They took 26 patients with stage I, II or II prostate cancer who were scheduled for radical prostatectomy (removing the prostate) and gave them a high dose of polyphenon (a green tea extract marketed by Polyphenon Pharma) until the day before the operation. As the most important outcome (survival) is hard to measure, especially in a single-armed (i.e. one group) trial like this, they measured serum biomarkers (signalling molecules in the serum) that seem to correlate with prostate cancer progression: HGF, VEGF, PSA and the IGF-1/IGFBP-3 ratio before taking the pills and before surgery. They mention the small sample size and lack of a control group but don’t, to my mind, make these limitations clear when they discuss their results. There’s nothing wrong with it, as a purely preliminary trial, but it is very poor evidence. The American Association for Cancer Research (AACR, the body that published the study) press-release has a very important statement about this which was left out of all the reports I’ve seen on this study:

William G. Nelson, V., M.D., Ph.D., professor of oncology, urology and pharmacology at the Johns Hopkins Kimmel Cancer Center, believes the reduced serum biomarkers of prostate cancer may be attributable to some sort of benefit relating to green tea components.

“Unfortunately, this trial was not a randomized trial, which would have been needed to be more sure that the observed changes were truly attributable to the green tea components and not to some other lifestyle change (better diet, taking vitamins, etc.) men undertook in preparation for surgery,” added Nelson, who is also a senior editor for Cancer Prevention Research. However, “this trial is provocative enough to consider a more substantial randomized trial.”

This is the most sensible thing I’ve read about the study, clearly summing up the fact that while there may be an effect, this trial doesn’t show it at all convincingly. It also occurs to me that the samples from before and after treatment were all run on the same ELISA, i.e. the sample from before the treatment was stored at -80°C for between 12 and 214 days longer than the second sample. This may not be a problem, but I didn’t notice any acknowledgement that the researchers considered it. This is why we need control groups! With more participants and a control group you could say ‘well…they might have changed their lifestyles and some parts of the serum may be destroyed by cold and Mercury may have entered Vesuvius, but the treatment group still shows a benefit”. Clearly, such a preliminary trial is not worthy for publishing on its own, so the researchers also tested the safety of such high doses by measuring the liver function (verdict: safe) of the 26 patients and an in vitro study of fibroblast cells treated with EGCG (epigallocatechin gallate, the most common catechin in polyphenon).

Regarding the BBC article itself, I noticed three errors that are quite revealing:

  1. It claims that polyphenon is a compound, which it is not. It is a complex extract from green tea leaves. This is the difference between buying a stack of bricks from the manufacturer and bulldozing a house – a compound is quite pure, while a complex mixture has many unknown components. While not necessarily important to the story, this mistake somewhat undermines my confidence in the scientific literacy of the author.
  2. The location of the study was given as Philadelphia – the address of each researcher (all in Louisiana) is given on the first page of the study. Where could this mistake have come from?
  3. The age range was given as 41-72, but the original paper seems to think it was 41-68.

The age-range of 41-72 also appeared on the Polyphenon Pharma blurb about the trial, published a few days later. The source of the discrepancy: the press-release from the AACR. The 41-72 age range was presumably a typo – if they knew the original article to be incorrect they would presumably correct it: it hasn’t even been published on paper yet.

Imagine my surprise when I re-read the first paragraph of the press-release:

PHILADELPHIA – According to results of a study published in Cancer Prevention Research, a journal of the American Association for Cancer Research, men with prostate cancer who consumed the active compounds in green tea demonstrated a significant reduction in serum markers predictive of prostate cancer progression.

So THIS is where the ‘Philadelphia-based researchers’ comes from – the press-release was written in Philadelphia. And, if we look closer, the first paragraph might also be the source of my other gripe: the ‘compound called Polyphenon E’. I may be wrong, but it seems to me that the author of the BBC article simply read the press release and regurgitated it, without ever taking the time to read the original paper. To be fair, though, I maintain my faith in the BBC News service to some degree because this lax effort was clearly not sufficient. The author got a statement from the head of the Prostate Cancer Charity – a modicum of journalistic investigation was performed, even if it began and ended with a call to the most relevant charity.

The Reuters article is a little better: it has the correct location and age range, and has some extra statements from the lead researcher, James Cardelli. There is nothing to indicate that the author read the original paper, but at least there were no glaring errors. In addition, it finishes with the observation that it ‘was a small trial and bigger studies would be needed to confirm the results’. The only thing I take exception to is the claim that the patients took the pills for ‘about a month’ – while the median duration was 34.5 days, the maximum was 214 days (although that was a special case – the next longest duration was 73 days) and the minimum was 12 days. I guess I may just be hard to please.

In conclusion: while I think it would be good to keep an eye on green tea as a potential supplement to cancer treatments in the future, this study is not the shining beacon that proves it. For a more balanced appraisal of the evidence then you’ll find in the media, I recommend searching PubMed for metastudies and literature reviews. As I expected, they mostly seem to say that while the pre-clinical (on cells in culture and in model organisms) research looks compelling, there haven’t been enough large, good-quality trials to show anything particularly significant in humans.

P.S. If you are a cancer patient and the sketchy evidence on the benefits of green tea make you decide to drink lots of it, first consider: a) green tea has caffeine – some studies of green tea showed a lot of side effects that were presumably due to the caffeine, and b) EGCG may interact with some cancer drugs, especially Velcade (bortezomib) so consult with your doctor. In general, it is a very bad idea to attempt to treat yourself with the preliminary results of trials of unlicenced treatments. They are unlicensed because their effectiveness, and more importantly their safety, isn’t known.


McLarty, J., Bigelow, R., Smith, M., Elmajian, D., Ankem, M., & Cardelli, J. (2009). Tea Polyphenols Decrease Serum Levels of Prostate-Specific Antigen, Hepatocyte Growth Factor, and Vascular Endothelial Growth Factor in Prostate Cancer Patients and Inhibit Production of Hepatocyte Growth Factor and Vascular Endothelial Growth Factor In v Cancer Prevention Research DOI: 10.1158/1940-6207.CAPR-08-0167