2010's worst failure in peer review

Even though it is only August, I think I can safely call 2010's worst failure in the peer review process. Just as a sampler, here is the abstract:

Influenza or not influenza: Analysis of a case of high fever that happened 2000 years ago in Biblical time

Kam LE Hon, Pak C Ng and Ting F Leung

The Bible describes the case of a woman with high fever cured by our Lord Jesus Christ. Based on the information provided by the gospels of Mark, Matthew and Luke, the diagnosis and the possible etiology of the febrile illness is discussed. Infectious diseases continue to be a threat to humanity, and influenza has been with us since the dawn of human history. If the postulation is indeed correct, the woman with fever in the Bible is among one of the very early description of human influenza disease.

If you read the rest of the paper, it is riddled with flaws at every possible level. My main problems with this article are:

1. You can't build up a hypothesis on top of an unproven hypothesis. From the first sentence it is clear that the authors believe in the literal truth of the Bible and want to make conclusions out of the Bible, without drawing in any natural evidence. What they believe is their own business, but if they don't have any actual evidence to bring to the table they can't dine with scientists.

2. The discussion of the "case" is completely nonsensical. The authors rule out any symptom that wasn't specifically mentioned in the Bible ("it was probably not an autoimmune disease such as systemic lupus erythematousus with multiple organ system involvement, as the Bible does not mention any skin rash or other organ system involvement") because medical observation was so advanced 2000 years ago. They even felt the need to rule out demonic influence on the basis that exorcising a demon would be expected to cause "convulsion or residual symptomatology".

This really makes me so mad. The basis for getting published in science is really very simple - use the scientific method. The answer doesn't have to fit dogma or please anyone, but the question has to be asked in a scientific manner. How on earth did these authors manage to get a Bible pamphlet past what is meant to be rigorous peer review? Virology Journal is hardly Nature, but with an impact factor of 2.44 it is at least a credible journal (or was, until this catastrophe). At least the journal has apologised and promised to retract the paper:

As Editor-in-Chief of Virology Journal I wish to apologize for the publication of the article entitled ''Influenza or not influenza: Analysis of a case of high fever that happened 2000 years ago in Biblical time", which clearly does not provide the type of robust supporting data required for a case report and does not meet the high standards expected of a peer-reviewed scientific journal.

Okay, Nature has also made some colossally stupid mistakes in letting industry-funded pseudo-science into their pages, but in the 21st century you would hope that scientific journals would be able to tell the difference between evidence-based science, and faith-based pseudo-science.

A breakthrough for HIV prevention?

This week a breakthrough for HIV prevention was announced in Science. AIDS researchers in South Africa just completed a long-term study of Tenofovir Gel, and found that the gel, inserted into the vagina before sex, results in a 40% HIV protection rate for women. With 900 women being followed up for 30 months, the results look very solid, and potentially even better than the headline figure of 39% protection. As with all such studies, the protection rate given is with average usage, not ideal usage. The average study participant only actually used the gel for ~75% of sexual intercourse occasions. For the "high adherers", the group using the vaginal gel for >80% of sexual intercourse occasions, the protection rate was 54%. How important is this breakthrough? In a way, it is both bigger and smaller than the headlines would suggest.

A new tool to fight HIV spread

In the age of vaccines with efficacy rates of >99%, a ~40% protection rate sounds rather poor. Furthermore, this is currently a form of protection only against heterosexual transmission of HIV to women, with no data yet on any protection granted to males having sex with a HIV+ woman or as an anal gel for male homosexual transmission. HIV acquisition by non-sexual routes, such as intravenous drug use, will of course be unaffected by the gel. This is a very poor efficacy rate when compared to condom use. A Cochrane meta-analysis has determined that consistent use of condoms results in an 85% protection rate against HIV, which can go as high as 95% with correct usage. The protective effect is only on par with that of male circumcision, which multiple randomized trials have found protects males from heterosexual HIV transmission at a rate of around 60%.

Is the new gel then completely redundant? A downgrade from the condom? No, not for a key population group - the women of southern Africa. The ten countries of southern Africa together constitute 35% of global HIV cases, with HIV reaching a hyper-endemic situation with 10-30% of adults infected with HIV. In this region, heterosexual spread is the dominant form of HIV transmission, and indeed the risk factor of greatest magnitude at the population level goes to married women. Condom usage in Africa is generally very poor, with an average of only 4.6 condoms available per man per year, due to low demand. Only 7% of women in southern Africa reported using a condom the last time they had sexual intercourse with a regular partner. In particular, women who are food insecure are 70% less likely to use a condom when having sex, with less personal control over sexual relationships. Other women may not use a condom during sex for more personal reference - such as trying to conceive. A vaginal gel therefore provides (partial) HIV protection for the first time to any women who would not otherwise use a condom during sex, either because of personal choice, lack of sexual control, or through a desire to become pregnant.

The other important consideration is that any protection results in a greater number of cases being prevented than the effectiveness of the protection to the individual. This is because each case stopped also prevents the flow-on cases which would have spread from the infected individual. It has been estimated that a weakly protective vaccine, with only a 50% protection rate and only given to 30% of the population, would reduce new HIV infections by more than half, over 15 years. These figures are comparable to the results for Tenofovir Gel, so if the maximal potential is realized, this breakthrough has the ability to halve new African HIV cases.

A tool that will sit idle?

The problem, of course, is that the potential of this gel will not be realized. In many ways, the HIV epidemic is not a problem waiting for a medical solution, but rather a problem waiting for a social and political solution. Consider mother-to-child HIV prevention. Current medical treatment of HIV+ women during pregnancy and after birth reduces the transmission rate to the child by more than 99%. Even in developing countries, the treatment program has over 98% efficacy. And yet these cases, almost entirely preventable under current treatment, make up 15% of global HIV cases and 40% of HIV cases in southern Africa, since only 33% of pregnant HIV+ women in Africa get any form of anti-HIV treatment, let alone the recommended treatment program.

Other strategies, which are already proven to work, could make similar impacts if broadly implemented. Widespread male circumcision would reduce HIV rates by 60% in males and, by reducing prevalence, 30% in females. Comprehensive sexual education focused on preventing new infections can be highly successful. An aggressive campaign of university HIV testing and near universal antiretroviral treatment would be capable of reducing new HIV infections by 95% within 5 years. Just the simple treatment of individuals with genital herpes with current antiherpatic drugs could be expected to reduce transmission of HIV in southern Africa by 50%.

No, a new tool to fight HIV is not going to stop the virus. Realistically, the current tools available could cut new HIV cases by 99% within the decade, if only they were implemented. The true scourge of HIV is that it attacks the marginalised in society, hitting regions of great poverty, infecting those on the receiving side of racial and sexual discrimination. The people that, quite frankly, too many people feel deserve to be sick. Being interwoven with issues of sexuality, drugs, race and poverty, people in power have not only been slow to move - they have often moved in the wrong direction, such as the $15 billion pledged in aid by George W. Bush, with its focus on replacing effective condom use with ineffective "abstinence only" programs.

A major part of the problem is certainly lack of resources, both funding and public health infrastructure. The response to HIV has been delayed, fragmented, inconsistent and grossly under-resourced. Lesotho launched a national voluntary counselling and testing campaign aiming at universal testing, which fell through due to a lack of resources. In South Africa only 28% of HIV+ people have access to antiretrovirals. In Zimbabwe only 4.4% of HIV+ pregnant women are receiving antiretroviral treatment to prevent mother to child transmission. In Nigeria 10% of all HIV transmission events are due to lack of funds for hospitals to screen transfused blood, a situation which requires only funding to remedy. However, funding is not the only impediment to an efficient HIV prevention campaign. Policy makers have repeatedly failed to spend limiting resources on HIV prevention, concentrating on medical treatment without adequate care and support. This is despite the cost of most HIV prevention techniques being well under the $4770 per infection prevented that it would take to create a cost savings compared to simple treatment. What is needed to end the HIV crisis is, in fact, simple in health terms and is difficult only in political implementation – a coordinated and adequately funded approach to integrate evidence-based HIV prevention strategies, in concert with major social and economic development efforts to eliminate gender disparities, race- and sexuality-based discrimination and extreme poverty.

The ethics of biobanking

The University of Leuven hosted two lectures on biobanking today, one by Hainaut from the International Agency for Research on Cancer and the other by Juhl from the biobanking company Indivumed.

Biobanking is a tricky ethical area, with little consensus and vague law. Who owns the material taken from a patient? The patient? The hospital? The surgeon? If someone wants to use the material, what is the default position? Should the patient have to provide consent or is consent assumed unless the patient opts out? Does the patient even have the right to opt out at a latter time point? Hainaut made the case that there is a moral duty on every person to allow access to their biological samples for the good of humanity. His example was that a excised breast cancer not only belongs to that woman, but also to all other women who may develop breast cancer in the future.

This is an attractive argument but has flaws. If the information generated goes into the public sphere, such that new treatments can be developed and accessed, it may be reasonable to use the moral argument, in the same way that organ donation as the default option can be argued on moral grounds. However, to me this argument is flawed if the information generated does not go into the public sphere. If the information is not published (a secretive researcher or company keeping back information for potential future uses) or if it is published with restrictions on use (ie, patented) that information is not open to all of humanity. Isn't it unethical for a biobank to appeal to the moral duty to all of humanity unless legal restrictions are placed on the biobank to ensure that the proceeds of the bank are available to all of humanity? Doesn't informed consent require donors to be told the status of information generated from their samples?

Unfortunately, Hainaut was not able to answer this question when asked, as Juhl (CEO of a biobanking company that only publishes a fraction of the data it generates) jumped in with a rant about for-profit vs not-for-profit. His contention was that every person acts through the personal profit motive, so that whether the biobank made a profit or not didn't matter. His position is that only private companies have the money to put forward to do the research, and they deserve a profit for the research they do. Perhaps, but irrelevant to the ethical question. If the research outcomes are utilitarian then the utilitarian argument should be put to prospective donors - such as DeCode offering all future drugs free of charge to Icelandic people in exchange for access to the medical records and genome of the Icelandic people. Material can be collected for a utilitarian motive using utilitarian appeals, or for a moral motive using moral appeals. What is unethical is to use a moral appeal to collect material destined for a utilitarian purpose.

Hopefully we will see future legislation reflect the ethical considerations of biobanking in more a more thoughtful manner than was presented today. Donations made by the public for the public good should be legally bound to this use. It is illegal for a charity to accept a monetary donation, keep 90% of the money for personal use and spend 10% on charitable works. Likewise it should be illegal for a biobank that accepts material presented as a public donation to only release 10% of the data produced by the donation, and keep 90% to itself.

Faith, post-modernism, science and the approximation of truth

Faith, post-modernism and science all have a different approach to truth.

With faith, the underlying premise (whether articulated or not) is that an Absolute Truth exists, and what is more that the believer has an insight into this Truth. Already knowing Truth, evidence contrary to this Truth must be false and can therefore be ignored. End of debate.

Post-modernism is either the opposite of faith, or just a subset of faith. Under post-modernist thought, there is no objective Truth or Reality, merely individual truths or realities that each person constructs for themselves. Every belief or truth then becomes equally valid, it is just as true to describe the sun as a galactic turnip as it is to talk about hydrogen fusion. Ironically enough, post-modernism does have unquestioning faith in one Truth, the Absolute Truth that there are no absolute truths. The irony is generally ignored.

Science has a third, and fundamentally different, way of conceptualising truth. Interestingly, science uses aspects of both the faith and post-modernistic concepts of truth. Science agrees with faith on the claim that there is an objective truth, or rather an objective reality, that exists independent of any observer. However science also agrees with post-modernism on the claim that an individual cannot grasp objective truth, only subjective truth. The unique contribution of science to the concept of truth is the approach of approximation.

Science does not claim to know Truth the way faith does, nor does it give up on the entire venture as a human abstraction the way post-modernism does. Instead science acknowledges that objective truth exists and attempts to reach the closest possible approximation of truth. Science starts with a model of reality. Scientists then attempt to disprove this model in every conceivable way. Inevitably, every model shows a flaw, an experiment which does not act in quite the predicted manner. The scientific model of objective truth / reality is then forced to change to explain the discordant data. Sometimes an entire model is discarded and a new model is picked up, but far more commonly the original model can continue to stand with a few modified improvements. Scientists then attack this modified model of the truth with renewed vigour. Cycle upon cycle, incremental improvements are made to the model, making it harder and harder to find flaws. Science will never be able to reach absolute truth, but it is extraordinary adept at producing an ever-increasingly accurate approximation of truth. The technology we take for granted today is just one display of how accurate scientific approximations of truth are – the scientific model of the atom does not claim perfection, but our daily use of electron flow (electricity) indicates that the scientific approximation is more functionally useful than any other statement of atomic Truth.

A Self-correcting System

The ability of science as a method to understand reality is demonstrated by the countless successes science has had in developing technology. Antibiotics, vaccination, flight, agriculture, all of these advances clearly work. Why is this? People came up with many ideas to prevent smallpox in the past, but they consistently failed. The development of a smallpox vaccine which actually worked does not demonstrate that scientists have any unique intelligence, but rather it is testimony to the power of a self-correcting system.

Hypotheses are worthless if they are not tested and then discarded if they fail testing. The process of science is not just coming up with an idea of how to cure smallpox, many people clung to their ideas of what would cure smallpox even as they died. Rather, science is testing this idea by looking at the evidence. Uniquely, science discards ideas that just don't work. The simple process of keeping ideas that work and discarding ideas that don't work has built an amazing edifice of knowledge.

The real beauty of the scientific method is that it does not depend on any single person being right or wrong, being ethical or unethical. There will always be scientists who lie or cheat, falsify data or hide experiments that disprove their pet theory. But the hypotheses that these people put forward will always be discarded, because they will fail tests by other scientists.

Best of all, scientists have a vested interest in knocking down incorrect theories. Often you will hear from anti-science campaigners that scientists are hiding data that the theory of [evolution] / [global warming] / [insert hated theory here] is incorrect. They believe in a vast conspiracy of scientists each trying to hold up a false theory for some unexplained nefarious purpose, assuming that scientists don't want to prove a theory incorrect. They fundamentally do not understand the system of science.  Personal glory does not come to the scientists who prove yet again that the theory of gravity works, personal glory comes to the scientist who finds an exception, who proves a theory incomplete, who can unravel the fatal flaw in a centuries old dogma! Einstein, Newton, Copernicus, Darwin, these are all scientists who destroyed the prevailing theories of their age. Every scientist today would love to join their glorious ranks.

A scientist who could prove today that the theory of relativity, evolution or global warming was wrong would publish in the highest journals, win the Nobel Prize, earn household recognition and become rich. There are only two ways a theory such as evolution could still stand today:

1) Every scientist working in the field is deliberately concealing data that disproves evolution, despite knowing that breaking the nefarious conspiracy would earn them recognition as a leader of science, a place in the history books and a lot of personal glory;
or
2) There are no experiments that reveal a fatal flaw.

That is the beauty of science, individuals have huge power to make advances but very little ability to make delays, since theories are judged by experimental results. To reject science you have to reject human nature and believe in an alternative reality where everyone acts uniformly against their personal interests. Trust in science is not trust is individual scientists, it is trust in a system that for thousands of years has produced results, a system that is self-correcting, a system that acts as an 'invisible hand' to select only the models of reality that actually work, regardless of whether the individuals involved were motivated by a selfless search for truth or a greedy struggle for personal glory. The scientific method is an emergent phenomenon which self-corrects the activities of individual scientists to develop only the most robust theories that have so far resisted every attempt to knock them down.