BMJ: Is animal research sufficiently evidence based to be a cornerstone of biomedical research?

Poor quality and reporting of animal studies

The overall number of systematic reviews of animal studies remains lamentably low, with the ratio of reviews to total number of publications being about 10-fold higher in human studies.5 In 2011 Korevaar and colleagues identified 244 systematic reviews of preclinical studies up until 2010, estimating that the number was doubling every three years.6

As the number of systematic reviews increased, the poor quality of much preclinical animal research became increasingly apparent.7 Evidence accumulated that many animal studies failed to address important threats to internal and external validity, making prediction to humans tenuous at best.8 9 For example, the National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs) surveyed 271 animal studies conducted between 1999 and 2005 and found that only 32 (12%) reported using random allocation to treatment or control and that investigators were blinded to the allocation in only 14% (5/35) of studies that used qualitative scoring.10 Systematic reviews of animal studies also revealed evidence of selective analysis and outcome reporting bias11 as well as publication bias12 leading to overstatement of the validity of entire bodies of research.13

The Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies (CAMARADES) has been at the forefront of conducting systematic reviews of animal studies. Initially focusing on stroke, it later expanded to include neurological disease, bone cancer, multiple sclerosis, and Parkinson’s disease. By 2012 John Ioannidis, professor of health research and policy at Stanford, concluded that CAMARADES had found consistent suggestions of serious bias in animal studies, making it: “nearly impossible to rely on most animal data to predict whether or not an intervention will have a favourable clinical benefit-risk ratio in human subjects.”14

Lack of benefit for humans

Concerns have been raised that compounds with little or no therapeutic potential could proceed to clinical trials because overoptimistic conclusions are drawn about their efficacy as a result of flaws in experimental design and inadequate control of bias.15 16 17 18 19 Several studies have shown that even the most promising findings from animal research often fail in human trials and are rarely adopted into clinical practice.20 21 22 For example, one study found that fewer than 10% of highly promising basic science discoveries enter routine clinical use within 20 years.23 In stroke medicine, despite decades of immense human, animal, and financial investment, animal models have failed to yield a single neuroprotective treatment for humans.24 25 Similarly, none of more than 100 drugs studied in an established mouse model of amyotrophic lateral sclerosis, many of which had been reported to slow down the disease, was ultimately found to be beneficial after more rigorous experiments. Eight of these drugs had been used in thousands of patients who participated in failed clinical trials.26 A similar lack of translation has become apparent in inflammation.27

Falling investment in basic and animal research

Public funding bodies are becoming aware of the lack of return on investment, and public and charitable spending on basic research has decreased in the UK from 68.3% in 2004-5 to 59.4% in 2009-10.28 This seems wise since retrospective analysis of the payback from research is beginning to suggest that it is clinical rather than basic research that has most effect on patient care.29 30 Almost half of all research involving animals in the UK in 2012 was conducted by universities (48%), the remainder occurring in commercial organisations (27%), public bodies (13%), and non-profit organisations (9%).31 The drug industry is also beginning to decrease its reliance on animal research because each translational failure represents huge losses of invested capital.21 32 In Europe drug companies have reportedly decreased their use of animals by more than 25% from 2005 to 2008.33

A broken model?

The animal research community continues to cite selected instances of how research on animals has resulted in medical advances, or will one day do so (see www.understandinganimalresearch.org.uk/resources/animal-research-news-feed/). However, these convey little confidence about the overall reliability and success of animal models, taking into account the total evidence. Given the large amount of animal research being undertaken, some findings will extrapolate to humans just by chance. Understanding Animal Research, a British organisation financed mainly by those conducting or funding animal research, highlights four reports purporting to support the validity of animal research,34 all of which rely solely on expert opinion, one of the weakest forms of evidence according to widely agreed standards.35

Would improvements in preclinical experimental procedures and research reporting enhance the prediction from animals to humans and provide greater benefits for humans? In an article reviewing developments in the field of stroke, Sutherland and colleagues note that despite researchers adhering to recommendations intended to improve the quality of preclinical stroke studies for over 10 years, there is no evidence of an increased rate of successful translation.25 Others argue that animal models will always fail to predict human outcomes reliably because humans and animals are such complex interactive systems with different evolutionary trajectories that even small differences between species could be important.36 The genomic and inherent differences between rodent and human physiology are increasingly acknowledged,37 and even non-human primates have many differences in the epigenome that fundamentally affect the functionality of the genome38 and may account for their lack of success in predicting clinical response.39 40 41 Even if the research was conducted faultlessly, animal models might still have limited success in predicting human responses to drugs and disease because of inherent inter-species differences in molecular and metabolic pathways.42

The use of transgenic animals, in which the genome has been changed by insertion of foreign genetic material, attempts to increase the validity of animal models by making them more closely resemble human phenotypes of interest. Yet transgenic models, where genes are regarded as operating largely independently of each other, have been criticised as limited,43 oversimplistic, and, at least to date, as contributing more to an idea of therapeutic promise than actual clinical outcomes.21 36 Furthermore, it has been observed that transgenic animals do not always produce the desired phenotype after cross breeding several generations, thereby undermining the rationale for this research strategy.26

Attempts to improve animal research and reporting

In response to the serious deficiencies found in the conduct and reporting of animal studies the ARRIVE (Animal Research: Reporting In Vivo Experiments) guidelines 44 were produced in 2010. Over 300 journals and the major UK funding agencies have endorsed these guidelines, but a recent survey of papers published in Nature and PLoS found little improvement in reporting standards.45 A Gold Standard Publication Checklist has also been developed by SYRCLE (Systematic Review Centre for Laboratory Animal Experimentation) in the Netherlands to encourage more rigour in the conduct, not just reporting, of animal research.46

Michael Festing, a retired Medical Research Council scientist, recently acknowledged that few basic scientists receive any formal teaching, most relying on what they learn from their supervisor.47 Similarly, the leadership of the National Institutes of Health in the US recognises that poor training may in part be responsible for the lack of reproducibility of animal models.48 The UK Fund for the Replacement of Animals in Medical Experiments now offers voluntary workshops in experimental design and statistical analysis, and an online course in experimental design (www.3rs-reduction.co.uk) has been developed. Training is also available for preclinical investigators to learn how to conduct systematic reviews (www.syrcle.nl).

In 2008 the Medical Research Council (MRC) funded a pilot “research translator” at an English university hospital site to try to facilitate the translation of findings from bench to bedside. One of the findings from a qualitative study investigating this initiative was that basic scientists’ motivation came from scientific discovery rather than the application of their findings to medicine.49 Recent attempts to improve translation within the animal research community include the “co-clinical trial” in which preclinical trials explicitly parallel ongoing human phase I and II trials50 and the development of a scoring system to identify biomarkers that better predict therapeutic success.51

Time for change

The culture within research is shifting, and animal research is no longer as immune from challenge or criticism as it once was. Nonetheless, although science is more self critical, in practice it can be difficult to achieve change because stakeholders (governments, funders, universities, allied research industries, and researchers) may all have interests, not infrequently financial,52 in continuing to do things as they have always been done. Although there are also valid criticisms of clinical research,53 urgent attention needs to be paid to the quality of animal research for important reasons.

Much clinical research follows on from animal research. If the foundations of the biomedical research enterprise are unsound, then whatever is built on these foundations will be similarly precarious.

The current situation is unethical. Poorly designed studies and lack of methodological rigour in preclinical research may result in expensive but ultimately fruitless clinical trials that needlessly expose humans to potentially harmful drugs or may result in other potentially beneficial therapies being withheld. Moreover, if poorly conducted studies produce unreliable findings, any suffering endured by animals loses its moral justification because their use cannot possibly contribute towards clinical benefit. Non-publication of animal studies is similarly unethical because the animals involved cannot contribute towards the accumulation of knowledge and because non-publication may result in further, unnecessary animal and human experiments. 13

In addition to intensifying the systematic review effort, providing training in experimental design and adhering to higher standards of research conduct and reporting, prospective registration of preclinical studies,54 and the public deposition of (both positive and negative) findings would be steps in the right direction.18 Greater public accountability might be provided by including lay people in some of the processes of preclinical research such as ethical review bodies55 and setting research priorities.28 However, if animal researchers continue to fail to conduct rigorous studies and synthesise and report them accurately, and if research conducted on animals continues to be unable to reasonably predict what can be expected in humans, the public’s continuing endorsement and funding of preclinical animal research seems misplaced.