The scientific process is an amazing thing. Between the Palaeolithic era (the era between 2 million and 20,000 years ago) and the year 1900, the average life expectancy at birth hardly increased at all, remaining around 30 years. Since then, as a result of the application of the scientific process to health care, it has more than doubled, with the worldwide average now being 71 years. Consider that. In just 100 years we have given ourselves more life that we managed in the previous 2 million years! Through science (the process of making observations, developing a hypothesis, testing that hypothesis and then refining and building on it) we have transformed our lives.Science is based on the fact that observations are consistent. If, for example, a surgeon washes his hands before operating, his patients will consistently get fewer infections. Without this consistency, this ability to repeat an experiment and get similar results, the whole scientific endeavour stops. This is why a recent publication in the journal PLOS Biology is so worrying.The article suggests that 50% of life sciences research cannot be reproduced and is therefore useless. Furthermore, the authors estimate that $28 billion is wasted in the US every year on studies that are never repeated. It must be pointed out that the way the authors defined “reproducible” was extremely broad, and as a result, there is a large margin of error in their estimate. However, the study does shine an uncomfortable light on the current state of science. It has been acknowledged that there is a “reproducibility problem” in the life sciences at the moment. A previous study by scientists at Bayer Healthcare stated that their own staff could only reproduce 24 of 67 studies published in peer-reviewed journals.
These reports suggests that most of the irreproducibility is as a result of poor study design, biological reagents that are not publically available, badly described protocols, and poor interpretation of results. Poor study design is mostly due to (inexcusable) bad lab practice, whereas the problems with reagents and protocols largely result from an extremely restrictive word count applied to the material and methods section of most journals. This word count means that the way experiments are carried out isn’t adequately described, and as a consequence others can’t repeat it. Finally poor interpretation of results is down to a lack of understanding of statistics and a selection bias that exists in science. This selection bias means that positive results are far more likely to be published than negative ones. In the extremely competitive scientific environment, this unfortunately forces authors toward sensationalising their findings in order to get them published, often to the detriment of the study itself.Theoretically, the process of peer review (wherein papers are critically analysed by their peers before publication) should ensure good quality science. This process however does not tackle the problem of the editorial policy, which does not reward studies whose results are unspectacular but reliable. Such studies are essential for scientific progress. For example, a paper was published in the journal Nature in 2013 demonstrating a method of generating stem cells. However, the same journal refused to publish a study the following year that showed that this method was entirely false.At present, a scientist is measured by the standard of his/her publications. To be awarded a grant you have to show that you can publish in a high impact journal. As a result, there is little incentive in science to reproduce other’s work, even if it is just part of a study. Furthermore, if a study does fail to reproduce another, it is usually filed away in a drawer and is not published. Unfortunately, the quality and reproducibility of someone’s work is not included when they are being judged as a scientist.A lot of money is spent every year in life sciences research. In the US, the budget is around £35 billion ($56 billion). Numbers are harder to come by in the UK, but the total spend on science and engineering is roughly £5.8 billion. While in the grand scheme of spending, this is a small amount (pensions alone cost the UK £1.5 trillion per year) it is still vital that the money is spent efficiently. The public, either through charity or governmental means, largely funds the life sciences. If there is a public perception that the results cannot be trusted, science risks losing the confidence of the public, and potentially the funding that is so essential for this work. The advances in public health that we have made in the last centaury prove how important this research is and how important it is that we remedy this reproducibility problem.
