If you ever get fed up with the amount of exhortations to eat more of something one week, followed by the command to eat less of it the next, you are not alone. I flick through my small pile of journals for a couple hours each week. There is a lot of information and frankly, I don’t have that much time.

In truth, I prioritise on the things that really matter. There are two types of essential reading. There are papers that, although drier than dust, are useful and deserving of thought. And then there are the things that I hear about on my way into work. I have to read up on these, because headlines are the best hint as to what I am going to get asked about in the surgery. It is a forewarning.

Unfortunately, though, there is usually little time to arm myself with the answers. Take, for example, the recent reporting of the Million Women Study in the Lancet, reporting the risk of breast cancer in women using HRT. Despite the ubiquity of email, fax and telephones, doctors seldom have any Department of Health guidelines waiting on their desks just as the telephones start ringing for advice. This, however, is understandable. The Department of Health probably heard the news at the same time as the rest of us, and a delay of 24 or 48 hours is the usual in getting official guidance to healthcare professionals.

Currently, the way that research is done and presented is the greatly debated. “Publish or perish” although clichéd, is fact. Doctors and scientists are under enormous pressure to obtain grants for research studies. Contrary to popular belief, most researchers are not guaranteed funding. Instead, many have a hand to mouth existence, with huge amounts of time spent filling in detailed grant application forms. The competition for some funding, e.g. the prestigious Medical Research Council grants, are intense. This investment in time and effort for an application has often little dividend.

There are few other willing plugs fitting the gaps. The pharmaceutical industry is a frequent sponsor of research. While without it, some research would probably never be done, and, in turn, new drugs may not be developed. But it is not without critics. There is concern that having an industry sponsor creates an inherent bias in the work, and that the researchers, who owe their living to it, may be under confidentiality agreements and may find objective impartiality difficult.

Even if researchers declare a pecuniary interest in the funding of their work, research findings are naturally resistant to neat translation into a snappy, accurate headline. Statistics, particularly those looking at risk assessments, can be difficult to interpret and can be open to wide ranging misinterpretation. A common problem is, for example, reporting that a treatment is likely to ‘double’ a risk. The importance of this doubling 'relative risk' depends on how big the 'absolute risk' (e.g. one in a thousand) was to start off with, and the latter may be tiny.

The Cochrane Collaboration aims, as it says on its website “to make up-to-date, accurate information about the effects of healthcare readily available worldwide.” Access to the Cochrane library - which is free and available through the National Electronic Library for Health, www.nelh.nhs.uk- allows examination of both the absolute and relative risks of treatments and disease.

After research headlines die down, there is usually a period of examination of the paper, followed by correspondence in the journal discussing points of contention. After that, if the results are felt to be valid, the findings may be incorporated in a Cochrane review, or be used as part of the evidence for NICE, or, in Scotland, SIGN guidelines. It can be a long time before published research makes an impact on clinical care, a source of frustration to patients and clinicians. And all that is for treatment that is though to work.

The Royal Society, concerned about how the public can be misled by prematurely enthusiastic publicity, have this week announced that they have set up a working group to investigate and improve the way in which scientific research is publicized.

The chair of the group, Vice President Sir Patrick Bateson, said “The results of scientific research can have quite profound effects on public opinion and policy, so it is important that scientists practice the highest standards of professionalism and integrity when communicating their results. At present, scientists rely almost exclusively on practice of peer review, in which other experts check the quality of results, analysis and interpretation before they are made public, as a safeguard against the communication of poorly conducted research.”

Currently, before a research paper is published, it is usually given to two independent individuals with expertise in the field. Usually unpaid, these ‘peer reviewers’ are asked to check the paper for its accuracy of conclusions and correct methodology.

While this method helps to ensure that no obviously erroneous research slips into publication, it is not perfect. It is still possible for a paper to be peer reviewed, published and then found to be statistically inadequate - or even based on false results.

Sir Patrick continues: “Peer review has been criticized for being too secretive, conducted behind closed doors and assessed by anonymous referees. It has also been suggested that it provides a way in which the establishment can prevent unorthodox ideas, methods and views, regardless of their merit, from being made public. We want to see if any evidence supports such a claim.” In any case, citing the claims about the existence of human clones earlier this year, it is possible, they Royal Society say, to attract huge amounts of publicity without either peer review, or indeed any proof of research.

But peer review is not the only hurdle to publication. Another team may get their results out first, making similar findings redundant; researchers may change jobs, move away, or funding may come to an end. Additionally, it has been known to Cochrane reviewers for some time that studies showing a positive result were more likely to be published, a phenomena known as ‘publication bias.” Negative or equivocal study results do not make it into print so often.

By examining all the current information available about a healthcare topic and by assessing the validity and strength of the evidence, Cochrane groups are able to inform people about the best in ‘evidence based medicine’. However, for this to work, they need to be able to assess all the studies done- including studies that were never published.

Attempting to combat this, several years ago a collective of medical journal editors proposed “an amnesty for unpublished trials”. The idea was that all those researchers having unpublished trial results should place them in a database. The database could be used to inform future researchers and contribute towards informing doctors and patients about the best current evidence for treatments.

A year on, the British Medical Journal declared that, in terms of absolute numbers of research studies submitted, the exercise had been 'a flop'. To minimize this problem continuing in the future, the NHS Research Governance Framework now requires all clinical trials to be registered at inception, and there has been good progress in making this information publicly available through www.controlled-trials.com.

A year on, the British Medical Journal declared that, in terms of absolute numbers of research studies submitted, the exercise had been ‘a flop’. However, to try and minimize this problem in the future, The National Research Registry, and a register within the Cochrane Collaboration, provide databases for researchers to enter the research they are doing. This not only ensuring that the unwitting duplication of research is minimized. This information about what other research teams were doing was previously only available when studies were eventually published- often years from the instigation of the trial.

However, most pharmaceutical companies have so far paid little attention either to registration, or to the Good Publication Practice guidelines produced from within the industry itself (www.gpp-guidelines.org.). This is of great concern to Sir Iain Chalmers, who has been campaigning for many years for greater openness in clinical research. "Patients have agreed to be in trials in the belief that they are furthering scientific knowledge. It is disgraceful that the industry is not reporting all the clinical trials. With electronic publication available, there can be no excuse for failing to publish work. What are these companies scared of? Is their concern for patients or for shareholders? I think that potential participants in clinical trials should not become involved unless they are assured - in writing - that the existence of the trial has been registered publicly and the data to which they will contribute will be made publicly available promptly on completion of the study."

I was trying to look up the Million Women HRT paper on the Internet for a lady who wanted some advice about her HRT. I read through the statistics, which confirmed my instinct that she would be better coming back to see me when I had the official guidance in hand. “That’s fine, she said “But you know, I don’t believe everything I hear.” She may be right; a little well placed cynicism might be rather good medicine.

Link to this article