Practice recommendations

* When consulting narrative review articles, carefully consider the possibility of citation bias and whether recommendations are based on patient-oriented and not disease-oriented outcomes.

* Consult credible evidence-based resources such as Cochrane before making changes in practice suggested by narrative reviews.

Abstract

Background Citations in scientific articles may tend to favor the views presented. We studied whether there is such reference bias in narrative review articles that discuss interventions against house dust mites for people with asthma.

Design Systematic review of reviews identified in a Medline search that expressed an opinion about the clinical effects of physical or chemical intervention methods.

Main outcome measure Positive bias was judged to have occurred if the reference list contained a higher proportion of trial references with significant results than among all trials available to the authors (published 2 years or more prior to the review).

Results Seventy reviews were included, of which 63 (90%) recommended physical interventions. Forty-six reviews had trial references, 4 of these only to chemical interventions. In the remaining 42 reviews, reference bias was detected (P=2 x 108). The most quoted trial had only 7 patients per group, its claimed significant result was probably erroneous, and it did not report a clinical outcome. Intervention recommendations were often based on nonrandomized evidence, and the most quoted nonrandomized controlled study had included only 10 patients per group but claimed very positive results.

Conclusion The narrative review articles were severely biased, and their positive intervention recommendations are at variance with the systematic Cochrane Review on this topic and a recent very large trial of physical intervention, both of which failed to find an effect.

Trial reports and narrative review articles sometimes favor references supporting the views of the authors, lending credence to a particular treatment' or hypothesis. (2,3) This reference bias may render the conclusions of an article less reliable. Such was the finding in our study of narrative review articles discussing interventions against house dust mites for people with asthma.

Systematic reviews are a more reliable source of information for busy clinicians than narrative reviews, but they can also be problematic. An assessment of papers on asthma, based on a validated tool, showed that 40 of 50 systematic reviews on prevention and treatment had serious or extensive flaws. (4) Cochrane systematic reviews on asthma were more rigorous and better reported than systematic reviews published in peer-reviewed, paper-based journals, (4) probably because Cochrane Reviews are conducted according to a set of standard methods (5) aimed at minimizing bias in the reviews, and because they usually include only randomized trials.

As the index review for our study, we chose the Cochrane Review on house dust mites control measures for asthma, (6) to allow us to judge whether the recommendations in narrative reviews reflected fairly the available, reliable scientific evidence.

* Methods

Inclusion criteria. Narrative reviews--that is, reviews that did not have a methods section with a search strategy for relevant research papers--were eligible if they expressed an opinion about the clinical effects of chemical or...