The Cochrane Skin Group: Title and subTitle BreakPreparing, Maintaining, and Disseminating Systematic Reviews of Clinical Interventions in Dermatology FREE

It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials.—Archie Cochrane¹

Thus spoke Prof Archie Cochrane in 1979, making an obvious and simple point about our collective ignorance on the effects of health care interventions.¹ Four years after his death in 1988, the Cochrane Collaboration (herewith referred to as the Collaboration) was formed in response to this challenge, "to prepare, maintain, and disseminate reviews of the effects of health care" based on randomized controlled clinical trials (RCTs).² The first Cochrane Centre was established in 1992 in Oxford, England, under the direction of Prof Iain Chalmers. Since then another 14 Cochrane Centres have become established throughout the world. The main working unit of the Collaboration is the collaborative review group—groups of professionals from a range of backgrounds and perspectives who volunteer to work on producing systematic reviews of clinical problems in a defined area of health care. Currently, 43 review groups are registered with the Collaboration representing diverse aspects of human health, such as stroke, parasitic diseases, addiction, schizophrenia, and childbirth.³ In addition to review groups, Cochrane Fields, a division of the Cochrane Collaboration, deals with issues that are common to several review groups, eg, the Cochrane Cancer Network, the Cochrane Primary Care Network, and the Cochrane Consumer Network. There are also various methods working groups within the Collaboration whose purpose is to develop appropriate methods for summarizing the effects of health care. One example of such development is the Review Manager software (RevMan),⁴ which helps reviewers organize, prepare, analyze, and present their systematic reviews. The Collaboration is administered by a Steering Group and Secretariat, which have professional links with organizations such as the National Library of Medicine.⁵

The main output of the Collaboration is the Cochrane Library, an electronic publication that is updated quarterly.³ The Cochrane Library contains 4 main elements:

Although the scale of these organizational issues may sound impressive, the Collaboration is all about people. It is a voluntary organization in which physicians, other professionals, and consumers work together unselfishly to try to understand the "truth" about medical interventions. The basic principles on which the Collaboration is based are as follows³ :

A systematic review is an explicit and structured method of summarizing the effects of a health care intervention. It aims to reduce bias and resolve conflict by using a systematic approach and techniques, such as meta-analysis, the statistical pooling of evidence from separate RCTs that are sufficiently similar.⁶ The steps involved in conducting a systematic review are as follows: (1) framing an important and answerable question, (2) defining the outcome measures, (3) systematically retrieving of all relevant study reports, (4) abstracting and critically appraising the quantitative information, (5) summarizing the evidence by using statistical pooling techniques if appropriate, and (6) interpreting the results. Each of these steps is discussed more fully by Li Wan Po and Williams⁷ and in the Cochrane Collaboration's manual.⁸

The interpretation of systematic reviews inevitably invites some subjective input on behalf of the authors of the reviews. In some instances, the strength of the evidence does not permit development of a clear bottom line.⁹ In these circumstances, explicitly describing the available evidence and pointing out the gaps in knowledge and what needs to be done to address the gaps is often better. Just like any study design, systematic reviews can be good or bad.¹⁰ Sometimes combining data quantitatively is inappropriate, and other techniques, such as best-evidence synthesis or a search for the reasons for conflicting findings of studies, are more informative.¹¹ There is no such thing as a "quick and dirty" systematic review. Each review is a painstaking process that takes many months to produce from protocol to final review. The process requires meticulous attention to detail and consideration of possible sources of bias, and it generally involves many persons. Readers should refer to Davey Smith and Egger^{12 - 13} for some excellent reviews of the limitations of systematic reviews and meta-analysis.

It is important to question the need for the Collaboration. During the last 30 years, there has been an exponential rise in the number of published scientific articles. More than 2 million health research articles are published each year, making it impossible for health care workers to keep up-to-date on all current information. The findings of one study in the United Kingdom suggested that a clinician's average weekly reading time is less than 1 hour.¹⁰ In an attempt to keep up, busy physicians often turn to journal review articles, which are usually narrative reviews. These articles may be useful for reminding physicians of the main issues and main published evidence, but they are notoriously prone to bias, with concluding remarks often at odds with the evidence discussed in the article. This is because such review articles are rarely developed in a structured and explicit way.¹⁴ As Ladhani and Williams¹⁵ found in a recent comparison of traditional and systematic reviews of treatments of postherpetic neuralgia, there is often a strong tendency for authors to quote studies to support their own subjective viewpoints, a paradox considering the scientific rigor required to develop an individual clinical trial, the unit of analysis in such reviews. One would not dream of conducting a clinical trial without first developing a detailed study protocol, so why should the developmental procedure be different for a review that attempts to summarize the evidence from RCTs?

Another problem with traditional reviews or textbook articles is that they often fail to help practitioners when there is apparently conflicting evidence from many small statistically underpowered trials.¹⁶ Statements such as "no conclusions can be drawn from the data on evening primrose oil"¹⁷ (for atopic dermatitis) are not helpful if sufficient RCTs are available that can be statistically combined to produce one overall answer. An example of the potential for harm resulting from failure to collate such evidence is the issue of RCTs of corticosteroids given to mothers to reduce fetal complications associated with premature birth, which began in 1972. Ten years later, 6 RCTs had been performed. Had a systematic review been available at the end of that 10-year period, it would have shown the clear benefit of corticosteroids for preventing the fetal complications. By 1991, reports of 7 more RCTs had become available. Because no systematic review of these trials had been published until 1989, most obstetricians had not realized that the treatment was so effective, and as a result, tens of thousands of premature babies probably suffered and died unnecessarily.¹⁸ Another example of the human costs of failure to perform systematic reviews is the use of intravenous streptokinase in the treatment of acute myocardial infarction. A cumulative meta-analysis showed clear evidence (very narrow confidence intervals for mortality reduction) of the benefit of such treatment by the 1970s, when more than 5000 people had participated in RCTs worldwide. Yet it took another 10 years and experimentation on another 30,000 patients in 15 RCTs for very little gain in the precision of the effect estimate before the procedure was widely adopted by clinicians.¹⁹

Not only does the medical profession fail to recommend effective treatments when there is overwhelming evidence to support the efficacy of the treatments, but it also continues to recommend the use of ineffective or possibly harmful treatments. For example, no clear benefit has been found for prophylactic lidocaine for the treatment of acute myocardial infarction. This lack of benefit is evident in the trial evidence from the late 1980s, yet textbooks continued to routinely or specifically recommend such treatment in the 1990s.²⁰

Unlike textbooks and journal article reviews, which quickly become outdated, the systematic reviews produced by the Collaboration are updated quarterly in the Cochrane Library.³ This type of "organic" publication places considerable responsibility on reviewers to maintain and update their reviews as more trial evidence becomes available.

The benefits of systematic reviews to dermatology may seem limited compared with areas such as stroke or coronary heart disease, in which there are many large RCTs and clear end points, such as death. While this idea may be partially true, it should not be a reason for complacency for 3 main reasons.

First, preliminary searches suggest that dermatology is no worse off in its yield of RCTs than many other branches of medicine.²¹ There are currently 3246 trials on the Cochrane Skin Group's Specialised Trials Register, of which about 80% are RCTs. Since it is impossible for even the most conscientious dermatologist to keep abreast of the 100 or so specialist dermatology journals, dermatologists should not assume, based on their own reading profiles or on the basis of a quick MEDLINE search, that trials are not being conducted or published. For example, at the early stages of a recent systematic review of treatments for molluscum contagiosum, the reviewers were confident that no relevant RCTs had been published. Yet, an exhaustive search identified 5 possible studies and 2 ongoing unpublished trials that provided useful data (R. MacSween, MRCP,written communication, May 1997).

Second is the role of systematic reviews of dermatological treatments for identifying gaps in knowledge that should become urgent research priorities. Sometimes a review based on an important question may not reveal any RCTs. This provides essential information for grant proposals for trials²² if a strong argument can be made for supporting such a trial, eg, widespread use of a treatment in the absence of evidence. Clinicians also may be reassured by such a "null" systematic review, which suggests that they are not missing some new evidence that should be used in their current practice and that they might be justified in continuing their own favorite therapeutic practices based on trial and error or anecdote until better evidence becomes available. It is only when we highlight our current collective ignorance of dermatological therapies that we can begin to ask the right questions and progress from the current state of clinical entropy that characterizes the treatment of so many dermatological diseases (Figure 1). Rarer dermatological diseases pose a particular problem, but judging from the success in organizing the results of RCTs into leukemia treatments, few diseases are not amenable to multicenter or international RCTs.

Related to the issue of identifying gaps in knowledge, is the role of individual reviewers in helping to frame the future research agenda for dermatology. If we confine systematic reviews to areas in which many RCTs are conducted, then the work of the group becomes driven by data rather than by questions. Since the clinical trial agenda of a small specialty such as dermatology is heavily governed by the priorities of the pharmaceutical industry, reviewers who limit their reviews to areas with substantial data risk collusion with such an agenda. For example, approximately half of the dermatologists in the United Kingdom use azathioprine to treat severe atopic dermatitis, yet we are not aware of one RCT to support its use.²³ This does not mean that azathioprine is not an effective treatment for this disease, since absence of evidence is not the same as evidence of no effect.²⁴ Yet, it is unlikely that manufacturers of this generically available drug would fund RCTs for a drug that is no longer under patent protection. Cyclosporine on the other hand, a relatively expensive and potentially toxic proprietary drug, has undergone at least 7 trials for the treatment of atopic dermatitis.^{25 - 31} A systematic review of the use of cyclosporine for atopic dermatitis may confirm its efficacy in this condition, but it will not address topics that may be of more interest to dermatologists and patients, such as the use of azathioprine, the role of diet in disease prevention, or the possible benefits of habit reversal.³² It is important, therefore, that the Cochrane Skin Group (herewith referred to as the Group) start with questions that are important to patients and that it does not limit itself to areas in which many trials are conducted.

The third main reason that dermatology can benefit from systematic reviews is that such reviews provide an excellent opportunity to identify and promote research into methodological problems that are common to dermatological diseases. Difficulties in measuring the outcomes of inflammatory dermatological diseases, such as acne, atopic dermatitis, and psoriasis, with reliable valid tools that measure aspects of the disease that are important to patients have been highlighted in previous attempts at meta-analyses.^{33 - 35} One study of 44 dermatology trials with "negative" conclusions found that all but one had a greater than 10% risk of missing a 25% relative treatment difference and that 70% were so small that they had a serious risk of missing a 50% relative treatment difference.¹⁶ The quality of drug trial reporting is another aspect that systematic reviews are able to highlight. For example, a study of 62 clinical trials of psoriasis listed in the Cumulated Index Medicus for the years 1988 and 1989 found that previous treatments (possibly an important determinant of a new treatment response) were not stated in 60 of those trials.³⁵ Another study of 62 dermatological clinical trials by Bigby et al³⁶ found that the method was reported only 41% of the time, ranging from 3% for power and method of randomization to 76% for loss to follow-up and use of control subjects. The European Dermato-Epidemiology Network (also known as EDEN) is conducting a systematic review of clinical trial methods for comparative trials of psoriasis treatments.³⁷ Such studies are useful for identifying methodological research priorities for these common diseases. It may be argued, for instance, that research to develop better internationally accepted outcome measures for dermatological diseases should be our No. 1 priority for dermatological research, rather than conducting many small trials using the profusion of outcome measures of unknown validity that are in use.³⁸

While dermatological systematic reviews may never achieve the same effect as similar reviews in cardiovascular medicine, the benefit of the work of the Group should be judged by what it can achieve, not by what it cannot achieve.

Interest in forming the Group dates back to 1992. During the following 4 years, interested persons from several countries held a number of informal exploratory meetings. In 1996, a formal exploratory meeting was held with potential collaborators with a range of interests. In 1997, registration with the Collaboration was pursued. In December 1997, the Collaboration approved the structures and the document identifying the scope of the Group, as well as how its work would relate to that of other collaborative review groups and the entire Collaboration, and formally registered the Group as a full Cochrane entity. Infrastructure support to allow the Group to work more effectively is provided by a grant from the Research and Development Programme of the United Kingdom National Health Service, London.

The Group has, from the very beginning like other Cochrane collaborative review groups, emphasized international and multidisciplinary perspectives. In addition, from the Group's inception, there has been strong and important involvement of consumers in the work of the Group. This underscores the fact that patients and consumers are partners with real and legitimate interests in the work of the Group.

The Group has an international board of editors (including a "comments and criticisms" editor), and protocols and completed reviews are subjected to peer review, including statistical review. The day-to-day functioning of the Group is administered by a full-time coordinator. The editorial base also appointed a part-time trials search coordinator late in July 1998.

The work of the Group is to produce the best evidence about the effects (good or harm) of health care interventions for dermatological diseases. The scope of the Group includes any dermatological problem that leads a person to seek help from a health care practitioner.³ The Group seeks to find and analyze all evidence on the effectiveness of preventive, medical, and surgical interventions and of different models of health care provision and management of dermatological diseases. This includes evidence about dermatological treatments that are sold over the counter or are widely available.

To produce the best evidence, all relevant data must be considered. This implies that the Group should use the full and complete set of records on all controlled trials, published and unpublished, available not only in the English-language health literature, but also in the literature in all other languages, and from all corners of the world. Searches of bibliographic databases (such as MEDLINE, EMBASE, SCISEARCH, and the Cochrane Library) will identify most but not all relevant studies. Many trials are not published in journals that are abstracted into these databases, and the records of many relevant studies are miscoded or misclassified in these databases. Therefore, database searches are supplemented with manual, page-by-page searching of the literature back to 1948 (the date of the first RCT of streptomycin for the treatment of tuberculosis), to identify all the relevant trials. This includes hand searching the "gray" literature, including journals that are not usually widely read or available, and records of proceedings of conferences and meeting abstracts.³⁹ Non–English language publications also are searched by hand and translated. The trials found by hand searching are assembled by the Collaboration into the Cochrane Controlled Clinical Trials Register to form the most complete register of health care trials available.

Moreover, many trials are unpublished; they may have small numbers of patients, may have produced negative results, or may have remained unpublished for some other reason.¹² The Group tries to identify these unpublished trials. In addition, the Group tries to identify trials that are ongoing for possible inclusion into reviews.

Reviewers are responsible for summarizing all of the available evidence and for adhering to strict methodological and peer review processes to produce quality reviews. They also update the reviews as new evidence (more trials) becomes available. To produce a review, the reviewers design protocols that prospectively identify relevant trials and include trials that meet inclusion criteria. They then extract the relevant data from each included study and enter the data into the Review Manager (RevMan) software.⁴ When appropriate, meta-analyses of the studies are performed.

The systematic reviews of the effects of interventions in dermatological health care are the most visible products of the Group. These reviews must be made readily available to health care professionals, managers, researchers, and consumers so that the output from the Group is available for advice and can influence the best professional practice and the direction of research. The Group is establishing an e-mail discussion list and plans to publish its protocols, reviews, and educational and training material on the World Wide Web. A list of current titles registered with the Group is listed in Table 1. As with any new Cochrane group, coverage of the 1000 or so dermatological disorders is limited, but it is envisaged that all dermatological trials will become part of a review and that all areas of dermatological care (including over-the-counter medicines) will be included in systematic reviews.

The Cochrane review is systematic, structured, and painstaking. It minimizes bias and ensures quality. When possible, Cochrane reviews are based only on RCTs because of the major biases associated with other study designs for assessing treatment effects.^{40 - 41} After approval according to the internal and external refereeing procedures of the Group, the review is published in the Cochrane Library.³ In addition, reviewers are required to maintain and update their reviews as new evidence becomes available and to respond to comments and criticisms of Cochrane Library users. Many journals have expressed enthusiasm for the copublication of Cochrane reviews, and the Collaboration has established a copublication policy with the editors of the Lancet and the British Medical Journal. This policy is generally adhered to by the editors of most of the specialty journals affiliated with these 2 journals and by the editors of the Canadian Medical Association Journal and the Annals of Internal Medicine. The International Committee of Medical Journal Editors (the Vancouver Group) also has agreed, in principle, to the copublication policy, and it is hoped that editors of the dermatological journals will follow this precedent. This means that reviewers may copublish their work in quality peer-reviewed journals, as well as in the Cochrane Library.

People can contribute to the work of the Group as hand searchers, translators, reviewers and coreviewers, disseminators and subeditors, content referees, consumers, and volunteers. These roles are summarized in Table 2.

Throughout this issue of the ARCHIVES, reference is made to evidence-based medicine as being the conscientious and judicious use of the current best evidence to make decisions about the care of individual patients.¹⁰ Thus, there is no such thing as evidence-based medicine without current best evidence. Mega-trials are unheard of in dermatology. The conclusions of the traditional alternative sources of summaries of evidence, such as narrative review articles and textbooks, cannot be trusted to be up-to-date, complete, or unbiased.^{14 - 15} Just as a clinical trial requires an explicit protocol, so does an article that aims to summarize the evidence surrounding a health care intervention. This is the rationale of the systematic reviews produced by the Collaboration. Earlier in 1998, the Lancet described the Collaboration as "the most important effort to gather evidence that might change practice."⁴² In scale and importance to modern medicine, the Cochrane Library has been likened to the human genome project.⁴³ Although the Cochrane Skin Group has just started, it aims to become the best source of current unbiased external evidence for summarizing the effects of dermatological interventions. The editors of the Group welcome inquiries from any persons who wish to contribute (Table 2) or learn more about summarizing the effects of health care in dermatology.

Accepted for publication July 24, 1998.

We thank all persons who have contributed to the Cochrane Skin Group.

Reprints: Hywel C. Williams, FRCP, PhD, Coordinating Editor, Cochrane Skin Group, Department of Dermatology, Queen's Medical Centre, Nottingham, NG7 2UH, England (e-mail: hywel.williams@nottingham.ac.uk).

For further information about the Cochrane Skin Group and the Cochrane Collaboration, write or send an e-mail message to the group's coordinator, Kayode Adetugbo, PhD, Dermato-Epidemiology Unit, Queen's Medical Centre, Nottingham, NG7 2UH, United Kingdom (e-mail: kayode.adetugbo@nottingham.ac.uk). Useful Web sites include the following:

Cochrane Skin Group: http://www.mailbase.ac.uk/docs/

Canadian Cochrane Centre: http://hiru.mcmaster.ca/cochrane

The Cochrane Library (information): http://www.cochrane.co.uk

To download the Cochrane Collaboration Handbook: http://www.medlib.com/cochranehandbook/