What Proportion of Dermatological Patients Receive Evidence-Based Treatment? FREE

DERMATOLOGY is considered to be a highly empirical field of medicine,and therapeutic interventions in dermatological patients are said to be basedsimply on "experience" in many instances.¹ However, we found only 1 report in the literature on the topic of the extentto which evidence-based medicine influences clinical decisions in the day-to-daypractice of a dermatology department.² Theauthors of that study, a retrospective review of a sample of case notes from115 outpatients, conclude that most treatments offered to the patients arebased on scientific evidence. However, they do consider a "primary level"² of evidence, derived from randomized controlled trials(accepted at "face value,"² without undertakingcritical appraisal), and also a "secondary level"² of evidence, derived "by transference"² betweendiseases with pathogenic or clinical similarities or by "careful well-documentedfollow-up studies."²

We therefore designed the present study to determine the extent to whichour own patients are offered evidence-based therapy, by applying more stringentcriteria on the evidence available, according to the well-known principlesand methods of evidence-based medicine, as already applied in the field ofgeneral medicine.³ We also aimed at highlightingareas of intervention where evidence is in fact not available and should be.

We included every adult patient (ie, aged ≥18 years) seen for thefirst time and treated at one of the clinics of Istituto Dermopatico dell'Immacolata(a dermatological reference center for central and southern Italy) in Apriland May 1999. Medical records were reviewed by a member of the clinical team(C.R.G.) and by an external observer (C.M.) who recorded the primary diagnosisand the primary intervention for each patient together with demographic informationand clinical history.

Primary diagnosis was defined as the condition for which the patientwas seeking dermatological help at that time. Primary intervention was thetreatment that represented the most important attempt to cure, alleviate,or care for the patient with respect to the primary diagnosis given, and itcould be a combination of more than 1 drug or treatment modality. Other adjunctivetreatment (eg, supportive) that the team caring for the patient considereda nonessential part of the primary intervention was indicated as "associated"therapy.

For each primary diagnosis–primary intervention combination, wesearched evidence in electronic databases (MEDLINE, NHS Center for Reviewand Dissemination, Cochrane Library, and the Controlled Trials Registry) fromJanuary 1966 to December 1999.

An original search strategy was developed by modifying the one proposedby default in PubMed⁴ and presently consideredthe standard bibliographic search strategy for clinicians looking to findthe best evidence in caring for their patients.⁵ We developed 23 different strategies and tested them on 3 diagnosis-treatmentcombinations of dermatological relevance. We then chose the most sensitiveand specific for identifying relevant trials, systematic reviews, and meta-analyses(Figure 1). When we were unableto identify any trials with this search strategy, "wider" queries (ie, moresensitive) were tried to find out whether the more specific one had missedrelevant work. Traditional reviews, overviews, and the American Academy ofDermatology guidelines of care were reference searched for relevant trialsbut were not considered as sources of evidence per se.

The articles were independently evaluated by 2 of us (an epidemiologist[D.A.] and a dermatologist [C.M.]) and considered relevant if they dealt withliving human beings, were prospective, and compared 2 or more interventions(one of which could be a placebo or a treatment of proven efficacy). Articlesin English, Italian, French, German, and Spanish languages were assessed.

Trials dealing with a combination of diseases that were analyzed asa whole were excluded, and trials considering nonclinical outcomes or involvinghealthy individuals in whom the disease was experimentally induced (eg, contactdermatitis) were considered and critically appraised only if no other sourceof evidence could be found.

The quality of each trial was ranked by applying a modified versionof Sackett's criteria for clinical evidence⁶ :(1) type A trials, large, randomized, double-blind, placebo-controlled studies;(2) type B trials, also randomized, double-blind, placebo-controlled studies,but they include a small number of patients, yielding an imprecise point estimateof treatment effects and increasing the likelihood of high false-positiveand/or false-negative errors; (3) type C trials, lack 1 or more of the above-mentionedcriteria (or they do not include precise information on study design) or includefewer than 20 patients in each treatment group. When more than 1 type A orB trial was available as supporting evidence, type C trials were not considered.The quality of the trials and their clinical significance were further appraisedby applying the checklist proposed by Downs and Black⁷ that contains questions on reporting, external validity, internal validity,and power of the study to detect a clinically important effect.

Evidence-based reviews and meta-analyses were also included as sourcesof evidence while case reports and case series were not. Disagreements wereresolved by discussion.

On the basis of the evidence found, every primary intervention was classifiedas follows: a, intervention with evidence from randomizedcontrolled trials (type A or type B study); b, interventionwith evidence from type C studies only, as described in the paragraph above; c, intervention based on convincing nonexperimental evidence,defined as interventions whose validity is so evident that randomized trialswere unanimously judged unnecessary or, if a placebo would have been involved,unethical; and d, intervention in common use butnot meeting the criteria defined for either experimental or convincing nonexperimentalevidence.

As already reported in previous work,³ we chose to select patients, and not therapeutic interventions, as the denominatorfor our proportions to be estimated. The proportion of patients who were offeredan intervention and were classified as a, b, or c was calculated as the main outcomemeasure.

During the study period, 152 new patients were cared for. No primarydiagnosis was made for 3, and another 13 patients were not given any specifictherapy (just examinations and reassurance), leaving a study sample of 136participants. After analysis of the medical records, 61 different diagnosis-treatmentcombinations were generated.

When "primary therapies" included more than 1 drug or treatment modality,queries on electronic databases were generated both for the single componentsof the treatment and for the association of them. As a result, we performed94 queries on electronic databases of biomedical publications and identifieda total of 1832 articles. Of these, 329 were selected and evaluated. The other1503 were excluded because they did not fulfill 1 or more of the criteriadescribed in the "Evaluation of Evidence" subsection of the "Patients, Materials,and Methods" section, or because it was impossible for us to properly evaluatethem (eg, language unknown to any of the authors and coworkers and no abstractavailable in English with sufficient information) or to locate them (eg, veryold issues of journals untraceable in libraries in Italy).

The results of the study are summarized in Table 1. Eighty-seven patients (64%) were judged by our criteriato have received evidence-based interventions. Approximately half of the patients(69/136) received interventions previously shown to do more good than harmin 1 or more randomized controlled trials. Another 14 patients (10.3%) receivedinterventions for which evidence could be found only from type C trials, thereforenot allowing us to reach definite conclusions on the value of the treatmentunder analysis. However, 24 of the participants who received evidence-basedtherapy also received additional non–evidence-based treatment (mainlysupportive and symptomatic measures and antibiotics) considered to be partof the primary therapy by the clinical team. Forty-nine patients (36% of thesample) received interventions without substantial experimental evidence.

Table 2 and Table 3 provide the diagnosis-treatment combinations, a summaryof the grade of evidence that could be traced for each of them, and the numberof patients treated accordingly. A selection of the references of the morerelevant trials, or other evidence-based articles, is given in Table 2.

As already documented by Ellis and coworkers³ for general medicine, and despite more pessimistic expectations based on commonbeliefs, it was found that in dermatological practice as well most of thepatients were offered evidence-based interventions, at least in the settingof a referral facility with special commitment to research. The proportionof evidence-based health care, as reviewed on a useful website³⁸ is quite variable depending on the different specialties and study designs.Most of the studies reported, including the one by Jemec and coworkers,² consider the therapeutic interventions as the denominatorof the proportions. Using the patients rather than the interventions as thedenominator for our proportions was particularly useful from a practical pointof view as treatments that are rarely used and less investigated did not receivethe same weight as common ones, which involve the majority of patients andhave had good-quality trials.

Our observed proportions should probably be considered as minimum estimatesof the proportion of patients receiving evidence-based treatment in our study.In fact, we used only search tools and information sources available to cliniciansin our institute. Had we had person-time to devote to a systematic hand searchof the relevant literature and had we obtained funding to subscribe to expensiveelectronic databases or to pay for translations for languages we do not know(eg, Japanese, Russian, Norwegian), the estimated proportion could have beenhigher. Systematic reviews are needed to narrow the gap between the amountof information that may be retrieved by clinicians during their practice andall the information that is potentially available. In fact, the diagnosis-therapycombinations for which we were able to find evidence-based reviews or meta-analyseswere only 8, involving 21 patients, or 15.4% of the study sample. Evidence-basedreviews are an invaluable tool to help the dermatologist who needs to be up-to-dateon evidence without spending whole days in a library appraising hundreds ofgood and, often, bad articles. The effort of the Cochrane Skin Group³⁹ and others in producing such publications is of utmostimportance for clinical practice.

Our estimate of the proportion of patients receiving evidence-basedtherapy may have been affected also by the referral pattern to our instituteand the case mix generated by choosing a single clinic (this choice contributedto the exclusion of patients referred to other specific clinics operatingin our institute, eg, pigmented lesions), a short period (with differentialselection of "seasonal" diseases), and, more importantly, patients seen forthe first time. At the end of the study we realized that this last choiceled to the preferential selection of diseases with high incidence and shortduration. In fact, patients with chronic conditions (eg, psoriasis), and particularlythose with low-incidence diseases (eg, systemic sclerosis), although seenin the clinic tended to be excluded because of previous visits. More comprehensivestudies are needed to better define the level of evidence-based interventionswhen considering different groups of diseases and different settings of carealthough in the same field.

As a cautionary note, it is important to point out that even good trials,conducted for high-prevalence diseases, such as psoriasis and atopic dermatitis,often failed in defining clinically significant outcomes, as is the case fortopical steroids where efficacy was almost always assessed in a period ofa few weeks with no information on further follow-up and relapses.⁴⁰

On the other hand, given the high cost of producing experimental evidence,low-profile, low-technology interventions, though considered effective incommon experience (eg, eosin in eczema, or sulfur ointment in scabies), areusually not tested by means of controlled trials because conducting trialson them is probably judged to be worthless while many good-quality trialshave been conducted for high-cost, high–marketing interest interventions(eg, ketoconazole for seborrheic dermatitis). This is in accordance with thefact that the majority of good trials are funded by pharmaceutical industriesor directly conducted by their research teams (eg, 13 of 22, or 59%, in ourlist of references).

Only four patients (2.9%) were judged to be offered self-evident treatmentsbecause in the field of dermatology there are really few conditions for whichconducting a placebo-controlled trial should be considered unethical. Besides,there have been repeated demonstrations that treatments previously thoughtto be "self-evidently" effective were in fact useless or even harmful. Onthe other hand, many of the symptomatic and supportive measures we includedin Table 3 could probably be consideredself-evidently effective, at least when considering the goal of simply relievingsymptoms (eg, emollients for conditions with dry skin, or topical corticosteroidsfor acute inflammatory skin reactions).

Symptomatic and supportive measures account for most of the interventionsincluded in Table 3. However,also well-accepted and important treatment regimens for conditions such asdiscoid lupus erythematosus,⁴¹ psoriatic arthritis,or severe drug eruptions⁴² had to be includedamong treatments lacking substantial evidence of their effectiveness at thetime patients were visited. In many instances, only case reports and caseseries have been produced, particularly for rare conditions, for which trialsneed to be multicentric in design if reliable evidence is to be attained.

Even in the cases in which good experimental evidence is produced, asecond line of problems arises when trying to find the evidence, criticallyappraise it, and get it into practice. Electronic databases are not alwaysavailable, are sometimes too expensive, and are not as easy to search as thought.We compared the results of search strategies developed by one of us (C.M.)(dubbed the "naive dermatologist" strategies) with those obtained by an experiencedlibrarian (R.A.), and observed that the naive dermatologist found many uselessarticles and missed some important ones.

In addition, when articles are identified, their validity must be carefullyassessed, as many so-called trials are produced with flawed designs and unreliableresults or unwarranted conclusions. Case reports and case series are particularlyat risk for drawing conflicting conclusions and for showing "effectiveness"of treatments that is purely due to well-known phenomena such as placebo effect,selection bias, and regression toward the mean.

As noted, well-designed and well-conducted trials also may fail in definingclinically relevant outcomes (such as trials with a few weeks' follow-up inlife-long diseases) and generally have problems with external validity (ie,the results are true for the small and highly selected group of patients onwhich they were conducted but may not be applicable to most of the patientswith that disease).

Finally, bringing research findings into clinical practice requiresconstant and rigorous updating of the physicians who tend to remain adherentto traditional interventions (eg, rifampin for acne) even if better optionshave become available, or, on the contrary, may be induced by commercial pressureto apply new treatment regimens still lacking substantial evidence on theirvalidity.

A cooperative effort of the Clinical Epidemiology Unit of the IstitutoDermopatico dell' Immacolata–Istituto di Ricovero e Cura a Carattere Scientifico(IDI-IRCCS) and the Archives of Dermatology

Accepted for publication March 28, 2001.

This work was partially funded by the "Progetto Ricerca Finalizzata1998" of the Italian Ministry of Health, Rome, Italy.

Presented in part at the 58th Annual Meeting of the American Academyof Dermatology, San Francisco, Calif, March 10-15, 2000.

The authors thank the following dermatologists for their help in thepreparation of medical records: G. Cianchini, MD, S. Pallotta, MD, L. Colonna,MD, G. Di Lella, MD, M. Giani, MD, E. Scala, MD, L. Pirrotta, MD, E. C. Guerra,MD, G. Monticone, MD, and C. Barbieri, MD; and V. Salvatori who collectedthe papers to be reviewed.

Corresponding author: Damiano Abeni, Clinical Epidemiology Unit,Istituto Dermopatico dell'Immacolata, Via Monti di Creta 104, 00167 Rome,Italy (e-mail: d.abeni@idi.it).