Teledermatologic Consultation and Reduction in Referrals to Dermatologists:  A Cluster Randomized Controlled Trial FREE

In Western countries, health services face increasing pressures because of the aging population and patients' increasing demands for evaluation by specialists. The demand results in increased referral to specialists and longer waiting times for appointments. This has led to a debate regarding the appropriateness of referrals and what treatment general practitioners (GPs) should be able to provide before referral.^1- 2 Improved communication between GPs and specialists leads to lower referral rates.^3- 4 For example, Vierhout et al⁴ showed a significant decrease in referrals with joint consultations, but these consultations require GPs and specialists to be present in the same room at the same time.

The Internet offers an alternative way to communicate at a distance. In 2002, Wallace et al⁵ investigated virtual joint consultations in 8 different specialties using videoconferencing. This “real-time” telemedicine still required GPs and specialists to be available at the same time, although no longer in the same place. Telemedicine using videoconferencing allows valuable direct interaction, but asynchronous telemedicine using e-mail or Web sites—the “store-and-forward” variant—is cheaper and much more flexible in both time and place.⁶ Dermatology is a popular specialty for telemedicine. This is partly because of its visual nature but also because skin diseases are a frequent reason for visits to GPs and referral to dermatologists from GPs, which causes long waiting times for outpatient dermatologic consultations. To date, most analyses of teledermatologic consultation have been feasibility studies, and few provide sound evidence on clinical effects, such as referral rates.^7- 8 The aim of our cluster randomized trial (PERFECT_D [Primary care Electronic Referrals: Focus on Efficient Consultation using Telemedicine in Dermatology]¹) was to determine the extent to which teledermatologic consultation can decrease outpatient referrals by GPs.

The PERFECT_D study was performed in the catchment areas of 2 Dutch district general hospitals in Almere (February 2004 to April 2005) and Zeist (February 2005 to January 2006) with approximately 53 000 and 98 000 new outpatient visits per year, respectively. Five dermatologists in these hospitals (Almere, 3 and Zeist, 2) were board certified, had at least 3 years of experience in dermatology, and had an Internet connection so were therefore eligible for inclusion. After obtaining consent from both hospitals and all 5 eligible dermatologists, we recruited participants from the general practices that referred patients to these dermatologists. Eligible practices were required to have facilities to send digital images over the Internet. Familiarity with a digital camera and/or the Internet was not required for GPs or dermatologists. The GPs who were current users of teledermatologic consultation or who had used it more than 4 times in the previous 12 months were excluded. Patients were eligible if they were referred by their GP to one of the recruited dermatologists and did not require a dermatologic consultation within 2 days. The ethics committee deemed this study to be exempt from review because the research did not interfere with usual care.

We randomized GP practices rather than individual patients to prevent contamination. In this clustered design, all GPs at any single practice used either teledermatologic consultation or standard procedures (control group). Using dedicated randomization software, practices were assigned to teledermatologic consultation or standard care. For each of 3 rounds of randomization (1 in Almere and 2 in Zeist), codes were assigned to practices. A special allocation concealment procedure (eBox) was followed to ensure that no allocation bias could occur.^9- 10

In a pretrial pilot program conducted in another region, use of the camera and study Web site, the training program, and trial logistics were tested and improved. The improved training program for all intervention GPs included instructions on taking digital images, downloading images to the computer, managing files, and using the Web site. In standardized sessions, dermatologists were taught how to use the Web site and complete the study forms.

For every eligible patient, intervention GPs took 4 digital images of the skin problem (2 closeups and 2 overviews) and attached these to a semistructured form, which they completed on a secure Web site. We used the KSYOS TDCS teledermatology Web site (www.ksyos.org), which was adapted for the study (for study purposes, only anonymous patient data were used and at the end of the consultation, participants completed evaluation forms) and Kodak EasyShare CX6230 digital cameras (2.0 megapixel with macro facility; Eastman Kodak Company, Rochester, New York). The Web form required answers to questions about the duration and location of the skin condition and the reason for referral. Entry of the presumed diagnosis was optional because ignorance of the diagnosis is a valid reason for referral. The GPs selected the main reason for referral: establishing the diagnosis, treatment advice, or reassurance.¹¹ In addition, GPs were able to refer patients to any dermatologist in their region. The dermatologist to whom a patient was assigned received a notification e-mail message, viewed the images and other data, and responded using the same secure Web site. The dermatologist advised the GP about further procedures (ie, treatment advice, further investigations needed, and standard or urgent referral). The dermatologist's feedback was provided within 48 hours, at which time the GP received a notification e-mail message. The GPs were able to respond once to the dermatologist with additional questions or information before proceeding to act on the advice.

In control practices, all referred patients were seen by a dermatologist according to the usual procedures. In most cases this involved patients visiting the outpatient clinic with a letter in which the GP described findings pertinent to the case.

All patients, intervention or control group, were required to visit a dermatologist, according to the regular waiting time of approximately 1 month, irrespective of their degree of recovery. Intervention group patients visited the same dermatologist who had performed the teledermatologic consultation (Figure 1).

The primary outcome measure was the proportion of and reasons for preventable consultations, as judged by the dermatologists. Using teledermatologic advice, GPs were able to advise or treat the patient or perform additional tests, eg, for allergies, as suggested by the dermatologist. If the GP's treatment was successful and the patient recovered or was recovering, the dermatologic consultation in the office was considered preventable. Some control patients also had recovered before the outpatient dermatologic consultation. Immediately after the consultation in the office for control and intervention patients, the dermatologist recorded on a detailed form the reasons why a consultation might have been prevented (Table 1). Because the proportion of preventable consultations might be influenced by certain patient characteristics, dermatologist characteristics, or disease groups, these data were also captured. The disease groups were based on the diagnoses reported in the correspondence from the dermatologist to the GP. An anonymous copy of this correspondence carrying the patient's study code was received by trial administrators.

The main secondary measure was patient satisfaction in general and about interpersonal aspects of the consultation. To assess satisfaction, patients completed a shortened version of the Patient Satisfaction Questionnaire III developed by Ware et al (20 of 43 items)^12- 13 immediately after the dermatologic office consultation. This questionnaire was translated and validated in the Dutch language by Hagedoorn et al.¹³

In consultation with GPs and dermatologists, we defined an absolute difference of 10% in the preventability of dermatologist consultations as worth detecting. Assuming a control group rate of 35% preventable visits,^14- 17 α of 5%, power of 80%, and an intracluster correlation coefficient of 1%, we calculated that we needed to include 50 GPs and 1000 patients.

The proportions of preventable dermatologist office consultations in both trial arms were compared using logistic regression analysis. To account for potential correlation between outcomes from the same GP or the same practice, we used generalized estimation equations with robust variance estimators.¹⁸ All statistical analyses were conducted with S-PLUS Professional statistical software, version 6.2 (Insightful Corporation, Seattle, Washington).

Three situations could lead to nonrecording of the primary outcome, ie, whether the dermatologist consultation was judged necessary, post hoc: the patient never visited the dermatologist after referral, possibly because she or he had recovered; the patient visited the dermatologist but by oversight no study form was completed; or, by mistake, the patient visited a dermatologist in a nonparticipating hospital. During the trial, various measures were taken to avoid these situations, such as sending reminders about the study to dermatologists and patients. To assess the potential effect of missing data, 4 additional sensitivity analyses were carried out after the primary statistical analysis was completed. First, for patients who visited the dermatologist but whose outcome form was missing, the dermatologist was asked to complete the form later using information from patient records, GP correspondence, or referral or teledermatologic data. The original statistical analysis was then repeated, taking the additional, newly collected outcome data into account. Second, for patients who never visited the dermatologist, we asked all GPs about any GP or hospital visits at least 6 months after inclusion and, if there were none, whether they thought the patient had recovered. Again, the statistical analysis was repeated with this extension of the original data. Third, the analysis was repeated taking account of all the data that were used in both of these analyses. Fourth, we repeated the analysis with the original data (complete cases) using inverse probability weighting to correct for missing outcomes.¹⁹ The weights were determined by a mixed-effects logistic regression analysis of missing outcomes, using trial group, GP practice size, patient age, advice of the teledermatologist, and referral reason as fixed effects and GP and GP practice as nested random effects. For the inverse probability weighting analysis, a cluster-level weighted t test, as described by Donner and Klar,²⁰ was used instead of generalized estimating equations because the latter procedure does not allow discounted weights at the level of individual patients.

Of 56 GP practices eligible for participation, 36 (53%), including 110 GPs, agreed to participate and were randomized (Figure 2). Twenty-nine practices refused to participate because they were too busy with other studies; 2 had no Internet connection, and 1 was already using teledermatologic consultation. From the practices that did participate, 85 GPs enrolled 631 patients (46 intervention GPs, 327 patients; 39 control GPs, 304 patients). One control group GP (3 patients) was excluded because we received neither referral letters nor signed consent forms, which meant 84 GPs were included in the analysis. In addition, 26 patients were excluded, mainly because the data were not fully entered in the Web site (n = 9) or the patient visited a nonparticipating dermatologist (n = 7). This left 605 patients (95.9%) for analysis. Table 2 shows the characteristics of the 3 cluster levels: practices, GPs, and patients (see also Figure 2).

For 236 (39.0%) of 605 patients, information on the main outcome variable was missing. Sixty-six patients (intervention, 48; control, 18) did not visit a dermatologist or their GP for treatment of the dermatologic condition after inclusion in the study; 150 patients visited the dermatologist but the study form was not completed or was incomplete (n = 12), and for 20 patients we and the GP lacked information necessary to trace the reason why the form was missing.

Among the remaining 369 patients, dermatologists considered a live consultation preventable for 78 of 200 teledermatologic consultation group patients (39.0%) and 31 of 169 control group patients (18.3%), yielding a difference of 20.7% (95% confidence interval, 8.5%-32.9%). Because intracluster correlation was strongest at the level of the GPs (GPs, 0.080; practices, 0.071), the generalized estimation equation analysis was conducted using GP as the grouping variable.

The complete main outcome form was missing for 138 of 605 patients (22.8%) who visited a dermatologist. After reviewing these cases, dermatologists were able to answer the main outcome measure question for 77 of 138 patients (55.8%). In Zeist, the GPs indicated that 21 of 34 patients who did not visit a dermatologist had recovered. The remaining 13 patients' status was unknown. In the sensitivity analysis, all 34 cases were considered to be preventable consultations.

The mixed-effects logistic regression model that was used in the inverse probability weighting analysis to predict the probability of missing outcomes had an area under the receiver operating characteristic curve of 0.794. Predictions made by the model ranged from 0.204 (yielding a weight of 4.90) to 0.943 (yielding a weight of 1.06).

The estimated difference in proportion of preventable consultations ranges from 19.0% to 25.5% in the 4 additional analyses (Table 1), and each of them reached the same conclusion as the original analysis regarding the effect of teledermatologic consultation.

For each preventable consultation, dermatologists recorded the reason why they considered the consultation to be preventable (Table 3). Full or partial recovery before the consultation was the most common reason consultations were deemed preventable (intervention group, 40 [51%] of 78; control group, 7 [23%] of 31). Another frequent reason was that the dermatologist judged that GPs might have treated the patient themselves (intervention group, 30 [39%]; control group, 21 [68%]). For nonpreventable consultations (n = 260), an office visit was usually required because the intervention could only be performed by a dermatologist (intervention group, 87 [71.3%] of 122; control group, 93 [67.4%] of 138). Patient request was another reason for a nonpreventable consultation. This occurred for 3.3% of teledermatologic consultation patients and 11.6% of control patients.

In total, 20.0% of 200 teledermatologic consultation patients were described as recovered or partially recovered compared with 4.1% of 169 in the control group (Table 3). The difference in the overall percentage of GP treatable cases in the teledermatologic consultation (15.0%) and control groups (12.4%) was less than the percentage of preventable cases in both groups.

The most frequent diagnosis was benign skin tumors (eg, nevi), and 16.4% of these consultations were considered preventable (Table 4). Eczema was frequent in both trial arms but was more often associated with a preventable consultation in the teledermatologic consultation (51.7%) than in the control group (27.3%). Different preventable consultation rates between the trial arms were also found for infections (intervention group, 61.1%; control group, 18.2%) and psoriasis (intervention group, 45.5%; none in the control group). Overall, psoriasis and malignant tumors occurred more often in the teledermatologic group, whereas premalignant tumors occurred more often in the control group.

A patient satisfaction questionnaire was completed by 350 patients (57.8%), of whom 191 received teledermatologic consultation (54.6%). The mean (SD) scores regarding interpersonal aspects were similar for the teledermatologic (4.13 [0.62]) and control groups (4.15 [0.73]). The mean scores for general satisfaction were the same for both groups (3.8 [0.59]). At the question level, there were no significant differences between the 2 groups.

Across all diagnoses, the availability of teledermatologic consultation in general practice resulted in about one-fifth of all referrals for a dermatologic consultation being judged preventable. The main reason that consultations were judged preventable was because the patient recovered before the outpatient appointment. Assuming that patients who did not visit a dermatologist or a GP after the teledermatologic consultation had recovered or at least partially recovered, nearly 50% of teledermatologic consultation patients improved or recovered compared with 15% in the control group. Another common reason that dermatologists judged consultations preventable was their belief that the GPs could have helped the patients. This applied more often to the control vs the teledermatologic consultation groups, but we found it applied to almost the same proportion of patients in both overall groups. This might indicate that, even when dermatologists had the opportunity through teledermatologic consultation to advise a GP, they still believed that the GP could have provided more treatment. Potential explanations are faulty perceptions by dermatologists about the dermatologic knowledge, tools, and skills of GPs and GPs lack of ability in or confidence to treat dermatologic problems.

The most frequent diagnosis, benign skin tumors, appeared unsuitable for standard teledermatologic consultation. Informal discussions with dermatologists in the pilot phase suggested that dermatologists preferred to see these patients in person to exclude malignant neoplasms. In the future, the new techniques of teledermoscopy and macrophotography might enable a dermatologist to advise a GP how to care for these patients confidently.^21- 22 In our study, we included all dermatologic patients without focusing on a certain disease group. It is likely that teledermatologic consultation will have a different effect on preventable consultations when applied to specific patient groups, such as nursing home residents with ulcers.²³

We found suggestive evidence that teledermatologic consultation is useful in reassuring patients. The proportion of patients for whom a consultation was nonpreventable because of patient request was much higher in the control group than in the teledermatologic consultation group. Future studies should explore this possible advantage of teledermatology and perhaps telemedicine in general because it might offer a solution for dealing with an increasing number of demanding patients. General patient satisfaction and satisfaction regarding interpersonal aspects were the same for the intervention and control groups. Using the same questionnaire, Wallace et al⁵ found a significant difference between the intervention and control groups, whereas Bowns et al²⁴ found no difference. Although not specifically measured in our study, GPs in the teledermatologic consultation group reported high levels of patient enthusiasm when being included in the study. It was surprising that this enthusiasm was not captured by the Patient Satisfaction Questionnaire even when a significant proportion of teledermatologic consultation group patients recovered. Another study that evaluated a special-interest GP dermatology service found no difference in patient satisfaction even though intervention patients recovered earlier than control patients.²⁵ It seems that earlier recovery is not the most relevant issue for dermatology patients.

To our knowledge, this is the first cluster randomized controlled trial of telemedicine. Such a study design requires a large number of clusters, in our case GP practices and GPs, which makes it difficult to retain adequate recruitment numbers and coordinate the study. However, we believe that cluster randomization was the most appropriate design because substantial intracluster correlation (0.071) was found in the primary outcome variable, preventability of dermatologic consultation, at the practice level. Because the intracluster correlation at the GP level was slightly higher (0.080), GPs were used as the clustering unit in the statistical analysis. The sample size calculation showed that we needed 50 GPs and 1000 patients. We estimated that we needed to include 25 to 30 patients per GP, which was feasible according to the national registry of referrals to dermatologists. During the trial, GPs included far fewer patients than we estimated; however, the number of GPs, in our case the number of clusters, reached a higher number than expected: 85 instead of the initially planned 50 GPs. The study had sufficient power even with fewer patients.

One could argue whether the randomization succeeded and whether the case mix of the 2 groups is comparable. We found that patient demographic characteristics were similar between groups, but there were some differences in diagnosis, which might cause confounding. Owing to our randomization procedure, we minimized selection bias before randomization. However, it is likely that some GPs were selective when inviting patients to participate in the study, despite instructions to invite all eligible patients. Our study design was pragmatic because this selection process for GPs reflects the selection that will occur when teledermatologic consultation is implemented in real practice.

Introducing teledermatologic consultation into practices required GPs to use computers more than usual. The new technology and high number of different participants, including GPs, dermatologists, and patients, resulted in a number of missing outcome forms. Other large telemedicine randomized controlled trials have experienced similar problems.^5,21 The results of standard statistical analyses restricted to subjects with complete data can be biased in the presence of missing outcome data.¹⁹ However, using several additional analyses with outcome data collected afterward, and one principled method for dealing with missing outcome data, we were able to show that our results are robust across a wide range of reasonable assumptions.

Whited et al¹⁶ found that teledermatologic consultations prevented 18.5% of office visits compared with a control group. Another randomized trial demonstrated that 54% of teledermatologic consultations could be managed within the GP practice, but this proportion was similar in the control group (55%).²⁶Our much larger and more rigorous study demonstrated that 20.7% of initial dermatologic office visits could have been prevented by the use of teledermatologic consultations. The proportion of consultations judged preventable by a dermatologist is often an outcome measure in telemedicine studies.^14,16 However, in reality it is the GP who decides to refer a patient for a consultation with a dermatologist, and there are probably differences in how GPs make this decision. Using teledermatologic consultation in a practice with a special interest in dermatology, such as in the United Kingdom,²⁵ might result in a higher proportion of preventable office visits. Teledermatologic consultation successfully enables GPs to treat patients they would otherwise refer to a dermatologist. Further research conducted with more specific patient groups as well as about patient satisfaction should be encouraged.

Correspondence: Nina Eminović, PhD, Department of Medical Informatics, Academic Medical Centre, PO Box 22660, 1100 DD Amsterdam, the Netherlands (n.eminovic@gmail.com).

Accepted for Publication: September 29, 2008.

AuthorContributions: Drs Eminović, ter Riet, and Peek had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Eminović, Wyatt, ter Riet, van Weert, Bruijnzeel-Koomen, and Bindels. Acquisition of data: Eminović and de Keizer. Analysis and interpretation of data: Eminović, ter Riet, and Peek. Drafting of the manuscript: Eminović and de Keizer. Critical revision of the manuscript for important intellectual content: Wyatt, ter Riet, Peek, van Weert, Bruijnzeel-Koomen, and Bindels. Statistical analysis: Eminović, Peek, and ter Riet. Obtained funding: Eminović, de Keizer, Wyatt, and Bindels. Administrative, technical, or material support: Eminović. Study supervision: de Keizer, Wyatt, and Bindels.

Financial Disclosure: None reported.

Funding/Support: This study was supported in part by Senter Novem (Agency of the Dutch Ministry of Economic Affairs) and ZonMw (Dutch Organization for Health Research and Development). KSYOS Health Management Research provided the digital cameras and teleconsultation software.

Role of the Sponsors: The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of data; or in the preparation, review, or approval of the manuscript.

Additional Contributions: The GPs from Uithoorn and dermatologist Marc Nahuys, MD, participated in the pilot study, which helped us to prepare and test our trial. We also thank the 5 study dermatologists (Almere: Jimmy Zeegelaar, PhD, MD; Irina Cairo, MD; and Roel Chin-a-Lien, MD; Zeist: Diederik de Geer, MD, and Margot Tegelberg-Stassen, MD) for their active role in this trial; Astrid Prins, MSc, and Sven Pippel, MSc, for their contribution to the logistics of the study; and Leonard Witkamp, PhD, MD, for the original idea, fundraising, and initiation of this study.