Impact of an Intrainstitutional Teledermatology Service: Mixed-Methods Case Study

Background: Teledermatology provides timely access to consultative dermatology services while reducing the need for travel among patients in rural and underserviced areas. However, knowledge about the potential benefits of such a service in urban areas is limited. Objective: This study aimed to determine the impact of a geographically unrestricted, intrainstitutional, secure, email teledermatology service for dermatology. Methods: We employed a mixed-methods approach using chart review, surveys, and semistructured interviews from the Canada Health Infoway Benefits Evaluation Framework. Patient charts were reviewed for demographics, clinical characteristics, and outcomes. Electronic and paper surveys were sent to patients and providers to quantify aspects of the service, such as satisfaction and usability, on a Likert scale. Semistructured interviews of referring providers and a convenience sample of academic consultant dermatologists who were considering teledermatology for their practice were conducted. Interviews were transcribed and analyzed using manual coding and thematic analysis by both the primary author and a second independent reviewer. All results were concurrently triangulated in an overarching analysis. Results: A total of 76 consultations were reviewed over a period of 18 months, of which 84% were completely managed without an in-person visit. Only 6% of rashes required a subsequent in-person visit to a dermatologist for management, compared to 41% of lesions. In addition, 28% (21/76) of patients responded to the survey. Patients “strongly agreed” to use the service again, were satisfied with the management of their skin issue, and thought the service saved them time. In general, providers who answered the electronic survey “strongly agreed” that the service demonstrated quality, timeliness, and an educational benefit, but increased their administrative time. A total of 9 interviews of 5 referring providers and 4 dermatologists were completed. Triangulation of all study components supported the hypothesis that teledermatology benefits providers, patients, and the health care system. Conclusions: Intrainstitutional teledermatology has high satisfaction among patients and providers and saves patients time, even when there are no geographic or systemic barriers to access. This service may be most effective when targeted at rashes rather than lesions. Additional research on the cost-effectiveness and educational benefits of this service is warranted. (JMIR Dermatol 2018;1(2):e11923) doi: 10.2196/11923


Introduction
Teledermatology allows dermatologists to manage patients with generally equivalent clinical outcomes and requires less resources and less time with the consultant than in-person visits [1]. Its impact is evident in geographic areas that do not have timely access to dermatology, which is the most-studied aspect of teledermatology; for example, in the United States, 80% of dermatologists are clustered in 3 large metropolitan areas [2]. There is a similar disproportionate clustering of dermatologists in metropolitan areas in Canada [3]. These substantial disparities in the geographic distribution of dermatologists have been increasing [4].
Knowledge of the potential of intrainstitutional teledermatology, wherein geographic access is not a factor of the net benefit assessment of the value of teledermatology, is limited. Other pilot studies have assessed the feasibility and diagnostic concordance of these implementations [5], but formal assessments of their impact are lacking.
We hypothesize that teledermatology will have value beyond geographic access if implemented by the consultants and referring physicians intrainstitutionally. We designed this study to investigate the potential benefits, barriers, and impact of teledermatology in such an intrainstitutional setting, where the traditional geographic barriers to access do not exist. We further hoped to identify specific characteristics of the teledermatology service that would be useful for implementing the service at other institutions. Studies have reported the usefulness of teledermatology in only a few specific scenarios, and there is disagreement about the clinical conditions that are suitable for teledermatology in the literature; for example, several studies have reported the potential of teledermatology for identifying any suspicious skin lesions [6,7]; in contrast, the American Telemedicine Association guidelines for teledermatology [8] state that it is difficult to assess pigmented lesions and special sites such as the scalp with teledermatology.

Overview
We performed a mixed-methods evaluation using broad categories of the Canada Health Infoway (CHI) Benefits Evaluation Framework [9] to include at least one aspect of qualitative and quantitative analyses for each category as well as case studies, surveys, and semistructured interviews. The key results were triangulated in a final, overarching analysis. This study was approved by the research ethics boards of Women's College Hospital and the University of Toronto in Toronto, Ontario, Canada.

Design of the Intrainstitutional Teledermatology Service
Parameters were chosen to balance convenience (encourage use) and pragmatic considerations of the practice guidelines for teledermatology [8]. We selected a store-and-forward design with secondary teledermatology, as it is a funded model in our region, and secure institutional email as a common, accessible technology. Clinically, the referring providers were informed that they could send any request they deemed appropriate and a response would be guaranteed within 1 week. The primary author was the sole consultant providing responses. The details of the implementation are described in Table 1. Table 1. Components of the intrainstitutional teledermatology service and the rationale for choosing the components.

Modality
Most-convenient method for the consultant dermatologist and requires the least amount of coordination or additional technology Asynchronous Meets identity and privacy guideline requirements, and secondary teledermatology is a regionally insured service by the provincial government Secondary teledermatology (referring a health care provider to a consultant dermatologist)

Technology
A convenient, ubiquitous technology that is secure; limited to institutional use, with no barrier to entry; and shared across clinics Secure intrainstitutional email Referring providers were requested to provide in-focus images; at least one close-up image; and if widespread, photos representing the overall distribution Photography Clinical A four-fold improvement over existing wait times at our institution, and enough time to discourage urgent consultations through this service, which would be better provided at an emergency care institution One-week response time To permit benefit evaluation, we did not restrict the clinical use of the system or make implicit suggestions about what conditions would be appropriate Free-form text permitted (no template)

Chart Review
Stored medical records for each consultation were reviewed from service initiation in late 2016 through 2017. All patients were previously registered in the hospital electronic medical record system, as they were a part of the referring family-practice unit. Contact information was collected for emailing or mailing surveys. Demographic information including age and gender was collated. Each chart was reviewed for clinical characteristics including diagnosis and management. For each patient, the hospital-wide electronic medical record was additionally reviewed for evidence of treatment failure, symptom recurrence of the treated dermatosis, or any other pertinent follow-up information.

Referring Provider Survey
Emails for each referring attending provider who submitted at least one consultation request were collated, and an email invitation from SurveyMonkey [10] was sent to request their participation in an electronic survey assessing satisfaction, time expenditure, educational value, and adverse events, which ranked these parameters on a Likert scale. If the provider did not complete the survey, invitations were resent 2 weeks and 1 month after the initial invitation. Health care providers rated each question on a scale of 1-7, where 1 was "Strongly disagree," 7 was "Strongly agree," and 4 was "Neutral."

Patient Survey
Patients were invited to participate in a survey assessing satisfaction with the service, potential benefits, and attitudes toward teledermatology. Email invitations were sent to the subset of patients who had provided an email address for hospital registration. An email was resent a month after the initial invitation if a survey was not received by the patient. For all patients, a paper survey with a self-addressed, stamped return envelope was mailed. Patients rated each question on a scale of 1-7, where 1 was "Strongly disagree," 7 was "Strongly agree," and 4 was "Neutral." The responses were averaged and interpreted according to the final score.
Participants were offered the opportunity to submit their email at the end of the study to be entered in a draw to receive a Can $100 gift card.

Referring Provider and Potential Consultant Semistructured Interviews
In the email invitations mentioned above (see Patient Survey section), referring providers were invited to participate in a semistructured interview exploring the perceived impact, current use, and attitudes and opinions toward teledermatology. Similarly, potential consultant dermatologists identified from a convenience sample of dermatologists in academic practice in the same hospital setting, who could potentially participate in a teledermatology service, were recruited via email for the semistructured interviews. Interview guidelines were prepared using the categories of the CHI Benefits Evaluation Framework to complement other data-collection strategies assessing satisfaction, use, educational benefits, and experience and attitudes toward teledermatology.
The primary author conducted interviews over a 3-month period after study initiation, with no other parties present. The interviews were audio-recorded and subsequently transcribed verbatim from the recordings by a professional transcription service.

Chart Review
Demographic and clinical information including complaints and diagnoses was collected. The primary author characterized complaints as "lesion," "rash," or "other," depending on whether the consultation was consistent with multiple or progressive skin findings over a generalized area (rash) or a single, stable skin finding in a localized area (lesion). We recorded the immediate follow-up plan including recommendation for subsequent in-person consultation, optional in-person consultation if the issue was not resolved by the prescribed method, and the proposed timing.

Referring Provider and Patient Surveys
Results from the Likert scales were collated, and the scores averaged with standard deviation. Free-form text answers were collated and reviewed using word cloud analysis and manual inspection for incorporation of the patterns or pertinent feedback into the final triangulation.

Referring Provider and Potential Consultant Interviews
For each set of interviews, the primary author and a secondary independent reviewer conducted independent qualitative analyses according to the interpretive description [11]. The transcripts were sequentially reviewed and evaluated with reference to categories from the CHI Benefits Evaluation Framework. Thematic analysis was used to identify unique emerging concepts potentially beyond the scope of the interview guide. The independent reviewers submitted their qualitative analyses to the primary author for review. Identified themes from both sets of analyses were included in the final mixed-methods analysis. No discordant themes required dispute management by a third party.

Mixed-Methods Concurrent Triangulation
For the final overarching analysis, we used the CHI Benefits Evaluation Framework to organize a mixed-methods concurrent triangulation strategy that used both qualitative and quantitative assessment methods. We added the evaluative components of potential educational benefits to this strategy. Emergent themes derived from any component were triangulated with other data analyses to support the conclusions.

Chart Review
A total of 76 participants used the service between November 2016 and December 2017, including 17 (22%) pediatric patients, 3 pregnant patients, 28 men (37%), and 48 women (63%). Consultation data for all patients were subjected to chart review. The average age of patients using the service was 39.3 years, and the average response time was 23.5 h, with a minimum response time of 0.5 h and a maximum time of 142.6 h (6 days).
Clinical characteristics were grouped into diagnostic dermatology categories, similar to previous teledermatology evaluations [12], and are presented in Table 2. Each case was broadly categorized into an easily recognizable presenting complaint: a "rash," which presents in multiple areas of the skin with the same general appearance, most typically represented by eczematous dermatoses, infections, or reactive skin conditions; a "lesion," which is a focal, persistent eruption on the skin, typified by benign and malignant growths; or a "question" about a treatment (Table 3). Of all cases, 12 (16%) required a subsequent in-person visit.
We assumed that clinicians would use photographs from a patient visit to their office; however, in 14% of all consultations, parent-or patient-initiated photographs were used. In 6 (35%) of the pediatric cases, photographs for analysis were provided by the parents, and 5 adult patients provided their own photographs to the clinician. None of these patients required additional photographs or in-person follow-up, and cases wherein the data were available, the conditions were appropriately resolved. In one case, the parent-initiated photograph showed evidence of a rare condition that occurs only under specific conditions.
Our results showed that it was possible to manage 94% of "rashes" by teledermatology alone, but only 59% of "lesions" could be managed by this method. Additionally, in 21 cases (27%), the primary management aimed to increase the prescribed potency of or choose a more-effective vehicle for the topical steroid used. Finally, in at least one case, a response time of 24 h permitted initiation of therapy to prevent postherpetic neuralgia complications of herpes zoster; however, the recommended 72-h time window for treatment [13] would have been exceeded if the maximum response time of the service (1 week) was applied in this case.

Referring Provider Survey
Of the 14 invitations sent, we received 11 responses for the online survey (79% response rate). Responses were averaged and interpreted according to the final score. The results of the survey and the Likert interpretation of the average score are presented in Table 4. Feedback was generally very positive for satisfaction and educational value. The clinicians reported that the service did not save any time, but increased the administrative time. However, they indicated that the benefits of the service outweighed the lost time and the service had educational value. No adverse events were reported.

Patient Survey
A total of 76 patients were sent paper surveys and 26 patients, who had provided an email address for provider registration at the institution, were sent a duplicate email invitation.
We received 22 responses (29%). Patients were generally satisfied with the service and reported that it saved time and money and prevented them from missing work. However, their views were divided when they were asked if they "prefer to use this system instead of going to see a specialist in-person." Their responses were instructive: Some patients indicated that they generally prefer in-person consultations, as they provide a greater opportunity to ask questions and clarify the rationale for therapeutic choices. These results are summarized in Table  5.   Table 4. Health care providers' responses to the survey (N=11).

Interpretation Mean (SD) Question
Strongly agree 6.6 (0.6) The responses from the dermatologist were complete. Strongly agree 6.8 (0.4) The responses from the dermatologist were timely. Strongly agree 6.5 (0.8) The service was reliable. Strongly agree 6.5 (0.7) I was satisfied with the answers to the clinical questions.
Strongly agree 6.4 (0.9) I was satisfied with the educational value of the system. Strongly agree 6.6 (0.5) The service was easy to use.            Although dermatologists were open to email communication, they expressed concern that additional administrative overhead may prevent uptake; for example, one dermatologist felt that,

[In some systems,] the platform is not efficient, it doesn't keep track of cases seen, billing codes, billing numbers, the kind of information that we need in the logistics of how we provide care...a proper kind of charting system or billing system. [Interview D3]
Interviews supported the data from patient surveys that both providers and patients were satisfied with the service. One provider stated, The patients are absolutely thrilled because...You can get back to them so quickly with such an informed opinion. [Interview F4] Table 6. Summary and interpretation of the unified mixed-methods evaluation using the Canada Health Infoway Benefits Evaluation Framework [9].

Interpretation Qualitative (interviews) Quantitative (surveys) Chart review Component
Institutional email facilitated a functional service with adequate performance and security Referring providers and dermatologists found the institutional email to be an appropriate medium, despite the increased administrative burden Providers "strongly agreed" that the service was reliable and easy to use The technology demonstrated robustness with no lost messages System quality: functionality, performance, and security A consultation format that incorporates morphology, diagnosis, reasoning used, and a treatment ladder was satisfactory and educational to providers Interviewees spoke positively about the structure of the consultations; one interviewee commented, "What I like to have the best is a plan that has multiple steps if the first doesn't work" (Interview F1) Providers "strongly agreed" that the responses were complete, provided satisfactory answers to the clinical questions, and had educational value All consultations were generally completed with one question and one response Information quality: content and availability A 24-h average response rate was appropriate for most outpatient clinical situations "It works because your notes are good and your turnaround time is fast" (Interview F5) Providers "strongly agreed" that the responses were timely. Patients also commented positively on the rapid response The average response time was consistent with that of other systems in the literature [14], and in at least one case, potentially prevented morbidity Service quality: responsiveness Although the evident benefits to patients and the educational value ameliorates the burden to providers, a more integrated workflow may have increased utilization by referees and potential consultants In interviews, dermatologists expressed that they would be more likely to offer the service if it represented an integrated workflow with medical records and billing Although the service did not generally save providers time, they "agreed" that the benefits outweighed the increased administrative burden Unexpectedly, parentand patient-initiated photos were used. Recent research suggests they can be used accurately for diagnosis while saving additional visits [15] Use: behavior, self-reported use, and intention to use The service demonstrated satisfaction for providers and patients "Most patients that I follow up with afterwards are really happy that they didn't have to go to any more measures (to be treated)" (Interview F2) Providers and patients "strongly agreed" that they were satisfied with the outcomes of the system A total of 92.8% (13 of 14) providers used the service more than once User satisfaction: competency, satisfaction, and ease of use Intrainstitutional teledermatology (concordant with existing literature) was safe and effective No adverse events or areas for improvement were reported in interviews No adverse events were reported in the survey. Patients "strongly agreed" that they would use this service again or recommend its use. Patient comments supported the idea that the correct management plan was identified In cases of independent follow-up, the results supported the diagnosis and management plan of the teledermatology consultant Quality: safety and outcomes Although easy to access, further integration of the patient into the physician-physician communication channel may have been beneficial Interviewees cited several patient factors (anxiety, sensitive locations, etc) that could theoretically prevent use, but no barriers noted in interviews actually prevented use Patients "strongly agreed" that they were comfortable if a physician they had never met evaluated their skin. However, they also "agreed" they would like to know what was communicated between the consultant and referring physician; this idea was supported by patient comments The service was used for a broad demographic of patients and complaints, suggesting no barriers to access Access: ability to access services, and patient and caregiver participation Intrainstitutional teledermatology increased productivity and efficiency even when geography was not a barrier to care Both potential dermatologists and referring providers characterized institutional email as an efficient tool for consultations. This efficiency increased when email communication was used between the provider and the patients Providers "strongly agreed" that the service saved health care system resources. Patients "agreed" that the service saved them time and money and resulted in less work missed

Triangulation Using the CHI Benefits Evaluation Framework
In the concurrent triangulation evaluation, we correlated the categories of the CHI Benefits Evaluation Framework with our multiple lines of inquiry ( Table 6). The hypothesis of net benefit was supported by both qualitative and quantitative evidence.

Principal Findings
The concurrent triangulation analysis was consistent with our hypothesis: The intrainstitutional teledermatology service was effective and satisfying to providers and patients even when geographic access was not a barrier. Store-and-forward teledermatology demonstrated diagnostic concordance [17]. In addition, a previous systematic review supported the idea that teledermatology is associated with high satisfaction [18]. Prior work has suggested that cost-effectiveness is not a universal feature of teledermatology [19] and would be highly dependent on regional factors. In our study, the service demonstrated 30% savings in visitation fees [16], and according to the patients, the service saved time and money and prevented them from missing work, which are important components of social cost.
To our knowledge, this is one of the first few studies to investigate the educational potential of teledermatology for referring providers embedded in a consultation service. Other studies have reported the value of incorporating trainees into the teleconsultation process [20] or initiatives such as project ECHO [21] that explore virtual education separate from the service. In our initiative, we found that an appropriately structured consultation note that explains clinical reasoning and offers a treatment ladder was educational and helpful to the referring provider. Furthermore, both dermatologists and referring providers suggested that the greatest educational value of this service was management, not diagnosis. This is supported by the chart review, which showed that 27% of all recommendations involved only alteration of the existing topical treatment. A dermatologist noted, "We all know that the challenge with skin diseases; people don't think about it from a morphology point of view. They think about it as; that person I saw with psoriasis three years ago, this looks like that, right? And so that educated piece, in terms of diagnostics, I think would be lost." (Interview D4). Incorporating formal management and educational initiatives into teleconsultation is an area of potential expansion and study in the future.
In our chart review, 41% of lesions required a subsequent in-person visit compared to only 6% of rashes. Therefore, in the future, such a service would be most productive and efficient if it targeted the management of "rashes" alone. This finding is in line with the view of dermatologists who expressed concern about their comfort with regard to managing lesions by traditional teledermatology and the abovementioned educational findings: There are fewer management options for lesions and therefore potentially less educational value in the discussion of those consultations.
Future services should consider a 24-h consultation response time to manage the most-serious outpatient clinical scenarios. Other pilot services in the literature meet or surpass this response time [22,23]. Important clinical conditions that are time sensitive, meaning that they need ≤24 h for diagnosis and rapid management in dermatology, are infections requiring intervention and serious drug rashes [24] because of the risk of significant morbidity and mortality. In our study, there was only one case of a treatable infection where identification within the period prevented morbidity; however, this may be because referring providers would less likely consider using teledermatology for high-acuity dermatoses if the "maximum" response time allotted was 1 week. However, setting the response time too low may decrease the number of potential dermatologists willing to participate in the service, unless adequately remunerated or alternatively structured (eg, shared "on-call" responsibilities or triage responsibilities delegated to trainees or other health care professionals).

Limitations
One limitation of this study is that this was a small case study at a single institution in a region that is supportive of virtual care. Therefore, the results may not be generalizable. As the primary author is currently the only consulting teledermatologist in our institution, there may be a better way to provide improved care and faster response times than a "usual case" scenario, which could lead to potential bias towards positivity. In addition, follow-ups or "second opinion" consultations may not have been accounted for if they were referred to other care clinics in the city outside the circle of care of the institution.

Conclusion
This is the first reported study on intrainstitutional teledermatology using a mixed-methods approach to explore the benefits of teledermatology, namely, rapid access, high satisfaction, safe and effective clinical outcomes, and increased efficiency and productivity in the health care system. The integration of teledermatology into a routine clinical workflow has remarkable potential value, irrespective of existing access to consultative dermatology. Specifically, by reducing healthcare visits, patients can save time and money and avoid missing work. Additionally, this study provides evidence that management of rashes is ideal via teledermatology, which may lead to better outcomes because of the rapid response and high proportion of cases that can be managed through teledermatology alone. This finding is concordant with the expressed opinions of dermatologists in our study who did not want to assess lesions with traditional teledermatology. The impact and satisfaction of an intrainstitutional teledermatology service that primarily targets rashes and, according to our study, ideally provides responses within 24 h to maximize safety, is worth evaluating in future studies. Further research is necessary to elucidate the educational value of intrainstitutional teledermatology for referring providers and its potential for saving costs.