Assessment of Patient Satisfaction With Dermatology Clinics According to Clinic Type: Mixed Methods Study

Introduction

Background

Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey responses are considered significant indicators of the quality of care and patient satisfaction. These scores are derived from a series of standardized patient surveys used to assess patients’ experiences and satisfaction with health care in the United States. These surveys include quality measures that patients are most qualified to assess, such as aspects related to communication and interaction with medical staff [1,2].

There is a pressing need to improve patient satisfaction rates as CAHPS scores are considered when determining the amount a facility will be reimbursed by the Centers for Medicare and Medicaid each year [2]. Thus, focusing on improving patient satisfaction can, in turn, help to improve the quality of care and patient outcomes, which are both primary initiatives in health care today.

The overall CAHPS scores for the dermatology clinics of a particular academic medical center (AMC; University of Alabama at Birmingham Hospital) have been unsatisfactory for various measures over the past several years. Anecdotally, the private clinic personnel suspected that the rapid access clinics were contributing to the overall low scores. Private clinics are set up such that patients make appointments in advance with the dermatologist of their choosing. In this environment, they can see the same dermatologist for every appointment. On the other hand, rapid access clinics are set up such that patients can get walk-in or “just-in-time” appointments with the dermatologist or dermatology nurse practitioner on duty. There is some crossover in dermatology clinicians; however, there is no guarantee that the patient can see the clinician of choice in the rapid access clinic environment. Additionally, the rapid access clinics involve a rapid approach. Patients are seen for situational care rather than routine care. For example, ideally, an annual dermatology examination would be scheduled in a private clinic where there is more time per appointment, whereas a newly occurring rash would be seen in a rapid access clinic where there is less time per appointment. This scheduling in the rapid access clinic environment is adopted to accommodate more patients. Assessment of the ambulatory CAHPS scores at a granular level was conducted to identify patterns and specific areas of measure between the two different types of dermatology clinics (private and rapid access) that are impacting the overall CAHPS scores. The overarching question being answered was as follows: Where should the efforts of patient satisfaction improvements be focused in the two different types of dermatology clinics?

CAHPS Survey as a Measure of Patient Experience

CAHPS survey data have been utilized and referenced as direct patient care quality indicators for health care organizations across the United States [1]. The push to move toward a pay-for-performance model and quality-driven health care in the United States has influenced the emergence of such measuring tools and initiatives [3]. According to the Agency for Healthcare Research and Quality, the creator of the CAHPS program, the CAHPS survey is an initiative that was designed to “support investigator-led research to better understand patient experience with health care and develop scientifically valid and feasible strategies and tools to assess patient experience, report survey results, and help organizations use the results to improve the quality of care” [1].

The results of CAHPS measures can effectively bring attention to the shortfalls of health care providers and emphasize the need for improvements in their delivery of quality care. However, there appears to be a gap between understanding the collected data and being able to use and interpret the data to create actionable goals and improvement initiatives [4]. The availability of such data is only valuable if it can be translated and applied to create relevant solutions and to optimize the impact such solutions would have on quality care outcomes for an organization. Thus, determining which care factors and initiatives correlate directly with outcome measures, such as those obtained from CAHPS surveys, remains a challenge in health care today.

Relationship of Data Analysis and Patient Care

Aside from existing challenges, research studies infer that data collection and analysis efforts of patient care and quality measures are highly useful for identifying shortcomings in the delivery of patient care and indicating the importance of continually measuring care plans [5,6]. According to one source, a key element that strengthens patient satisfaction and confidence in clinical care is viewing quality improvement as a constant, iterative, and transparent process [7]. There is also evidence indicating that the publication of care performance data influences quality improvement initiatives to be adopted by hospitals [8]. As such, the awareness of scores being accessible to the public can encourage organizations to be more conscious of their results [8].

Additionally, satisfaction scores are highly useful to identify opportunities for improvement in aspects of patient care. Furthermore, the literature suggests that patient satisfaction should be consistently evaluated and used to assess specific interventions and improvement areas within care practices [5]. For example, in research associated with a Healthcare Information and Management System Society study (HIMSS), measurement of patient satisfaction revealed that providers’ perception of their own performance does not always accurately align with patient reality, as approximately 72% of patients responded being extremely or very satisfied with their overall care experience as compared with only 39.7% of providers believing they were extremely or very effective (N=309 and N=204, respectively) [9], suggesting that anecdotal reports may not accurately reflect actual patient satisfaction. This supports the purpose and relevance of the analysis of the dermatology clinics’ CAHPS scores. The information contained in patient feedback and surveys can outline the areas in need of improvement and, based on what those areas are, indicate what type of recommendations need to be pursued. The objective of this study was to analyze the ambulatory CAHPS scores at a granular level in order to identify patterns and specific areas of measure in the two different types of dermatology clinics (private and rapid access clinics), which are impacting the overall CAHPS scores.

Methods

Study Design

This study had a concurrent mixed methods design, where the findings from a qualitative analysis were used to enrich the findings from a quantitative analysis. Deidentified secondary data derived from the University of Alabama at Birmingham Hospital’s Press Ganey website were analyzed for clinic type comparisons. The data were analyzed to determine the contributors responsible for each clinic not meeting national patient satisfaction benchmarks. Thereafter, these contributing factors were further analyzed to determine the difference in CAHPS scores between the private and rapid access clinics to determine if clinic type was a contributing factor to the overall scores.

Based on the most recent survey template provided, these surveys consist of over 60 different standardized CAHPS questions within 10 different domains, 13 additional nonstandard questions, and a section for additional patient comments that assess patient care experience and satisfaction with their visits to the clinics. The scope of this study required filtering of the dataset by dermatology. In order to separate the private clinics and rapid access clinics from the larger dataset, a holistic week-long schedule was assessed, and it outlined the times of operation and staffing for the dermatology clinics and provided the necessary information for comparison of the survey data by clinic type, which was conducted further in the analysis.

The data sample included 821 responses from May 24, 2017 through May 24, 2018. All data were analyzed using Microsoft Excel (Microsoft Corporation, Redmond, Washington, USA) and Tableau (Salesforce, Mountain View, California, USA) for visualization. This study was conducted at the University of Alabama at Birmingham Hospital (approval number: #300003087).

Analysis of the data was performed in three rounds. Round I (quantitative) identified which CAHPS measures and criteria consistently scored the lowest across all clinics and appeared as the greatest contributors to the department’s annual CAHPS scores. This round also tried to determine whether the rapid access clinics scores alone were the sole reason why the dermatology scores were low. Round II (quantitative) examined differences between the private and rapid access clinics. Round III (qualitative) took a more granular approach to further investigate the nonstandard quality measures and unstructured patient comments. Figure 1 illustrates the research process from data collection to the integration of findings.

‎View this figure

Quantitative Data Analysis

Round I: Contributors to the Annual CAHPS Score

The process for Round I of the analysis involved extracting the average scores for all standard CAHPS categories and measures for the most recent year, which included survey data of only the private clinics (2018-2019), and the prior year, which included survey data for both the rapid access and private clinics (2017-2018), for comparison.

Comparing CAHPS scores between these two years helped to show the impact of omitting rapid access survey data on dermatology clinics’ ratings, which was done in 2018-2019. The measures were then ranked from best to worst based on the average values for both years to determine which categories and individual measures had the most opportunities for improvement. This data review helped to highlight the primary areas in need of attention while also providing insights on specific themes or components of care that may be generally lacking among the clinics.

The format in which the data are presented in Press Ganey-generated reports is not ideal for in-depth analysis and requires substantial restructuring. For example, each measure is separated into different sections in the comma separated value (csv) file, with individual sets of rows and columns for each response.

In order to create standard response categories across measures for a more analysis-friendly format, the different response types were grouped together based on their equivalence to another response. For example, “no” and “never” responses were combined into one response subgroup “no/never,” whereas “yes, definitely,” “yes,” and “always” categories were compiled into the response subgroup “yes, definitely/yes/always.” CAHPS questions also included “yes, somewhat/sometimes” and “usually” as responses, but for the purpose of this analysis, only the highest and lowest response categories (“no/never” and “yes, definitely/yes/always”) were used for comparison. A sample of the data in restructured format is presented in Table 1.

Round II: CAHPS Score by Clinic Type

When using Press Ganey data for an organization’s specific needs, such as differentiating clinics, a process must be used to merge the data in ways other than the original intent when the survey was developed. For example, our organization’s need was to compare two clinics; however, the survey, in its original form, does not allow for this type of comparison. In this section, we detail the process for readers who have interest in duplicating this or a similar study at their organization. Press Ganey does not readily distinguish clinic types. Therefore, the only way to separate data by clinic type was to cross-analyze the Press Ganey data with a physician schedule tracking sheet. The physician schedule tracking sheet facilitated identification of clinics, as well as physicians associated with certain outcomes that could not be determined solely by the CAHPS data. The clinic schedule included days of the week and times of the day (am vs pm) specific physicians work in the private and rapid access clinics, whereas the CAHPS scores in Press Ganey can be viewed by visit date/time. Using the clinic schedules for comparisons with the data available in Press Ganey, we could match the clinic type with the corresponding CAHPS data.

For example, in order to divide the datasets by clinic type, four separate reports had to be pulled from Press Ganey and complied accordingly using schedule and physician criteria included in the schedule. The first report consisted of average patient satisfaction rates for Monday, Wednesday, and Thursday, which were unfiltered to sort out the first section of private clinic data. The second report then consisted of patient satisfaction scores for Tuesday and Friday (am appointments only). To separate the remaining data for only the private clinics, a third report was used to extract scores for Tuesday and Friday (pm appointments), which included only those providers who met patients on these specific days. Finally, the fourth report was extracted for only rapid access data, which included patient satisfaction scores for Tuesday and Friday (pm appointments only), and a filter was applied to exclude the providers of the private clinics (pm) for these specific days. This process had to be completed for each time specific data point (standard CAHPS measures, nonstandard quality measures, and patient comments) and was separated by clinic type.

Qualitative Data Analysis

Round III: Unstructured Patient Comments

A third round of analysis took a more granular approach to further investigate patient comments. The use of unstructured text comments required an additional data preparation step, in that they had to be assigned to standard groups that could allow for comparison. The preparation for these comments consisted of reviewing all patient comments for both types of dermatology clinics from 2017-2018 and categorizing them into response type themes as follows: positive, negative, indifferent, and not applicable. For example, a comment “I appreciated the receptionist helping me make my next appointment” was categorized as positive. By contrast, a comment “I waited 3 months for my appointment, only to have it cancelled by the doctor the week beforehand” was categorized as negative. The “positive” and “negative” categorized comments were then further categorized into subthemes based on aspects of care to which they were most frequently related. The subthemes included medical needs not met or questions not answered; lack of or delay with results or records; long wait or slow processing; appointment rushed or barely saw the doctor; lack of communication, direction, or follow up; medical staff rude, impersonal, or did not listen; difficult to schedule timely appointment or appointment cancelled by the doctor; check in process frustrating, receptionist unhelpful, or lack of assistance; and general or other. Thereafter, the data were processed through the same organizational approach described in Round I.

Results

Results Structure

Results are presented in the following order: demographics, contributors to CAHPS scores (Round I), results of nonstandard quality measures of CAHPS scores by clinic type (Round II), and unstructured patient data (Round III).

CAHPS Survey Respondents’ Demographic Data

As shown in Table 2, just over half of the respondents were female (464/821, 56.5%), an overwhelming majority were white (650/821, 83.0%), and most were between 50 and 79 years old (589/821, 71.0%). Moreover, a vast majority had some college education (670/821, 87.2%).

Quantitative Data Analysis Results

Round I: Contributors to the Annual CAHPS Score

As mentioned in the Methods, the annual CAHPS scores were first analyzed with and without the rapid access clinics included. The negative responses (no/never) are shown in Figure 2 and the positive responses (yes, definitely/yes/always) are shown in Figure 3. Both figures represent the comparison of “no/never” and “yes, definitely/yes/always” response rates overall (all weekdays) across CAHPS categories between the year with rapid access clinic scores (2017-2018) and the year without rapid access clinics scores (2018-2019). On examining Figure 2, stewardship of patient resources indicates, among other things, whether clinicians consider the needs of patients when making therapy, medication, or referral recommendations. For this variable, it was found that 66.8% (367/549) of respondents said “no/never” with the inclusion of rapid access clinics as compared with 70.9% (783/1104) of respondents without the inclusion of rapid access clinics, indicating that the data without the rapid access clinics were worse. Another example is how well providers communicate with patients, which is shown in Figure 2 as “physician communication quality,” indicating how effective is provider communication with patients. For this variable, it was found that 2.1% (13/638) of respondents said “no/never” with the inclusion of rapid access clinics (meaning communication was poor) as compared with 2.2% (27/1227) of respondents without the inclusion of rapid access clinics, indicating that the data without the rapid access clinics were worse, and thus, the data were worse for private clinics.

Figure 3 shows the positive responses while using the same quality measures. With regard to stewardship of patient resources, involving whether providers consider the needs of patients, 33.2% (182/549) of respondents said “yes, definitely/yes/always” with the inclusion of rapid access clinics as compared with 29.1% (321/1104) of respondents without the inclusion of rapid access clinics, indicating that a greater percentage thought that their needs were considered in data that included the rapid access clinics. Similarly, with regard to physician communication, 91.0% (581/638) of respondents with the inclusion of rapid access clinics answered “yes, definitely/yes/always” as compared with 90.6% (1112/1227) of respondents without the inclusion of rapid access clinics.

Considering Figures 2 and 3, there does not appear to be a wide variance between most of the measures. One exception is “access to specialists.” On assessing the data that included the rapid access clinics (Figure 2), 0.0% of respondents reported “no/never” access to specialists, indicating that specialists are accessible. However, on assessing the data that did not include the rapid access clinics (ie, private clinics only; Figure 2), 10.0% (122/1224) of respondents reported “no/never” access to specialists, indicating that specialists were not accessible. The variance is much wider in Figure 3, which presents the positive responses while using the same quality measures. With regard to access to specialists, Figure 3 demonstrates that 66.7% (2/3) of respondents said “yes, definitely/yes/always” with the inclusion of the rapid access clinics as compared with 20.0% (1/5) of respondents without the inclusion of the rapid access clinics, indicating that more respondents felt that there was access to specialists with inclusion of the rapid access clinics.

‎View this figure

‎View this figure

Figures 4 and 5 present comparisons of “no/never” and “yes, definitely/yes/always” response rates for Tuesday and Friday (rapid access clinic days) across CAHPS categories for the year with rapid access clinic scores (2017-2018) and the year without rapid access clinic scores (2018-2019). These results suggest the same conclusions as those involving data for all weekdays. It appears that there was also a slight increase in negative response ratings and a slight decrease in positive response ratings for several of the categories. However, in the case of both results, there were also some CAHPS categories that had an increase in the positive response average and a decrease in the negative response average. Essentially, the results are not consistent enough to indicate whether the inclusion or exclusion of rapid access clinic scores greatly impacted the dermatology clinics’ CAHPS scores as a whole. However, these results consistently showed that CAHPS scores for the “stewardship of patient resources” category was higher for the negative response rate (Figure 4), indicating that in the rapid access clinics, patients reported better stewardship of patient resources. Unexpectedly, both with and without inclusion of the rapid access clinics did not receive any responses (positive or negative) for the “access to specialists” category (Figures 4 and 5). Figure 4 shows that when the rapid access clinics were included, there was better consideration for patient resources.

It was noted earlier that the rapid access clinics only operated on Tuesday and Friday afternoons. However, the data in Figure 3 considered the entire day because comparisons were performed between two separate years (one including and the other excluding rapid access clinic data). Therefore, if the data were compared between the two years by Tuesday and Friday (pm), there would have been no rapid access clinic data available according to the criteria for 2018-2019. Using the data of the entire day for the weekdays when the rapid access clinics operated allowed us to observe the impact of excluding rapid access data for those particular days and the results associated with the private clinics alone for the most current year (2018-2019).

‎View this figure

‎View this figure

Round II: Data Review by Clinic Type

We show the results relative to how well the private clinics and rapid access clinics scored on average across all CAHPS categories in Figure 6. Thereafter, we show more granular results of the focus measures by clinic, with the results of the private clinics presented in Figure 7 and those of the rapid access clinics presented in Figure 8.

The focus measures were the measures that targeted physician communication quality, standard measures as well as additional measures based on relevance to care quality, and the potential for improvement relative to the effort for change management. All CAHPS categories were used in the first comparison, rather than selecting just those related to the focus measures, in order to conduct a general comparison of all CAHPS categories according to clinic type. In other words, Figure 6 shows the higher level CAHPS categories/domains that were measured. Each of these categories/domains had a set of related CAHPS measures. The categories/domains relevant to the focus measures chosen for analysis are indicated in Figure 6. These focus measures are further assessed in Figures 7 and 8.

The results of the overall CAHPS scores by clinic type, as depicted in Figure 6, showed that the private clinics had higher patient satisfaction rates across most CAHPS categories when compared with the rapid access clinics. When looking further at the focus measures by clinic type (Figures 7 and 8), most of the measures were higher for the private clinics as compared with the rapid access clinics, although some measures were only slightly higher.

We present the findings for focus measures in the private clinics and rapid access clinics in Figures 7 and 8, respectively. On comparing the two clinic types, the results showed that the private clinics scored higher in most areas. The rapid access clinics scored higher in “appointment for care right away,” “talk with provider about prescription,” “provider gives written medical instructions,” and “talk with provider about problem/concern.” These findings make sense when considering the nature of the rapid access clinics as a situational environment where a patient shows up with or without an appointment, has a targeted conversation with the provider about typically a singular problem or concern, gets a prescription or some other type of instruction, and leaves.

‎View this figure

‎View this figure

‎View this figure

We present the findings for nonstandard focus measures by clinic type in Figure 9. The results showed that the private clinics scored higher than the rapid access clinics. The marginal difference in patient satisfaction between the two clinic types was 12%-15%. Although the primary goal was to find a pattern in patient satisfaction between the clinic types, it can be noted that there is room for improvement in both private and rapid access clinics regarding these measures.

‎View this figure

Qualitative Data Analysis Results

Round III: Nonstandard Quality Measures and Patient Comments

We obtained the results of analyzing unstructured patient comments by clinic type. As listed in the Methods, the subthemes included the following: medical needs not met or questions not answered; lack of or delay with results or records; long wait or slow processing; appointment rushed or barely saw the doctor; lack of communication, direction, or follow-up; medical staff rude, impersonal, or did not listen; difficult to schedule timely appointment or appointment cancelled by the doctor; check in process frustrating, receptionist unhelpful, or lack of assistance; and general or other.

Figure 10 shows all patient comments by clinic type. It was found that the private clinics had a much higher rate of positive comments as compared with the rapid access clinics. On breaking down the amount each clinic type contributed to the overall comments for the positive, negative, indifferent, and not applicable categories, the private clinics appeared to contribute the most to each category. However, this finding is most likely influenced by the fact that there are more private clinics that run throughout the week and therefore a greater volume of comments associated with the private clinics. It is worth noting that there were 4454 comments from the private clinics and only 830 comments from the rapid access clinics.

We also assessed the distribution of the comments by clinic type within the sample (ie, the number of negative rapid access comments out of the total number of comments [N] for rapid access only; Figure 10). This analysis showed that the rapid access clinics had a greater makeup of negative comments as compared with the private clinics.

We expanded the negative comments by clinic, and we present the findings for the private clinics and rapid access clinics in Table 3. For the private clinics, the top three negative comments centered around “medical staff rude, impersonal, or did not listen,” “difficult to schedule timely appointment or appointment cancelled by the doctor,” and “check in process frustrating, receptionist unhelpful, or lack of assistance.” The “general or other” category included a variety of comments that did not appropriately fit into any other category (eg, singular words, such as “frustrating” and “disappointing,” without any context).

‎View this figure

Discussion

Principal Findings

This study assessed two distinct dermatology clinics and their individual and collective contributions to CAHPS scores for the Dermatology Department at an AMC to answer the question, “Where should the efforts of patient satisfaction improvements be focused for the two different types of dermatology clinics?” This study utilized a concurrent mixed methods approach (Figure 1) to answer this question.

The clinics consisted of private clinics (patients make appointments in advance with the dermatologist of their choosing and likely see the same dermatologist with whom they have established a relationship) and rapid access clinics (patients can get walk-in or just-in-time appointments with a dermatologist or dermatology nurse practitioner on duty).

The quantitative data analysis showed that when only private clinic data were collected, there was a slight increase in the negative response rate and a slight decrease in the positive response rate across many categories, perhaps indicating that the contribution of rapid access CAHPS scores to the overall scores may be underestimated [2]. When individual clinic data were expanded, the majority of reviewed CAHPS measures had lower satisfaction for the rapid access clinics (9%-17%) as compared with the private clinics. The exception was “talk with provider about problem/concern,” for which the rate was 0.34% higher in the rapid access clinics when compared with the private clinics. Although some of these differences in scores were not substantial, they were consistent enough across the CAHPS categories and measures to give a thought to the individual clinic contribution.

To elaborate further on the qualitative data analysis, the value of patient comments cannot be underestimated as they provide straightforward and insightful feedback from patients. The comments directly express dermatology patients’ perceptions of their care while specifically indicating areas in need of improvement, as well as satisfactory aspects of the clinics. As such, consistent with the literature suggesting continued review of CAHPS scores, a frequent review of these comments could contribute to and increase the richness of aggregated data [5,7].

Both quantitative and qualitative data analyses were performed in this study with the goal of the qualitative data analysis findings enriching the quantitative data analysis findings. Integration of the quantitative and qualitative findings from this study suggests that the rapid access clinics contributed to low overall CAHPS scores, but the contribution was not radical. Qualitative analysis of patient comments elaborated the results of quantitative findings, showing that even when patients know they are in an environment of situational care (ie, a rapid access clinic), they want to spend more time with the clinician, want shorter wait times, and want people to listen. The consideration of these comments provided an additional, valuable, and actionable layer to complement the quantitative findings.

Regardless of the clinic, this study suggested that there is work to be done in regards to how staff or clerks interact with patients, as this aspect was scored low in both clinics. There is also work to be done around communication. Clinicians need to be especially conscientious to avoid speaking to their patients in a frustrated or condescending tone.

The difference between real and perceived wait times presents challenges for even the best clinic environment. A patient’s perception relative to the wait time could be altered by providing frequent updates on the wait time so as to provide patients with a more accurate perception of the time they are actually waiting, which may be a smaller window of time as compared to what they perceive without receiving any updates [10]. Overbooking patients can lead to communication issues. If there are too many patients, the clinician may spend less time with each patient, giving the patient a sense of being rushed and not being heard.

There is also work to be done around effectively managing delays in the clinic and getting to the root cause of those delays. If found that delays are caused by patients, it would be helpful to identify patients who are frequently late or who do not show up to the clinic. These particular patients could then be flagged and scheduled in the afternoon for future appointments. Scheduling such patients later in the day would minimize the interference with the clinic schedule and overall patient flow throughout the day [10].

Our findings show that appointments are much more difficult to get in private clinics and patients need to wait a long time to get an appointment with their established provider. This is consistent with the nationwide shortage of dermatologists [11].

Limitations

This study had some limitations that need to be acknowledged. First, the data provided for this analysis by Press Ganey were already summarized, and this limited the statistical approaches that could be applied for further assessment. As such, analysis and result reporting were limited to descriptive statistics rather than more traditional statistical analysis to compare one dataset to another.

Another limitation was related to the survey design. The CAHPS survey does not have a not applicable (“N/A”) response available as an option for any of the questions. Subsequently, when any of the questions do not apply to patients, which was stated in patient comments, they are forced to answer “no/never” instead, artificially increasing the negative response rate for certain measures as a result.

Additionally, inconsistent response sample sizes (N) for each of the measures likely contributed to skewed reporting. For example, not every patient completed the survey, and those who completed the survey did not always answer every question available (most likely when questions did not apply). This was verified through a review of the original survey copies that were available through Press Ganey. Several of the sample surveys reviewed were only partially filled out.

Lastly, patient comments were fewer for the rapid access clinics as compared with the private clinics. The investigation of methods to increase comments represents an area of future research for researchers.

Nevertheless, this study provides valuable findings related to the degree of the contribution of individual clinics to CAHPS scores.

Conclusion

This study suggests that clinic type is relevant to patient satisfaction. However, the results also suggest that much of the dissatisfaction could be mitigated with better communication and level-setting expectations. There are gaps in research concerning the relationship between CAHPS scores and specific care factors. The findings and the common patterns between patient satisfaction measures and care components that were discovered through the execution of this analysis provide valuable insights into the drivers of patient satisfaction for not only the AMC’s dermatology clinics going forward, but also ambulatory facilities in general.

Future research in this area could investigate physician performance relative to clinic type. Additionally, patient satisfaction by time of the day according to clinic type may provide useful insights into patient satisfaction.