This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Dermatology Research, is properly cited. The complete bibliographic information, a link to the original publication on http://derma.jmir.org, as well as this copyright and license information must be included.
Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey responses are considered significant indicators of the quality of care and patient satisfaction. There is a pressing need to improve patient satisfaction rates as CAHPS survey responses are considered when determining the amount a facility will be reimbursed by the Centers of Medicare and Medicaid each year. Low overall CAHPS scores for an academic medical center’s dermatology clinics were anecdotally attributed to clinic type. However, it was unclear whether clinic type was contributing to the low scores or whether there were other factors.
This study aimed to determine where the efforts of patient satisfaction improvement should be focused for two different types of dermatology clinics (private and rapid access clinics).
This study used a concurrent mixed methods design. Secondary data derived from the University of Alabama at Birmingham Hospital’s Press Ganey website were analyzed for clinic type comparisons and unstructured data were qualitatively analyzed to further enrich the quantitative findings. The University of Alabama at Birmingham Hospital is an academic medical center. The data were analyzed to determine the contributors responsible for each clinic not meeting national benchmarks. Thereafter, a review of these contributing factors was further performed to assess the difference in CAHPS scores between the private and rapid access clinics to determine if clinic type was a contributing factor to the overall scores.
The data sample included 821 responses from May 2017 to May 2018. Overall, when both private clinics and rapid access clinics were viewed collectively, majority of the patients reported stewardship of patient resources as the most poorly rated factor (367/549, 66.8%) and physician communication quality as the most positively rated factor (581/638, 91.0%). However, when private clinics and rapid access clinics were viewed individually, rapid access clinics contributed slightly to the overall lower dermatology scores at the academic medical center.
This study determined that different factors were responsible for lower CAHPS scores for the two different dermatology clinics. Some of the contributing factors were associated with the mission of the clinic. It was suspected that the mission had not been properly communicated to patients, leading to misaligned expectations of care at each clinic.
Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey responses are considered significant indicators of the quality of care and patient satisfaction. These scores are derived from a series of standardized patient surveys used to assess patients’ experiences and satisfaction with health care in the United States. These surveys include quality measures that patients are most qualified to assess, such as aspects related to communication and interaction with medical staff [
There is a pressing need to improve patient satisfaction rates as CAHPS scores are considered when determining the amount a facility will be reimbursed by the Centers for Medicare and Medicaid each year [
The overall CAHPS scores for the dermatology clinics of a particular academic medical center (AMC; University of Alabama at Birmingham Hospital) have been unsatisfactory for various measures over the past several years. Anecdotally, the private clinic personnel suspected that the rapid access clinics were contributing to the overall low scores. Private clinics are set up such that patients make appointments in advance with the dermatologist of their choosing. In this environment, they can see the same dermatologist for every appointment. On the other hand, rapid access clinics are set up such that patients can get walk-in or “just-in-time” appointments with the dermatologist or dermatology nurse practitioner on duty. There is some crossover in dermatology clinicians; however, there is no guarantee that the patient can see the clinician of choice in the rapid access clinic environment. Additionally, the rapid access clinics involve a rapid approach. Patients are seen for situational care rather than routine care. For example, ideally, an annual dermatology examination would be scheduled in a private clinic where there is more time per appointment, whereas a newly occurring rash would be seen in a rapid access clinic where there is less time per appointment. This scheduling in the rapid access clinic environment is adopted to accommodate more patients. Assessment of the ambulatory CAHPS scores at a granular level was conducted to identify patterns and specific areas of measure between the two different types of dermatology clinics (private and rapid access) that are impacting the overall CAHPS scores. The overarching question being answered was as follows: Where should the efforts of patient satisfaction improvements be focused in the two different types of dermatology clinics?
CAHPS survey data have been utilized and referenced as direct patient care quality indicators for health care organizations across the United States [
The results of CAHPS measures can effectively bring attention to the shortfalls of health care providers and emphasize the need for improvements in their delivery of quality care. However, there appears to be a gap between understanding the collected data and being able to use and interpret the data to create actionable goals and improvement initiatives [
Aside from existing challenges, research studies infer that data collection and analysis efforts of patient care and quality measures are highly useful for identifying shortcomings in the delivery of patient care and indicating the importance of continually measuring care plans [
Additionally, satisfaction scores are highly useful to identify opportunities for improvement in aspects of patient care. Furthermore, the literature suggests that patient satisfaction should be consistently evaluated and used to assess specific interventions and improvement areas within care practices [
This study had a concurrent mixed methods design, where the findings from a qualitative analysis were used to enrich the findings from a quantitative analysis. Deidentified secondary data derived from the University of Alabama at Birmingham Hospital’s Press Ganey website were analyzed for clinic type comparisons. The data were analyzed to determine the contributors responsible for each clinic not meeting national patient satisfaction benchmarks. Thereafter, these contributing factors were further analyzed to determine the difference in CAHPS scores between the private and rapid access clinics to determine if clinic type was a contributing factor to the overall scores.
Based on the most recent survey template provided, these surveys consist of over 60 different standardized CAHPS questions within 10 different domains, 13 additional nonstandard questions, and a section for additional patient comments that assess patient care experience and satisfaction with their visits to the clinics. The scope of this study required filtering of the dataset by dermatology. In order to separate the private clinics and rapid access clinics from the larger dataset, a holistic week-long schedule was assessed, and it outlined the times of operation and staffing for the dermatology clinics and provided the necessary information for comparison of the survey data by clinic type, which was conducted further in the analysis.
The data sample included 821 responses from May 24, 2017 through May 24, 2018. All data were analyzed using Microsoft Excel (Microsoft Corporation, Redmond, Washington, USA) and Tableau (Salesforce, Mountain View, California, USA) for visualization. This study was conducted at the University of Alabama at Birmingham Hospital (approval number: #300003087).
Analysis of the data was performed in three rounds. Round I (quantitative) identified which CAHPS measures and criteria consistently scored the lowest across all clinics and appeared as the greatest contributors to the department’s annual CAHPS scores. This round also tried to determine whether the rapid access clinics scores alone were the sole reason why the dermatology scores were low. Round II (quantitative) examined differences between the private and rapid access clinics. Round III (qualitative) took a more granular approach to further investigate the nonstandard quality measures and unstructured patient comments.
Mixed methods research design populated with results and the integration of those results. CAHPS: Consumer Assessment of Healthcare Providers and Systems.
The process for Round I of the analysis involved extracting the average scores for all standard CAHPS categories and measures for the most recent year, which included survey data of only the private clinics (2018-2019), and the prior year, which included survey data for both the rapid access and private clinics (2017-2018), for comparison.
Comparing CAHPS scores between these two years helped to show the impact of omitting rapid access survey data on dermatology clinics’ ratings, which was done in 2018-2019. The measures were then ranked from best to worst based on the average values for both years to determine which categories and individual measures had the most opportunities for improvement. This data review helped to highlight the primary areas in need of attention while also providing insights on specific themes or components of care that may be generally lacking among the clinics.
The format in which the data are presented in Press Ganey-generated reports is not ideal for in-depth analysis and requires substantial restructuring. For example, each measure is separated into different sections in the comma separated value (csv) file, with individual sets of rows and columns for each response.
In order to create standard response categories across measures for a more analysis-friendly format, the different response types were grouped together based on their equivalence to another response. For example, “no” and “never” responses were combined into one response subgroup “no/never,” whereas “yes, definitely,” “yes,” and “always” categories were compiled into the response subgroup “yes, definitely/yes/always.” CAHPS questions also included “yes, somewhat/sometimes” and “usually” as responses, but for the purpose of this analysis, only the highest and lowest response categories (“no/never” and “yes, definitely/yes/always”) were used for comparison. A sample of the data in restructured format is presented in
Restructured data from Press Ganey survey responses.
CAHPSa measure (2017-2018) | Category | Sample size, n | No/never, n (%) | Yes, definitely/yes/always, n (%) |
Recommend this provider office | Global | 776 | 16 (2.1) | 706 (91.0) |
Provider explains in a way you understand | Physician communication quality | 777 | 14 (1.8) | 724 (93.2) |
Provider listens carefully to you | Physician communication quality | 776 | 18 (2.3) | 723 (93.2) |
Provider talks with you about a problem/concern | Physician communication quality | 775 | 94 (12.1) | 681 (87.9) |
Provider gives easy to understand instructions | Physician communication quality | 681 | 8 (1.2) | 628 (92.2) |
Provider knows important information/medical history | Physician communication quality | 775 | 32 (4.1) | 647 (83.5) |
Provider shows respect for what you say | Physician communication quality | 775 | 7 (0.9) | 734 (94.7) |
Provider spends enough time with you | Physician communication quality | 777 | 16 (2.1) | 710 (91.4) |
Clerks/receptionists are helpful | Office staff quality | 776 | 19 (2.4) | 673 (86.7) |
Clerks treat you with courtesy/respect | Office staff quality | 775 | 3 (0.4) | 718 (92.6) |
aCAHPS: Consumer Assessment of Healthcare Providers and Systems.
When using Press Ganey data for an organization’s specific needs, such as differentiating clinics, a process must be used to merge the data in ways other than the original intent when the survey was developed. For example, our organization’s need was to compare two clinics; however, the survey, in its original form, does not allow for this type of comparison. In this section, we detail the process for readers who have interest in duplicating this or a similar study at their organization. Press Ganey does not readily distinguish clinic types. Therefore, the only way to separate data by clinic type was to cross-analyze the Press Ganey data with a physician schedule tracking sheet. The physician schedule tracking sheet facilitated identification of clinics, as well as physicians associated with certain outcomes that could not be determined solely by the CAHPS data. The clinic schedule included days of the week and times of the day (am vs pm) specific physicians work in the private and rapid access clinics, whereas the CAHPS scores in Press Ganey can be viewed by visit date/time. Using the clinic schedules for comparisons with the data available in Press Ganey, we could match the clinic type with the corresponding CAHPS data.
For example, in order to divide the datasets by clinic type, four separate reports had to be pulled from Press Ganey and complied accordingly using schedule and physician criteria included in the schedule. The first report consisted of average patient satisfaction rates for Monday, Wednesday, and Thursday, which were unfiltered to sort out the first section of private clinic data. The second report then consisted of patient satisfaction scores for Tuesday and Friday (am appointments only). To separate the remaining data for only the private clinics, a third report was used to extract scores for Tuesday and Friday (pm appointments), which included only those providers who met patients on these specific days. Finally, the fourth report was extracted for only rapid access data, which included patient satisfaction scores for Tuesday and Friday (pm appointments only), and a filter was applied to exclude the providers of the private clinics (pm) for these specific days. This process had to be completed for each time specific data point (standard CAHPS measures, nonstandard quality measures, and patient comments) and was separated by clinic type.
A third round of analysis took a more granular approach to further investigate patient comments. The use of unstructured text comments required an additional data preparation step, in that they had to be assigned to standard groups that could allow for comparison. The preparation for these comments consisted of reviewing all patient comments for both types of dermatology clinics from 2017-2018 and categorizing them into response type themes as follows: positive, negative, indifferent, and not applicable. For example, a comment “I appreciated the receptionist helping me make my next appointment” was categorized as positive. By contrast, a comment “I waited 3 months for my appointment, only to have it cancelled by the doctor the week beforehand” was categorized as negative. The “positive” and “negative” categorized comments were then further categorized into subthemes based on aspects of care to which they were most frequently related. The subthemes included medical needs not met or questions not answered; lack of or delay with results or records; long wait or slow processing; appointment rushed or barely saw the doctor; lack of communication, direction, or follow up; medical staff rude, impersonal, or did not listen; difficult to schedule timely appointment or appointment cancelled by the doctor; check in process frustrating, receptionist unhelpful, or lack of assistance; and general or other. Thereafter, the data were processed through the same organizational approach described in Round I.
Results are presented in the following order: demographics, contributors to CAHPS scores (Round I), results of nonstandard quality measures of CAHPS scores by clinic type (Round II), and unstructured patient data (Round III).
As shown in
Survey respondents’ demographics (N=821).
Characteristic | Value, n (%) | ||
|
|
||
|
Female | 464 (56.5%) | |
|
Male | 357 (43.5%) | |
|
|
||
|
White | 650 (83.0%) | |
|
African American | 93 (11.9%) | |
|
Asian | 7 (0.9%) | |
|
Hispanic/Latino | 5 (0.7%) | |
|
Hawaiian/Pacific Islander | 2 (0.3%) | |
|
American Indian/Alaska Native | 3(0.4%) | |
|
Other | 13 (1.7%) | |
|
|
||
|
0-17 | 8 (1.0%) | |
|
18-34 | 79 (9.6%) | |
|
35-49 | 86 (10.5%) | |
|
50-64 | 254 (30.9%) | |
|
65-79 | 335 (40.8%) | |
|
80 or older | 59 (7.2%) | |
|
|
||
|
Eight grade or lower | 7 (0.9%) | |
|
Some high school | 12 (1.6%) | |
|
High school graduate | 80 (10.4%) | |
|
Some college | 195 (25.4%) | |
|
Four-year college graduate | 183 (23.8%) | |
|
Greater than 4 years of college | 292 (38.0%) |
As mentioned in the Methods, the annual CAHPS scores were first analyzed with and without the rapid access clinics included. The negative responses (no/never) are shown in
Considering
Negative responses with and without the rapid access clinics.
Positive responses with and without the rapid access clinics.
It was noted earlier that the rapid access clinics only operated on Tuesday and Friday afternoons. However, the data in
Negative responses with and without the rapid access clinics on Tuesday and Friday only. N/A: not applicable.
Positive responses with and without the rapid access clinics on Tuesday and Friday only. N/A: not applicable.
We show the results relative to how well the private clinics and rapid access clinics scored on average across all CAHPS categories in
The focus measures were the measures that targeted physician communication quality, standard measures as well as additional measures based on relevance to care quality, and the potential for improvement relative to the effort for change management. All CAHPS categories were used in the first comparison, rather than selecting just those related to the focus measures, in order to conduct a general comparison of all CAHPS categories according to clinic type. In other words,
The results of the overall CAHPS scores by clinic type, as depicted in
We present the findings for focus measures in the private clinics and rapid access clinics in
Consumer Assessment of Healthcare Providers and Systems survey score by clinic type. The asterisks indicate the categories/domains relevant to the focus measures chosen for analysis. N/A: not applicable.
Focus measures for the private clinics.
Focus measures for the rapid access clinics.
We present the findings for nonstandard focus measures by clinic type in
Nonstandard focus measures by clinic type. Asterisks indicate the categories/domains relevant to the focus measures chosen for analysis.
We obtained the results of analyzing unstructured patient comments by clinic type. As listed in the Methods, the subthemes included the following: medical needs not met or questions not answered; lack of or delay with results or records; long wait or slow processing; appointment rushed or barely saw the doctor; lack of communication, direction, or follow-up; medical staff rude, impersonal, or did not listen; difficult to schedule timely appointment or appointment cancelled by the doctor; check in process frustrating, receptionist unhelpful, or lack of assistance; and general or other.
We also assessed the distribution of the comments by clinic type within the sample (ie, the number of negative rapid access comments out of the total number of comments [N] for rapid access only;
We expanded the negative comments by clinic, and we present the findings for the private clinics and rapid access clinics in
Patient comments by clinic (distributed in positive, negative, indifferent, and not applicable categories).
Patient negative comments in the private clinics and rapid access clinics.
Comments | Private clinics (N=256), n (%) | Rapid access clinics (N=86), n (%) |
Medical needs not met or questions not answered | 13 (5.1%) | 7 (8.1%) |
Lack of or delay with results/records | 15 (5.9%) | 2 (2.3%) |
Long wait or slow processing | 18 (7.0%) | 14 (16.3%) |
Appointment rushed or barely saw the doctor | 21 (8.2%) | 12 (14.0%) |
Lack of communication, direction, or follow-up | 25 (9.8%) | 11 (12.8%) |
Medical staff rude, impersonal, or did not listen | 28 (10.9%) | 15 (17.4%) |
Difficult to schedule timely appointment or appointment canceled by the doctor | 32 (12.5%) | 5 (5.8%) |
Check in process frustrating, receptionist unhelpful, or lack of assistance | 37 (14.5%) | 4 (4.7%) |
General or other | 67 (26.2%) | 16 (18.6%) |
This study assessed two distinct dermatology clinics and their individual and collective contributions to CAHPS scores for the Dermatology Department at an AMC to answer the question, “Where should the efforts of patient satisfaction improvements be focused for the two different types of dermatology clinics?” This study utilized a concurrent mixed methods approach (
The clinics consisted of private clinics (patients make appointments in advance with the dermatologist of their choosing and likely see the same dermatologist with whom they have established a relationship) and rapid access clinics (patients can get walk-in or just-in-time appointments with a dermatologist or dermatology nurse practitioner on duty).
The quantitative data analysis showed that when only private clinic data were collected, there was a slight increase in the negative response rate and a slight decrease in the positive response rate across many categories, perhaps indicating that the contribution of rapid access CAHPS scores to the overall scores may be underestimated [
To elaborate further on the qualitative data analysis, the value of patient comments cannot be underestimated as they provide straightforward and insightful feedback from patients. The comments directly express dermatology patients’ perceptions of their care while specifically indicating areas in need of improvement, as well as satisfactory aspects of the clinics. As such, consistent with the literature suggesting continued review of CAHPS scores, a frequent review of these comments could contribute to and increase the richness of aggregated data [
Both quantitative and qualitative data analyses were performed in this study with the goal of the qualitative data analysis findings enriching the quantitative data analysis findings. Integration of the quantitative and qualitative findings from this study suggests that the rapid access clinics contributed to low overall CAHPS scores, but the contribution was not radical. Qualitative analysis of patient comments elaborated the results of quantitative findings, showing that even when patients know they are in an environment of situational care (ie, a rapid access clinic), they want to spend more time with the clinician, want shorter wait times, and want people to listen. The consideration of these comments provided an additional, valuable, and actionable layer to complement the quantitative findings.
Regardless of the clinic, this study suggested that there is work to be done in regards to how staff or clerks interact with patients, as this aspect was scored low in both clinics. There is also work to be done around communication. Clinicians need to be especially conscientious to avoid speaking to their patients in a frustrated or condescending tone.
The difference between real and perceived wait times presents challenges for even the best clinic environment. A patient’s perception relative to the wait time could be altered by providing frequent updates on the wait time so as to provide patients with a more accurate perception of the time they are actually waiting, which may be a smaller window of time as compared to what they perceive without receiving any updates [
There is also work to be done around effectively managing delays in the clinic and getting to the root cause of those delays. If found that delays are caused by patients, it would be helpful to identify patients who are frequently late or who do not show up to the clinic. These particular patients could then be flagged and scheduled in the afternoon for future appointments. Scheduling such patients later in the day would minimize the interference with the clinic schedule and overall patient flow throughout the day [
Our findings show that appointments are much more difficult to get in private clinics and patients need to wait a long time to get an appointment with their established provider. This is consistent with the nationwide shortage of dermatologists [
This study had some limitations that need to be acknowledged. First, the data provided for this analysis by Press Ganey were already summarized, and this limited the statistical approaches that could be applied for further assessment. As such, analysis and result reporting were limited to descriptive statistics rather than more traditional statistical analysis to compare one dataset to another.
Another limitation was related to the survey design. The CAHPS survey does not have a not applicable (“N/A”) response available as an option for any of the questions. Subsequently, when any of the questions do not apply to patients, which was stated in patient comments, they are forced to answer “no/never” instead, artificially increasing the negative response rate for certain measures as a result.
Additionally, inconsistent response sample sizes (N) for each of the measures likely contributed to skewed reporting. For example, not every patient completed the survey, and those who completed the survey did not always answer every question available (most likely when questions did not apply). This was verified through a review of the original survey copies that were available through Press Ganey. Several of the sample surveys reviewed were only partially filled out.
Lastly, patient comments were fewer for the rapid access clinics as compared with the private clinics. The investigation of methods to increase comments represents an area of future research for researchers.
Nevertheless, this study provides valuable findings related to the degree of the contribution of individual clinics to CAHPS scores.
This study suggests that clinic type is relevant to patient satisfaction. However, the results also suggest that much of the dissatisfaction could be mitigated with better communication and level-setting expectations. There are gaps in research concerning the relationship between CAHPS scores and specific care factors. The findings and the common patterns between patient satisfaction measures and care components that were discovered through the execution of this analysis provide valuable insights into the drivers of patient satisfaction for not only the AMC’s dermatology clinics going forward, but also ambulatory facilities in general.
Future research in this area could investigate physician performance relative to clinic type. Additionally, patient satisfaction by time of the day according to clinic type may provide useful insights into patient satisfaction.
academic medical center
Consumer Assessment of Healthcare Providers and Systems
The authors would like to thank everyone in the Department of Radiology Oncology and Dermatology at the University of Alabama at Birmingham Health System, who helped to guide the focus of this study. The authors would like to deeply thank Shikha Modi, MBA, PhD candidate, for her tireless formatting of this manuscript to prepare it for publication. The authors acknowledge the detailed feedback of the reviewers, which helped make this a stronger paper.
None declared.